2024-09-12T11:52:59.635 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T11:52:59.667 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T11:52:59.779 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901698' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNd8R41kp8ODl6WO+naVSpvfMxxZ6IV/TTl9HjeL++fYzARF/fKe2P5d7pJILbBIfbqps5/FOpqaiW0sZVJYpaU= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIspw1JGmsxAsMIwlDub56+QzTXI4D5iB+0w6oQM0dF0+PSqJd8+uFjPT7cFmqwxtkJOWyp8jUzAbYZyed7geJw= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA5VbX7RymzBzi96QSyPJdwnoK+/2e3AS/ir1q9e/iGYKnZRM1iPxhMNcyU82kudOG0AGq3RRiJysWibHMJ077E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T11:52:59.780 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T11:52:59.781 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T11:52:59.781 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T11:52:59.783 INFO:teuthology.task.internal:Checking packages... 2024-09-12T11:52:59.809 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T11:52:59.810 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T11:52:59.810 INFO:teuthology.packaging:ref: None 2024-09-12T11:52:59.810 INFO:teuthology.packaging:tag: None 2024-09-12T11:52:59.810 INFO:teuthology.packaging:branch: squid 2024-09-12T11:52:59.810 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T11:52:59.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T11:53:00.043 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T11:53:00.044 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T11:53:00.050 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T11:53:00.050 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T11:53:00.057 INFO:teuthology.task.internal:Saving configuration 2024-09-12T11:53:00.078 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T11:53:00.084 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T11:53:00.118 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 11:47:45.158397', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNd8R41kp8ODl6WO+naVSpvfMxxZ6IV/TTl9HjeL++fYzARF/fKe2P5d7pJILbBIfbqps5/FOpqaiW0sZVJYpaU='} 2024-09-12T11:53:00.151 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 11:47:45.159717', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIspw1JGmsxAsMIwlDub56+QzTXI4D5iB+0w6oQM0dF0+PSqJd8+uFjPT7cFmqwxtkJOWyp8jUzAbYZyed7geJw='} 2024-09-12T11:53:00.183 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 11:47:45.156842', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA5VbX7RymzBzi96QSyPJdwnoK+/2e3AS/ir1q9e/iGYKnZRM1iPxhMNcyU82kudOG0AGq3RRiJysWibHMJ077E='} 2024-09-12T11:53:00.183 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T11:53:00.190 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T11:53:00.190 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T11:53:00.190 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T11:53:00.190 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T11:53:00.304 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9e7e3fbbe0>, signals=[15]) 2024-09-12T11:53:00.304 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T11:53:00.314 INFO:teuthology.task.internal:Opening connections... 2024-09-12T11:53:00.314 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T11:53:00.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T11:53:00.404 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-09-12T11:53:00.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T11:53:00.493 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-12T11:53:00.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T11:53:00.577 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T11:53:00.583 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-09-12T11:53:00.601 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-09-12T11:53:00.601 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-09-12T11:53:00.659 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-09-12T11:53:00.660 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T11:53:00.661 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T11:53:00.661 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T11:53:00.661 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-09-12T11:53:00.689 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-09-12T11:53:00.707 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-09-12T11:53:00.707 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-09-12T11:53:00.766 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-09-12T11:53:00.766 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-09-12T11:53:00.766 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-09-12T11:53:00.766 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-09-12T11:53:00.766 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-09-12T11:53:00.766 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-09-12T11:53:00.766 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T11:53:00.767 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-09-12T11:53:00.767 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-09-12T11:53:00.767 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T11:53:00.767 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-09-12T11:53:00.767 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T11:53:00.767 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T11:53:00.767 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T11:53:00.767 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-09-12T11:53:00.797 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-12T11:53:00.816 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-12T11:53:00.817 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-12T11:53:00.875 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-09-12T11:53:00.875 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-09-12T11:53:00.875 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-09-12T11:53:00.875 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T11:53:00.876 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T11:53:00.877 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-12T11:53:00.905 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T11:53:00.916 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T11:53:00.921 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T11:53:00.921 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-09-12T11:53:00.923 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-09-12T11:53:00.926 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-12T11:53:00.942 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T11:53:00.948 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T11:53:00.948 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-09-12T11:53:00.981 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-09-12T11:53:00.984 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-12T11:53:01.099 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T11:53:01.203 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T11:53:01.217 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T11:53:01.217 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T11:53:01.217 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T11:53:01.218 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T11:53:01.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T11:53:01.235 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-09-12T11:53:01.294 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-503.el9.x86_64 2024-09-12T11:53:01.294 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.14.0-503.el9.x86_64 2024-09-12T11:53:01.294 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum install -y kernel 2024-09-12T11:53:04.683 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-12T11:53:07.051 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-12T11:53:11.518 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-12T11:53:12.994 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 19 kB 00:00 2024-09-12T11:53:13.748 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-12T11:53:19.171 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-12T11:53:21.168 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T11:53:21.169 INFO:teuthology.orchestra.run.smithi137.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T11:53:21.271 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-09-12T11:53:21.276 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-09-12T11:53:21.276 INFO:teuthology.orchestra.run.smithi137.stdout: Package Architecture Version Repository Size 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:Install 4 Packages 2024-09-12T11:53:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T11:53:21.278 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 79 M 2024-09-12T11:53:21.278 INFO:teuthology.orchestra.run.smithi137.stdout:Installed size: 126 M 2024-09-12T11:53:21.278 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-09-12T11:53:21.488 INFO:teuthology.orchestra.run.smithi137.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 256 kB/s | 23 kB 00:00 2024-09-12T11:53:22.939 INFO:teuthology.orchestra.run.smithi137.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-12T11:53:23.522 INFO:teuthology.orchestra.run.smithi137.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-12T11:53:24.306 INFO:teuthology.orchestra.run.smithi137.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 10 MB/s | 29 MB 00:02 2024-09-12T11:53:24.306 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-09-12T11:53:24.306 INFO:teuthology.orchestra.run.smithi137.stdout:Total 26 MB/s | 79 MB 00:03 2024-09-12T11:53:24.989 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-09-12T11:53:25.336 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-09-12T11:53:25.337 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-09-12T11:53:26.328 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-09-12T11:53:26.329 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-09-12T11:53:28.803 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-09-12T11:53:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T11:53:29.598 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T11:53:32.391 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T11:53:32.516 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T11:53:37.320 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T11:53:37.358 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T11:53:42.207 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T11:54:20.269 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T11:54:20.308 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T11:54:20.673 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T11:54:20.673 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T11:54:20.673 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T11:54:20.673 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T11:54:21.524 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T11:54:21.525 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-09-12T11:54:21.907 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T11:54:21.986 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T11:54:21.986 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T11:54:21.987 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T11:54:21.987 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T11:54:21.987 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T11:54:21.987 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T11:54:21.987 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T11:54:21.987 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T11:54:22.045 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T11:54:22.045 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T11:54:22.046 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-09-12T11:54:22.077 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.06 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 82.el9 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Group : Unspecified 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 8274670 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T11:54:22.078 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T11:54:22.079 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T11:54:22.079 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : builder@centos.org 2024-09-12T11:54:22.079 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-09-12T11:54:22.079 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T11:54:22.079 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-09-12T11:54:22.079 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-09-12T11:54:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T11:54:22.080 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T11:54:22.080 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T11:54:22.080 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T11:54:22.080 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-09-12T11:54:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T11:54:22.081 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T11:54:22.081 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T11:54:22.082 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 5.14.0-505.el9.x86_64 2024-09-12T11:54:22.082 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T11:54:22.752 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-09-12T11:54:24.093 INFO:teuthology.orchestra.run.smithi137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T11:54:24.141 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-09-12T11:54:24.144 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-09-12T11:54:24.159 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.fH3Lmw1f44 2024-09-12T11:54:24.159 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fH3Lmw1f44 2024-09-12T11:54:24.226 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.fH3Lmw1f44 2024-09-12T11:54:24.373 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.fH3Lmw1f44 is 6KB 2024-09-12T11:54:24.421 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.fH3Lmw1f44 2024-09-12T11:54:24.436 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T11:54:24.499 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T11:54:24.499 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T11:54:24.499 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T11:54:24.499 INFO:teuthology.orchestra.run.smithi137.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T11:54:24.500 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T11:54:24.651 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-09-12T11:54:54.654 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T11:54:54.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-12T11:54:54.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T11:55:29.213 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-09-12T11:55:38.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-12T11:55:38.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T11:55:56.609 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-09-12T11:56:08.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-12T11:56:08.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T11:56:08.959 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-09-12T11:56:09.306 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-12T11:56:09.306 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T11:56:09.306 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-09-12T11:56:09.320 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-505.el9.x86_64 2024-09-12T11:56:09.321 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T11:56:09.321 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T11:56:09.321 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T11:56:10.321 DEBUG:teuthology.parallel:result is None 2024-09-12T11:56:10.322 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T11:56:10.329 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T11:56:10.329 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T11:56:10.332 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T11:56:10.335 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T11:56:10.354 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T11:56:10.359 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T11:56:10.365 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T11:56:10.365 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T11:56:10.384 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T11:56:10.393 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T11:56:10.570 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T11:56:10.576 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T11:56:10.576 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T11:56:10.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T11:56:10.593 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T11:56:10.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T11:56:10.611 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T11:56:10.614 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T11:56:10.636 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T11:56:10.671 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T11:56:10.698 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T11:56:10.706 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T11:56:10.719 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T11:56:10.727 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T11:56:10.756 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T11:56:10.758 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T11:56:10.764 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T11:56:10.764 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T11:56:10.767 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T11:56:10.769 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T11:56:10.796 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T11:56:10.805 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T11:56:10.805 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T11:56:10.829 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T11:56:10.838 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T11:56:10.854 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T11:56:10.934 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T11:56:10.967 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T11:56:11.053 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T11:56:11.085 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T11:56:11.085 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T11:56:11.146 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T11:56:11.198 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T11:56:11.242 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T11:56:11.333 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T11:56:11.372 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T11:56:11.373 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T11:56:11.441 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T11:56:11.495 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T11:56:11.545 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T11:56:11.640 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T11:56:11.675 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T11:56:11.675 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T11:56:11.743 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-09-12T11:56:11.749 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-09-12T11:56:11.751 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-12T11:56:11.784 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T11:56:11.795 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T11:56:11.822 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T11:56:12.239 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T11:56:12.247 INFO:teuthology.task.internal:Starting timer... 2024-09-12T11:56:12.247 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T11:56:12.257 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T11:56:12.265 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T11:56:12.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-09-12T11:56:12.368 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-09-12T11:56:12.370 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-09-12T11:56:12.437 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-09-12T11:56:12.439 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-09-12T11:56:12.503 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-09-12T11:56:12.505 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T11:56:12.505 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-09-12T11:56:12.551 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-09-12T11:56:12.552 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-09-12T11:56:12.574 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-09-12T11:56:12.575 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-09-12T11:56:12.604 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-09-12T11:56:12.604 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-09-12T11:56:12.604 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T11:56:12.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T11:56:12.670 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T11:56:12.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T11:56:12.701 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T11:56:12.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T11:56:12.730 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T11:56:12.730 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-09-12T11:56:12.799 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-09-12T11:56:12.831 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-09-12T11:56:12.881 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T11:56:12.891 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T11:56:12.902 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T11:56:12.903 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi137.front.sepia.ceph.com,smithi171.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T12:00:21.109 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-12T12:00:21.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-12T12:00:21.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:00:21.214 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-09-12T12:00:21.301 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-09-12T12:00:21.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-12T12:00:21.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:00:21.385 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-12T12:00:21.464 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-12T12:00:21.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-12T12:00:21.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T12:00:21.549 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-12T12:00:21.629 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-12T12:00:21.629 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T12:00:21.640 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T12:00:21.641 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T12:00:21.641 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:00:21.644 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T12:00:21.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:00:21.647 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T12:00:21.647 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:00:21.675 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T12:00:21.676 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T12:00:21.690 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T12:00:21.691 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T12:00:21.700 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T12:00:21.715 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T12:00:21.716 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-09-12T12:00:21.716 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-09-12T12:00:21.727 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-09-12T12:00:21.729 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-09-12T12:00:21.741 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-09-12T12:00:21.742 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T12:00:21.744 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T12:00:21.752 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-09-12T12:00:21.756 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T12:00:21.758 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T12:00:21.767 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T12:00:21.781 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T12:00:21.810 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:00:21.812 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:00:21.814 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:00:21.814 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-12T12:00:21.814 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.815 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.815 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.815 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.816 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:00:21.816 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-09-12T12:00:21.817 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.817 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.817 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.817 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.838 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:00:21.842 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:00:21.842 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-12T12:00:21.842 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.843 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.843 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.843 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:00:21.843 INFO:teuthology.run_tasks:Running task install... 2024-09-12T12:00:21.852 DEBUG:teuthology.task.install:project ceph 2024-09-12T12:00:21.852 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T12:00:21.852 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T12:00:21.852 INFO:teuthology.task.install:Using flavor: default 2024-09-12T12:00:21.858 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T12:00:21.858 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T12:00:21.859 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T12:00:21.859 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:00:21.861 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T12:00:21.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:00:21.863 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T12:00:21.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:00:22.009 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T12:00:22.009 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T12:00:22.025 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T12:00:22.025 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T12:00:22.026 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T12:00:22.026 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T12:00:22.132 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T12:00:22.132 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:00:22.133 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T12:00:22.142 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T12:00:22.143 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:00:22.143 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T12:00:22.153 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T12:00:22.153 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:00:22.153 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T12:00:22.159 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T12:00:22.159 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T12:00:22.168 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T12:00:22.168 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T12:00:22.179 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T12:00:22.179 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T12:00:22.225 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T12:00:22.235 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T12:00:22.246 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T12:00:22.299 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T12:00:22.313 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T12:00:22.321 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T12:00:22.363 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-09-12T12:00:22.365 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-09-12T12:00:22.378 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-09-12T12:00:22.379 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-09-12T12:00:22.384 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-09-12T12:00:22.386 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-09-12T12:00:22.605 INFO:teuthology.orchestra.run.smithi189.stdout:45 files removed 2024-09-12T12:00:22.609 INFO:teuthology.orchestra.run.smithi137.stdout:45 files removed 2024-09-12T12:00:22.610 INFO:teuthology.orchestra.run.smithi171.stdout:45 files removed 2024-09-12T12:00:22.632 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T12:00:22.636 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T12:00:22.637 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T12:00:23.185 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 309 kB/s | 84 kB 00:00 2024-09-12T12:00:23.187 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 302 kB/s | 84 kB 00:00 2024-09-12T12:00:23.202 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-12T12:00:23.469 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T12:00:23.471 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T12:00:23.485 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T12:00:23.745 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T12:00:23.748 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T12:00:23.761 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T12:00:25.761 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-12T12:00:26.462 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-09-12T12:00:27.547 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.2 MB 00:03 2024-09-12T12:00:28.854 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-12T12:00:30.348 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-12T12:00:34.865 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-12T12:00:35.438 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.5 MB 00:02 2024-09-12T12:00:36.063 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 20 MB 00:08 2024-09-12T12:00:36.349 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-12T12:00:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 19 kB 00:00 2024-09-12T12:00:37.487 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-12T12:00:38.076 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-12T12:00:41.906 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-12T12:00:43.076 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-12T12:00:43.365 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-12T12:00:43.565 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T12:00:44.194 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-12T12:00:45.346 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:00:45.347 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:00:45.360 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T12:00:45.361 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T12:00:45.362 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T12:00:45.363 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T12:00:45.365 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T12:00:45.368 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T12:00:45.369 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T12:00:45.433 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T12:00:45.448 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T12:00:45.449 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T12:00:45.450 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T12:00:45.451 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T12:00:45.452 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T12:00:45.453 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T12:00:45.454 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T12:00:45.455 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T12:00:45.456 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T12:00:45.457 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T12:00:45.458 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T12:00:45.459 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T12:00:45.460 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout:====================================================================================== 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout:Install 148 Packages 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 8 Packages 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:00:45.461 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 226 M 2024-09-12T12:00:45.462 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-12T12:00:45.885 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:00:45.886 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:00:45.899 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T12:00:45.900 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T12:00:45.901 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T12:00:45.902 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T12:00:45.904 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T12:00:45.906 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T12:00:45.907 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T12:00:45.971 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-09-12T12:00:46.031 INFO:teuthology.orchestra.run.smithi137.stdout:====================================================================================== 2024-09-12T12:00:46.031 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-09-12T12:00:46.031 INFO:teuthology.orchestra.run.smithi137.stdout:====================================================================================== 2024-09-12T12:00:46.031 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-09-12T12:00:46.031 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T12:00:46.032 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T12:00:46.033 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T12:00:46.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T12:00:46.035 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T12:00:46.036 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T12:00:46.037 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T12:00:46.038 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T12:00:46.039 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T12:00:46.040 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T12:00:46.041 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T12:00:46.042 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T12:00:46.043 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T12:00:46.044 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:00:46.045 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-09-12T12:00:46.045 INFO:teuthology.orchestra.run.smithi137.stdout:====================================================================================== 2024-09-12T12:00:46.045 INFO:teuthology.orchestra.run.smithi137.stdout:Install 148 Packages 2024-09-12T12:00:46.045 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 8 Packages 2024-09-12T12:00:46.045 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:00:46.045 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 226 M 2024-09-12T12:00:46.045 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-09-12T12:00:46.127 INFO:teuthology.orchestra.run.smithi189.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-12T12:00:46.394 INFO:teuthology.orchestra.run.smithi189.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.3 MB/s | 897 kB 00:00 2024-09-12T12:00:46.494 INFO:teuthology.orchestra.run.smithi189.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-12T12:00:46.520 INFO:teuthology.orchestra.run.smithi189.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.1 MB/s | 145 kB 00:00 2024-09-12T12:00:46.533 INFO:teuthology.orchestra.run.smithi137.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 34 kB/s | 6.5 kB 00:00 2024-09-12T12:00:46.687 INFO:teuthology.orchestra.run.smithi189.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-12T12:00:47.012 INFO:teuthology.orchestra.run.smithi189.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-12T12:00:47.075 INFO:teuthology.orchestra.run.smithi137.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 897 kB 00:00 2024-09-12T12:00:47.175 INFO:teuthology.orchestra.run.smithi137.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-12T12:00:47.295 INFO:teuthology.orchestra.run.smithi189.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-12T12:00:47.337 INFO:teuthology.orchestra.run.smithi189.stdout:(8/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 1.9 MB/s | 1.5 MB 00:00 2024-09-12T12:00:47.404 INFO:teuthology.orchestra.run.smithi189.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 379 kB/s | 25 kB 00:00 2024-09-12T12:00:48.016 INFO:teuthology.orchestra.run.smithi137.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 2.8 MB/s | 2.4 MB 00:00 2024-09-12T12:00:48.421 INFO:teuthology.orchestra.run.smithi189.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 17 MB 00:01 2024-09-12T12:00:48.475 INFO:teuthology.orchestra.run.smithi137.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 2.5 MB/s | 5.4 MB 00:02 2024-09-12T12:00:48.530 INFO:teuthology.orchestra.run.smithi189.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 8.6 MB/s | 11 MB 00:01 2024-09-12T12:00:48.555 INFO:teuthology.orchestra.run.smithi189.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.7 MB/s | 228 kB 00:00 2024-09-12T12:00:48.567 INFO:teuthology.orchestra.run.smithi137.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 2.8 MB/s | 1.5 MB 00:00 2024-09-12T12:00:48.597 INFO:teuthology.orchestra.run.smithi189.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 406 kB/s | 27 kB 00:00 2024-09-12T12:00:48.630 INFO:teuthology.orchestra.run.smithi189.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 422 kB/s | 32 kB 00:00 2024-09-12T12:00:48.722 INFO:teuthology.orchestra.run.smithi189.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.7 MB/s | 163 kB 00:00 2024-09-12T12:00:48.756 INFO:teuthology.orchestra.run.smithi189.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.6 MB/s | 752 kB 00:00 2024-09-12T12:00:48.806 INFO:teuthology.orchestra.run.smithi189.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.5 MB/s | 127 kB 00:00 2024-09-12T12:00:48.848 INFO:teuthology.orchestra.run.smithi189.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.4 MB/s | 506 kB 00:00 2024-09-12T12:00:48.948 INFO:teuthology.orchestra.run.smithi189.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 451 kB/s | 45 kB 00:00 2024-09-12T12:00:49.049 INFO:teuthology.orchestra.run.smithi189.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.3 MB/s | 136 kB 00:00 2024-09-12T12:00:49.336 INFO:teuthology.orchestra.run.smithi137.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 5.5 MB/s | 4.7 MB 00:00 2024-09-12T12:00:49.399 INFO:teuthology.orchestra.run.smithi189.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 9.0 MB/s | 5.3 MB 00:00 2024-09-12T12:00:49.474 INFO:teuthology.orchestra.run.smithi189.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 381 kB/s | 162 kB 00:00 2024-09-12T12:00:49.499 INFO:teuthology.orchestra.run.smithi189.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.1 MB/s | 321 kB 00:00 2024-09-12T12:00:49.558 INFO:teuthology.orchestra.run.smithi189.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.5 MB/s | 301 kB 00:00 2024-09-12T12:00:49.583 INFO:teuthology.orchestra.run.smithi189.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-12T12:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 928 kB/s | 85 kB 00:00 2024-09-12T12:00:49.750 INFO:teuthology.orchestra.run.smithi189.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-12T12:00:49.842 INFO:teuthology.orchestra.run.smithi189.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 348 kB/s | 32 kB 00:00 2024-09-12T12:00:49.867 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 7.7 kB/s | 1.7 kB 00:00 2024-09-12T12:00:49.926 INFO:teuthology.orchestra.run.smithi189.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.7 MB/s | 146 kB 00:00 2024-09-12T12:00:50.018 INFO:teuthology.orchestra.run.smithi189.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 7.1 MB/s | 3.1 MB 00:00 2024-09-12T12:00:50.627 INFO:teuthology.orchestra.run.smithi189.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.1 MB/s | 3.6 MB 00:00 2024-09-12T12:00:50.677 INFO:teuthology.orchestra.run.smithi189.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T12:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 9.7 MB/s | 7.4 MB 00:00 2024-09-12T12:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 394 kB/s | 49 kB 00:00 2024-09-12T12:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 394 kB/s | 16 kB 00:00 2024-09-12T12:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-12T12:00:50.895 INFO:teuthology.orchestra.run.smithi137.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 6.8 MB/s | 11 MB 00:01 2024-09-12T12:00:50.911 INFO:teuthology.orchestra.run.smithi189.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-12T12:00:50.936 INFO:teuthology.orchestra.run.smithi137.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 608 kB/s | 25 kB 00:00 2024-09-12T12:00:50.937 INFO:teuthology.orchestra.run.smithi189.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 219 kB/s | 18 kB 00:00 2024-09-12T12:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-12T12:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 637 kB/s | 191 kB 00:00 2024-09-12T12:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 952 kB/s | 801 kB 00:00 2024-09-12T12:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-12T12:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 914 kB/s | 106 kB 00:00 2024-09-12T12:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-09-12T12:00:52.129 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:00:52.130 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T12:00:52.143 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T12:00:52.144 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T12:00:52.145 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T12:00:52.146 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T12:00:52.148 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T12:00:52.151 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T12:00:52.152 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T12:00:52.155 INFO:teuthology.orchestra.run.smithi189.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-12T12:00:52.213 INFO:teuthology.orchestra.run.smithi189.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-12T12:00:52.215 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-12T12:00:52.224 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T12:00:52.225 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T12:00:52.226 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T12:00:52.227 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T12:00:52.228 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T12:00:52.229 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T12:00:52.230 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T12:00:52.231 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T12:00:52.232 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T12:00:52.233 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T12:00:52.234 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T12:00:52.235 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T12:00:52.236 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-09-12T12:00:52.237 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T12:00:52.237 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:00:52.237 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-12T12:00:52.237 INFO:teuthology.orchestra.run.smithi171.stdout:====================================================================================== 2024-09-12T12:00:52.237 INFO:teuthology.orchestra.run.smithi171.stdout:Install 148 Packages 2024-09-12T12:00:52.237 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 8 Packages 2024-09-12T12:00:52.237 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:00:52.238 INFO:teuthology.orchestra.run.smithi137.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 4.6 MB/s | 17 MB 00:03 2024-09-12T12:00:52.238 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 226 M 2024-09-12T12:00:52.238 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-12T12:00:52.280 INFO:teuthology.orchestra.run.smithi189.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-12T12:00:52.295 INFO:teuthology.orchestra.run.smithi137.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-12T12:00:52.345 INFO:teuthology.orchestra.run.smithi137.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-12T12:00:52.396 INFO:teuthology.orchestra.run.smithi137.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 636 kB/s | 32 kB 00:00 2024-09-12T12:00:52.422 INFO:teuthology.orchestra.run.smithi189.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-12T12:00:52.464 INFO:teuthology.orchestra.run.smithi189.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-12T12:00:52.506 INFO:teuthology.orchestra.run.smithi189.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-12T12:00:52.513 INFO:teuthology.orchestra.run.smithi137.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 6.3 MB/s | 752 kB 00:00 2024-09-12T12:00:52.539 INFO:teuthology.orchestra.run.smithi189.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-09-12T12:00:52.563 INFO:teuthology.orchestra.run.smithi137.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-12T12:00:52.565 INFO:teuthology.orchestra.run.smithi189.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-12T12:00:52.606 INFO:teuthology.orchestra.run.smithi189.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-09-12T12:00:52.621 INFO:teuthology.orchestra.run.smithi137.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-12T12:00:52.649 INFO:teuthology.orchestra.run.smithi189.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-12T12:00:52.707 INFO:teuthology.orchestra.run.smithi189.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-09-12T12:00:52.713 INFO:teuthology.orchestra.run.smithi137.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.4 MB/s | 506 kB 00:00 2024-09-12T12:00:52.732 INFO:teuthology.orchestra.run.smithi189.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-12T12:00:52.758 INFO:teuthology.orchestra.run.smithi189.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T12:00:52.783 INFO:teuthology.orchestra.run.smithi189.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-12T12:00:52.817 INFO:teuthology.orchestra.run.smithi189.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-12T12:00:52.850 INFO:teuthology.orchestra.run.smithi189.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-12T12:00:52.853 INFO:teuthology.orchestra.run.smithi171.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-12T12:00:52.875 INFO:teuthology.orchestra.run.smithi189.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-12T12:00:52.901 INFO:teuthology.orchestra.run.smithi189.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-12T12:00:52.926 INFO:teuthology.orchestra.run.smithi189.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-12T12:00:52.951 INFO:teuthology.orchestra.run.smithi189.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-12T12:00:52.977 INFO:teuthology.orchestra.run.smithi189.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-12T12:00:53.085 INFO:teuthology.orchestra.run.smithi189.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T12:00:53.120 INFO:teuthology.orchestra.run.smithi171.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.3 MB/s | 897 kB 00:00 2024-09-12T12:00:53.152 INFO:teuthology.orchestra.run.smithi189.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 25 MB/s | 4.9 MB 00:00 2024-09-12T12:00:53.178 INFO:teuthology.orchestra.run.smithi171.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.4 MB/s | 145 kB 00:00 2024-09-12T12:00:53.194 INFO:teuthology.orchestra.run.smithi189.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-12T12:00:53.228 INFO:teuthology.orchestra.run.smithi189.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-12T12:00:53.253 INFO:teuthology.orchestra.run.smithi189.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-12T12:00:53.312 INFO:teuthology.orchestra.run.smithi189.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-12T12:00:53.387 INFO:teuthology.orchestra.run.smithi189.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-12T12:00:53.412 INFO:teuthology.orchestra.run.smithi189.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-09-12T12:00:53.462 INFO:teuthology.orchestra.run.smithi171.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 7.3 MB/s | 5.4 MB 00:00 2024-09-12T12:00:53.463 INFO:teuthology.orchestra.run.smithi189.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-12T12:00:53.480 INFO:teuthology.orchestra.run.smithi137.stdout:(18/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 3.0 MB/s | 21 MB 00:07 2024-09-12T12:00:53.513 INFO:teuthology.orchestra.run.smithi171.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 7.1 MB/s | 2.4 MB 00:00 2024-09-12T12:00:53.539 INFO:teuthology.orchestra.run.smithi137.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 775 kB/s | 45 kB 00:00 2024-09-12T12:00:53.546 INFO:teuthology.orchestra.run.smithi189.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-09-12T12:00:53.563 INFO:teuthology.orchestra.run.smithi189.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-12T12:00:53.597 INFO:teuthology.orchestra.run.smithi189.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-12T12:00:53.622 INFO:teuthology.orchestra.run.smithi189.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-09-12T12:00:53.623 INFO:teuthology.orchestra.run.smithi137.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-12T12:00:53.638 INFO:teuthology.orchestra.run.smithi171.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-12T12:00:53.647 INFO:teuthology.orchestra.run.smithi189.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-12T12:00:53.672 INFO:teuthology.orchestra.run.smithi189.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-12T12:00:53.698 INFO:teuthology.orchestra.run.smithi137.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 5.4 MB/s | 5.3 MB 00:00 2024-09-12T12:00:53.698 INFO:teuthology.orchestra.run.smithi189.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-12T12:00:53.723 INFO:teuthology.orchestra.run.smithi137.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 162 kB 00:00 2024-09-12T12:00:53.748 INFO:teuthology.orchestra.run.smithi189.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-12T12:00:53.840 INFO:teuthology.orchestra.run.smithi189.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-09-12T12:00:53.865 INFO:teuthology.orchestra.run.smithi189.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-12T12:00:53.890 INFO:teuthology.orchestra.run.smithi189.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T12:00:53.916 INFO:teuthology.orchestra.run.smithi189.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-12T12:00:53.941 INFO:teuthology.orchestra.run.smithi189.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-12T12:00:53.966 INFO:teuthology.orchestra.run.smithi189.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T12:00:53.992 INFO:teuthology.orchestra.run.smithi189.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T12:00:54.017 INFO:teuthology.orchestra.run.smithi189.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T12:00:54.042 INFO:teuthology.orchestra.run.smithi189.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T12:00:54.084 INFO:teuthology.orchestra.run.smithi189.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-12T12:00:54.134 INFO:teuthology.orchestra.run.smithi189.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 888 kB/s | 44 kB 00:00 2024-09-12T12:00:54.140 INFO:teuthology.orchestra.run.smithi137.stdout:(23/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 15 MB/s | 49 MB 00:03 2024-09-12T12:00:54.174 INFO:teuthology.orchestra.run.smithi137.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 669 kB/s | 301 kB 00:00 2024-09-12T12:00:54.185 INFO:teuthology.orchestra.run.smithi189.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T12:00:54.199 INFO:teuthology.orchestra.run.smithi137.stdout:(25/156): python3-rados-19.1.1-298.gb53b6d3e.el 641 kB/s | 321 kB 00:00 2024-09-12T12:00:54.213 INFO:teuthology.orchestra.run.smithi171.stdout:(7/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 21 MB 00:01 2024-09-12T12:00:54.224 INFO:teuthology.orchestra.run.smithi137.stdout:(26/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-12T12:00:54.227 INFO:teuthology.orchestra.run.smithi189.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 867 kB/s | 36 kB 00:00 2024-09-12T12:00:54.249 INFO:teuthology.orchestra.run.smithi137.stdout:(27/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-12T12:00:54.275 INFO:teuthology.orchestra.run.smithi137.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T12:00:54.280 INFO:teuthology.orchestra.run.smithi171.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 6.1 MB/s | 4.7 MB 00:00 2024-09-12T12:00:54.292 INFO:teuthology.orchestra.run.smithi137.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 762 kB/s | 32 kB 00:00 2024-09-12T12:00:54.326 INFO:teuthology.orchestra.run.smithi137.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T12:00:54.339 INFO:teuthology.orchestra.run.smithi171.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 432 kB/s | 25 kB 00:00 2024-09-12T12:00:54.385 INFO:teuthology.orchestra.run.smithi189.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-09-12T12:00:54.411 INFO:teuthology.orchestra.run.smithi189.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-12T12:00:54.436 INFO:teuthology.orchestra.run.smithi189.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 6.0 MB/s | 304 kB 00:00 2024-09-12T12:00:54.461 INFO:teuthology.orchestra.run.smithi189.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T12:00:54.487 INFO:teuthology.orchestra.run.smithi189.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T12:00:54.512 INFO:teuthology.orchestra.run.smithi189.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T12:00:54.537 INFO:teuthology.orchestra.run.smithi189.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-09-12T12:00:54.562 INFO:teuthology.orchestra.run.smithi189.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-09-12T12:00:54.605 INFO:teuthology.orchestra.run.smithi189.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-12T12:00:54.646 INFO:teuthology.orchestra.run.smithi189.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-12T12:00:54.671 INFO:teuthology.orchestra.run.smithi189.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 838 kB/s | 19 kB 00:00 2024-09-12T12:00:54.705 INFO:teuthology.orchestra.run.smithi189.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 3.1 MB/s | 308 kB 00:00 2024-09-12T12:00:54.730 INFO:teuthology.orchestra.run.smithi189.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T12:00:54.756 INFO:teuthology.orchestra.run.smithi189.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T12:00:54.781 INFO:teuthology.orchestra.run.smithi189.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-12T12:00:54.815 INFO:teuthology.orchestra.run.smithi189.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-12T12:00:54.856 INFO:teuthology.orchestra.run.smithi171.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 17 MB 00:01 2024-09-12T12:00:54.856 INFO:teuthology.orchestra.run.smithi189.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-12T12:00:54.890 INFO:teuthology.orchestra.run.smithi189.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-12T12:00:54.900 INFO:teuthology.orchestra.run.smithi137.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 4.4 MB/s | 3.1 MB 00:00 2024-09-12T12:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-12T12:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-12T12:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 856 kB/s | 14 kB 00:00 2024-09-12T12:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 10 MB/s | 172 kB 00:00 2024-09-12T12:00:54.967 INFO:teuthology.orchestra.run.smithi137.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-12T12:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-12T12:00:54.998 INFO:teuthology.orchestra.run.smithi171.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 14 MB/s | 11 MB 00:00 2024-09-12T12:00:55.009 INFO:teuthology.orchestra.run.smithi137.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 1.2 MB/s | 49 kB 00:00 2024-09-12T12:00:55.023 INFO:teuthology.orchestra.run.smithi171.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.3 MB/s | 228 kB 00:00 2024-09-12T12:00:55.025 INFO:teuthology.orchestra.run.smithi189.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-12T12:00:55.049 INFO:teuthology.orchestra.run.smithi171.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 541 kB/s | 27 kB 00:00 2024-09-12T12:00:55.050 INFO:teuthology.orchestra.run.smithi189.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-12T12:00:55.074 INFO:teuthology.orchestra.run.smithi171.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 628 kB/s | 32 kB 00:00 2024-09-12T12:00:55.093 INFO:teuthology.orchestra.run.smithi137.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 9.7 MB/s | 7.4 MB 00:00 2024-09-12T12:00:55.109 INFO:teuthology.orchestra.run.smithi189.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-12T12:00:55.118 INFO:teuthology.orchestra.run.smithi137.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 151 kB/s | 16 kB 00:00 2024-09-12T12:00:55.133 INFO:teuthology.orchestra.run.smithi171.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-12T12:00:55.134 INFO:teuthology.orchestra.run.smithi189.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-12T12:00:55.144 INFO:teuthology.orchestra.run.smithi137.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.5 MB/s | 285 kB 00:00 2024-09-12T12:00:55.158 INFO:teuthology.orchestra.run.smithi171.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-12T12:00:55.159 INFO:teuthology.orchestra.run.smithi189.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-12T12:00:55.185 INFO:teuthology.orchestra.run.smithi189.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 702 kB/s | 18 kB 00:00 2024-09-12T12:00:55.192 INFO:teuthology.orchestra.run.smithi171.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-12T12:00:55.202 INFO:teuthology.orchestra.run.smithi137.stdout:(37/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 3.9 MB/s | 3.6 MB 00:00 2024-09-12T12:00:55.210 INFO:teuthology.orchestra.run.smithi189.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 934 kB/s | 23 kB 00:00 2024-09-12T12:00:55.225 INFO:teuthology.orchestra.run.smithi171.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-12T12:00:55.235 INFO:teuthology.orchestra.run.smithi189.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-12T12:00:55.261 INFO:teuthology.orchestra.run.smithi189.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-12T12:00:55.261 INFO:teuthology.orchestra.run.smithi137.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 156 kB/s | 18 kB 00:00 2024-09-12T12:00:55.301 INFO:teuthology.orchestra.run.smithi171.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 600 kB/s | 45 kB 00:00 2024-09-12T12:00:55.311 INFO:teuthology.orchestra.run.smithi189.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-09-12T12:00:55.337 INFO:teuthology.orchestra.run.smithi189.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T12:00:55.345 INFO:teuthology.orchestra.run.smithi137.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.3 MB/s | 765 kB 00:00 2024-09-12T12:00:55.361 INFO:teuthology.orchestra.run.smithi189.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T12:00:55.368 INFO:teuthology.orchestra.run.smithi171.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T12:00:55.370 INFO:teuthology.orchestra.run.smithi137.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-12T12:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-09-12T12:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-09-12T12:00:55.444 INFO:teuthology.orchestra.run.smithi171.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 162 kB 00:00 2024-09-12T12:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-12T12:00:55.462 INFO:teuthology.orchestra.run.smithi137.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-12T12:00:55.479 INFO:teuthology.orchestra.run.smithi189.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T12:00:55.487 INFO:teuthology.orchestra.run.smithi137.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-12T12:00:55.505 INFO:teuthology.orchestra.run.smithi189.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T12:00:55.512 INFO:teuthology.orchestra.run.smithi137.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-12T12:00:55.527 INFO:teuthology.orchestra.run.smithi171.stdout:(22/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-12T12:00:55.530 INFO:teuthology.orchestra.run.smithi189.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-12T12:00:55.538 INFO:teuthology.orchestra.run.smithi137.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-12T12:00:55.555 INFO:teuthology.orchestra.run.smithi189.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-12T12:00:55.563 INFO:teuthology.orchestra.run.smithi137.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T12:00:55.569 INFO:teuthology.orchestra.run.smithi171.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.5 MB/s | 321 kB 00:00 2024-09-12T12:00:55.580 INFO:teuthology.orchestra.run.smithi189.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T12:00:55.588 INFO:teuthology.orchestra.run.smithi137.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T12:00:55.595 INFO:teuthology.orchestra.run.smithi171.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-12T12:00:55.606 INFO:teuthology.orchestra.run.smithi189.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-12T12:00:55.614 INFO:teuthology.orchestra.run.smithi137.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T12:00:55.620 INFO:teuthology.orchestra.run.smithi171.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T12:00:55.631 INFO:teuthology.orchestra.run.smithi189.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-12T12:00:55.646 INFO:teuthology.orchestra.run.smithi171.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 85 kB 00:00 2024-09-12T12:00:55.656 INFO:teuthology.orchestra.run.smithi189.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-12T12:00:55.682 INFO:teuthology.orchestra.run.smithi189.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T12:00:55.708 INFO:teuthology.orchestra.run.smithi189.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T12:00:55.721 INFO:teuthology.orchestra.run.smithi171.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.2 MB/s | 171 kB 00:00 2024-09-12T12:00:55.732 INFO:teuthology.orchestra.run.smithi189.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-12T12:00:55.757 INFO:teuthology.orchestra.run.smithi189.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-12T12:00:55.780 INFO:teuthology.orchestra.run.smithi171.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 19 MB/s | 3.1 MB 00:00 2024-09-12T12:00:55.782 INFO:teuthology.orchestra.run.smithi189.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-09-12T12:00:55.805 INFO:teuthology.orchestra.run.smithi171.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 381 kB/s | 32 kB 00:00 2024-09-12T12:00:55.833 INFO:teuthology.orchestra.run.smithi189.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T12:00:55.839 INFO:teuthology.orchestra.run.smithi171.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.4 MB/s | 146 kB 00:00 2024-09-12T12:00:55.864 INFO:teuthology.orchestra.run.smithi137.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-09-12T12:00:56.098 INFO:teuthology.orchestra.run.smithi171.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 12 MB/s | 3.6 MB 00:00 2024-09-12T12:00:56.165 INFO:teuthology.orchestra.run.smithi171.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-12T12:00:56.233 INFO:teuthology.orchestra.run.smithi189.stdout:(148/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x 5.5 MB/s | 49 MB 00:08 2024-09-12T12:00:56.265 INFO:teuthology.orchestra.run.smithi171.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 17 MB/s | 7.4 MB 00:00 2024-09-12T12:00:56.290 INFO:teuthology.orchestra.run.smithi171.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 393 kB/s | 49 kB 00:00 2024-09-12T12:00:56.322 INFO:teuthology.orchestra.run.smithi137.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-12T12:00:56.324 INFO:teuthology.orchestra.run.smithi171.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 280 kB/s | 16 kB 00:00 2024-09-12T12:00:56.373 INFO:teuthology.orchestra.run.smithi171.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.4 MB/s | 285 kB 00:00 2024-09-12T12:00:56.400 INFO:teuthology.orchestra.run.smithi189.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 5.5 MB/s | 3.4 MB 00:00 2024-09-12T12:00:56.406 INFO:teuthology.orchestra.run.smithi171.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.1 MB/s | 765 kB 00:00 2024-09-12T12:00:56.423 INFO:teuthology.orchestra.run.smithi137.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 826 kB/s | 689 kB 00:00 2024-09-12T12:00:56.432 INFO:teuthology.orchestra.run.smithi171.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 310 kB/s | 18 kB 00:00 2024-09-12T12:00:56.448 INFO:teuthology.orchestra.run.smithi137.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 173 kB/s | 22 kB 00:00 2024-09-12T12:00:56.482 INFO:teuthology.orchestra.run.smithi137.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-12T12:00:56.525 INFO:teuthology.orchestra.run.smithi189.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 4.6 MB/s | 3.2 MB 00:00 2024-09-12T12:00:56.540 INFO:teuthology.orchestra.run.smithi137.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T12:00:56.565 INFO:teuthology.orchestra.run.smithi137.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 279 kB/s | 33 kB 00:00 2024-09-12T12:00:56.574 INFO:teuthology.orchestra.run.smithi171.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-12T12:00:56.657 INFO:teuthology.orchestra.run.smithi137.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-12T12:00:56.682 INFO:teuthology.orchestra.run.smithi137.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-12T12:00:56.775 INFO:teuthology.orchestra.run.smithi137.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-12T12:00:56.800 INFO:teuthology.orchestra.run.smithi137.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-12T12:00:56.825 INFO:teuthology.orchestra.run.smithi189.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 136 kB/s | 41 kB 00:00 2024-09-12T12:00:56.917 INFO:teuthology.orchestra.run.smithi137.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.9 MB/s | 3.0 MB 00:00 2024-09-12T12:00:56.975 INFO:teuthology.orchestra.run.smithi137.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-12T12:00:56.991 INFO:teuthology.orchestra.run.smithi171.stdout:(40/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T12:00:57.017 INFO:teuthology.orchestra.run.smithi171.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 432 kB/s | 191 kB 00:00 2024-09-12T12:00:57.017 INFO:teuthology.orchestra.run.smithi137.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-12T12:00:57.075 INFO:teuthology.orchestra.run.smithi171.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-12T12:00:57.076 INFO:teuthology.orchestra.run.smithi137.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-12T12:00:57.134 INFO:teuthology.orchestra.run.smithi171.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-12T12:00:57.134 INFO:teuthology.orchestra.run.smithi137.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-12T12:00:57.169 INFO:teuthology.orchestra.run.smithi189.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.5 MB/s | 2.3 MB 00:00 2024-09-12T12:00:57.193 INFO:teuthology.orchestra.run.smithi137.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-12T12:00:57.201 INFO:teuthology.orchestra.run.smithi171.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-09-12T12:00:57.226 INFO:teuthology.orchestra.run.smithi171.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-12T12:00:57.235 INFO:teuthology.orchestra.run.smithi137.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 384 kB/s | 38 kB 00:00 2024-09-12T12:00:57.252 INFO:teuthology.orchestra.run.smithi171.stdout:(46/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 974 kB/s | 253 kB 00:00 2024-09-12T12:00:57.285 INFO:teuthology.orchestra.run.smithi171.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-12T12:00:57.293 INFO:teuthology.orchestra.run.smithi189.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 533 kB/s | 249 kB 00:00 2024-09-12T12:00:57.311 INFO:teuthology.orchestra.run.smithi137.stdout:(66/156): libtool-2.4.6-46.el9.x86_64.rpm 1.1 MB/s | 581 kB 00:00 2024-09-12T12:00:57.311 INFO:teuthology.orchestra.run.smithi171.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-12T12:00:57.343 INFO:teuthology.orchestra.run.smithi189.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-12T12:00:57.386 INFO:teuthology.orchestra.run.smithi171.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-12T12:00:57.393 INFO:teuthology.orchestra.run.smithi189.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 1.4 MB 00:00 2024-09-12T12:00:57.412 INFO:teuthology.orchestra.run.smithi171.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-09-12T12:00:57.454 INFO:teuthology.orchestra.run.smithi171.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 323 kB/s | 22 kB 00:00 2024-09-12T12:00:57.504 INFO:teuthology.orchestra.run.smithi171.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-12T12:00:57.596 INFO:teuthology.orchestra.run.smithi171.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-09-12T12:00:57.646 INFO:teuthology.orchestra.run.smithi171.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-12T12:00:57.652 INFO:teuthology.orchestra.run.smithi137.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-12T12:00:57.689 INFO:teuthology.orchestra.run.smithi171.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-12T12:00:57.719 INFO:teuthology.orchestra.run.smithi137.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-12T12:00:57.722 INFO:teuthology.orchestra.run.smithi171.stdout:(56/156): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-09-12T12:00:57.786 INFO:teuthology.orchestra.run.smithi137.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 6.7 MB/s | 910 kB 00:00 2024-09-12T12:00:57.790 INFO:teuthology.orchestra.run.smithi171.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-12T12:00:57.802 INFO:teuthology.orchestra.run.smithi189.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.7 MB/s | 2.4 MB 00:00 2024-09-12T12:00:57.808 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-12T12:00:57.808 INFO:teuthology.orchestra.run.smithi189.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-12T12:00:57.836 INFO:teuthology.orchestra.run.smithi137.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-12T12:00:57.972 INFO:teuthology.orchestra.run.smithi137.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-09-12T12:00:57.978 INFO:teuthology.orchestra.run.smithi171.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 177 kB/s | 45 kB 00:00 2024-09-12T12:00:58.079 INFO:teuthology.orchestra.run.smithi137.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-09-12T12:00:58.157 INFO:teuthology.orchestra.run.smithi171.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-12T12:00:58.216 INFO:teuthology.orchestra.run.smithi171.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 421 kB/s | 24 kB 00:00 2024-09-12T12:00:58.274 INFO:teuthology.orchestra.run.smithi171.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-09-12T12:00:58.341 INFO:teuthology.orchestra.run.smithi171.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.2 MB/s | 662 kB 00:00 2024-09-12T12:00:58.383 INFO:teuthology.orchestra.run.smithi171.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-12T12:00:58.459 INFO:teuthology.orchestra.run.smithi171.stdout:(64/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-09-12T12:00:58.509 INFO:teuthology.orchestra.run.smithi171.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-12T12:00:58.534 INFO:teuthology.orchestra.run.smithi171.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 255 kB/s | 38 kB 00:00 2024-09-12T12:00:58.571 INFO:teuthology.orchestra.run.smithi137.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-12T12:00:58.713 INFO:teuthology.orchestra.run.smithi137.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 433 kB/s | 61 kB 00:00 2024-09-12T12:00:58.718 INFO:teuthology.orchestra.run.smithi171.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-09-12T12:00:58.789 INFO:teuthology.orchestra.run.smithi171.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 15 MB/s | 4.9 MB 00:00 2024-09-12T12:00:58.869 INFO:teuthology.orchestra.run.smithi171.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-12T12:00:58.911 INFO:teuthology.orchestra.run.smithi171.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 4.2 MB/s | 1.6 MB 00:00 2024-09-12T12:00:59.020 INFO:teuthology.orchestra.run.smithi171.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 3.0 MB/s | 910 kB 00:00 2024-09-12T12:00:59.042 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-12T12:00:59.062 INFO:teuthology.orchestra.run.smithi171.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-09-12T12:00:59.088 INFO:teuthology.orchestra.run.smithi137.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 652 kB/s | 245 kB 00:00 2024-09-12T12:00:59.146 INFO:teuthology.orchestra.run.smithi171.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-12T12:00:59.171 INFO:teuthology.orchestra.run.smithi171.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 579 kB/s | 87 kB 00:00 2024-09-12T12:00:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-12T12:00:59.192 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-12T12:00:59.221 INFO:teuthology.orchestra.run.smithi171.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 814 kB/s | 61 kB 00:00 2024-09-12T12:00:59.314 INFO:teuthology.orchestra.run.smithi171.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-12T12:00:59.339 INFO:teuthology.orchestra.run.smithi171.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-09-12T12:00:59.389 INFO:teuthology.orchestra.run.smithi171.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-09-12T12:00:59.406 INFO:teuthology.orchestra.run.smithi171.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-09-12T12:00:59.456 INFO:teuthology.orchestra.run.smithi171.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-12T12:00:59.463 INFO:teuthology.orchestra.run.smithi137.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 662 kB/s | 248 kB 00:00 2024-09-12T12:00:59.482 INFO:teuthology.orchestra.run.smithi171.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-12T12:00:59.557 INFO:teuthology.orchestra.run.smithi171.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-12T12:00:59.572 INFO:teuthology.orchestra.run.smithi137.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-09-12T12:00:59.649 INFO:teuthology.orchestra.run.smithi171.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-12T12:00:59.674 INFO:teuthology.orchestra.run.smithi171.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-09-12T12:00:59.688 INFO:teuthology.orchestra.run.smithi137.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-09-12T12:00:59.750 INFO:teuthology.orchestra.run.smithi171.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 564 kB/s | 42 kB 00:00 2024-09-12T12:00:59.775 INFO:teuthology.orchestra.run.smithi171.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 613 kB/s | 77 kB 00:00 2024-09-12T12:00:59.859 INFO:teuthology.orchestra.run.smithi171.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-12T12:00:59.884 INFO:teuthology.orchestra.run.smithi171.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-12T12:00:59.943 INFO:teuthology.orchestra.run.smithi171.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-09-12T12:00:59.968 INFO:teuthology.orchestra.run.smithi171.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-12T12:00:59.989 INFO:teuthology.orchestra.run.smithi137.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 574 kB/s | 172 kB 00:00 2024-09-12T12:01:00.061 INFO:teuthology.orchestra.run.smithi171.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.4 MB/s | 5.1 MB 00:00 2024-09-12T12:01:00.077 INFO:teuthology.orchestra.run.smithi171.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 400 kB/s | 54 kB 00:00 2024-09-12T12:01:00.106 INFO:teuthology.orchestra.run.smithi137.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 298 kB/s | 35 kB 00:00 2024-09-12T12:01:00.119 INFO:teuthology.orchestra.run.smithi171.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-09-12T12:01:00.137 INFO:teuthology.orchestra.run.smithi171.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-09-12T12:01:00.229 INFO:teuthology.orchestra.run.smithi171.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 390 kB/s | 36 kB 00:00 2024-09-12T12:01:00.254 INFO:teuthology.orchestra.run.smithi171.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-12T12:01:00.313 INFO:teuthology.orchestra.run.smithi171.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-09-12T12:01:00.380 INFO:teuthology.orchestra.run.smithi171.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 866 kB/s | 58 kB 00:00 2024-09-12T12:01:00.405 INFO:teuthology.orchestra.run.smithi171.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-12T12:01:00.464 INFO:teuthology.orchestra.run.smithi171.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-12T12:01:00.489 INFO:teuthology.orchestra.run.smithi171.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-09-12T12:01:00.548 INFO:teuthology.orchestra.run.smithi171.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-09-12T12:01:00.574 INFO:teuthology.orchestra.run.smithi171.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-12T12:01:00.641 INFO:teuthology.orchestra.run.smithi171.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-12T12:01:00.667 INFO:teuthology.orchestra.run.smithi171.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-12T12:01:00.691 INFO:teuthology.orchestra.run.smithi171.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-12T12:01:00.800 INFO:teuthology.orchestra.run.smithi171.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T12:01:00.834 INFO:teuthology.orchestra.run.smithi171.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-12T12:01:00.859 INFO:teuthology.orchestra.run.smithi171.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 442 kB/s | 26 kB 00:00 2024-09-12T12:01:00.893 INFO:teuthology.orchestra.run.smithi171.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-12T12:01:00.918 INFO:teuthology.orchestra.run.smithi171.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.3 MB/s | 151 kB 00:00 2024-09-12T12:01:00.943 INFO:teuthology.orchestra.run.smithi171.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 444 kB/s | 49 kB 00:00 2024-09-12T12:01:00.986 INFO:teuthology.orchestra.run.smithi171.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-12T12:01:01.000 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-12T12:01:01.001 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-12T12:01:01.011 INFO:teuthology.orchestra.run.smithi171.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T12:01:01.036 INFO:teuthology.orchestra.run.smithi171.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-12T12:01:01.062 INFO:teuthology.orchestra.run.smithi171.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 636 kB/s | 32 kB 00:00 2024-09-12T12:01:01.087 INFO:teuthology.orchestra.run.smithi171.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 278 kB/s | 14 kB 00:00 2024-09-12T12:01:01.113 INFO:teuthology.orchestra.run.smithi171.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-12T12:01:01.138 INFO:teuthology.orchestra.run.smithi171.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-12T12:01:01.172 INFO:teuthology.orchestra.run.smithi171.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-12T12:01:01.197 INFO:teuthology.orchestra.run.smithi171.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T12:01:01.223 INFO:teuthology.orchestra.run.smithi171.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-12T12:01:01.239 INFO:teuthology.orchestra.run.smithi171.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 644 kB/s | 11 kB 00:00 2024-09-12T12:01:01.273 INFO:teuthology.orchestra.run.smithi171.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 530 kB/s | 18 kB 00:00 2024-09-12T12:01:01.307 INFO:teuthology.orchestra.run.smithi171.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 694 kB/s | 23 kB 00:00 2024-09-12T12:01:01.332 INFO:teuthology.orchestra.run.smithi171.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-12T12:01:01.366 INFO:teuthology.orchestra.run.smithi171.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 587 kB/s | 20 kB 00:00 2024-09-12T12:01:01.417 INFO:teuthology.orchestra.run.smithi171.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.3 MB/s | 2.0 MB 00:00 2024-09-12T12:01:01.442 INFO:teuthology.orchestra.run.smithi171.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T12:01:01.467 INFO:teuthology.orchestra.run.smithi171.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T12:01:01.493 INFO:teuthology.orchestra.run.smithi171.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T12:01:01.527 INFO:teuthology.orchestra.run.smithi171.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-12T12:01:01.552 INFO:teuthology.orchestra.run.smithi171.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-12T12:01:01.577 INFO:teuthology.orchestra.run.smithi171.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-12T12:01:01.603 INFO:teuthology.orchestra.run.smithi171.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T12:01:01.628 INFO:teuthology.orchestra.run.smithi171.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-12T12:01:01.654 INFO:teuthology.orchestra.run.smithi171.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-12T12:01:01.679 INFO:teuthology.orchestra.run.smithi171.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T12:01:01.705 INFO:teuthology.orchestra.run.smithi171.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T12:01:01.730 INFO:teuthology.orchestra.run.smithi171.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-09-12T12:01:01.755 INFO:teuthology.orchestra.run.smithi171.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T12:01:01.781 INFO:teuthology.orchestra.run.smithi171.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T12:01:01.814 INFO:teuthology.orchestra.run.smithi171.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-12T12:01:01.840 INFO:teuthology.orchestra.run.smithi171.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-12T12:01:01.865 INFO:teuthology.orchestra.run.smithi171.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-12T12:01:01.891 INFO:teuthology.orchestra.run.smithi171.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T12:01:01.941 INFO:teuthology.orchestra.run.smithi171.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T12:01:02.108 INFO:teuthology.orchestra.run.smithi171.stdout:(148/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.6 MB/s | 16 MB 00:02 2024-09-12T12:01:02.242 INFO:teuthology.orchestra.run.smithi171.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-12T12:01:02.309 INFO:teuthology.orchestra.run.smithi171.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.6 MB/s | 3.2 MB 00:00 2024-09-12T12:01:02.426 INFO:teuthology.orchestra.run.smithi171.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 349 kB/s | 41 kB 00:00 2024-09-12T12:01:02.568 INFO:teuthology.orchestra.run.smithi171.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-12T12:01:02.743 INFO:teuthology.orchestra.run.smithi171.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.7 MB/s | 2.3 MB 00:00 2024-09-12T12:01:02.785 INFO:teuthology.orchestra.run.smithi171.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 6.2 MB/s | 1.4 MB 00:00 2024-09-12T12:01:02.844 INFO:teuthology.orchestra.run.smithi171.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 3.0 MB/s | 1.8 MB 00:00 2024-09-12T12:01:03.723 INFO:teuthology.orchestra.run.smithi137.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 996 kB/s | 5.8 MB 00:05 2024-09-12T12:01:03.944 INFO:teuthology.orchestra.run.smithi171.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-12T12:01:03.949 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-12T12:01:03.949 INFO:teuthology.orchestra.run.smithi171.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-12T12:01:04.098 INFO:teuthology.orchestra.run.smithi137.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 669 kB/s | 250 kB 00:00 2024-09-12T12:01:04.481 INFO:teuthology.orchestra.run.smithi137.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 579 kB/s | 222 kB 00:00 2024-09-12T12:01:04.665 INFO:teuthology.orchestra.run.smithi137.stdout:(84/156): python3-babel-2.9.1-2.el9.noarch.rpm 927 kB/s | 6.0 MB 00:06 2024-09-12T12:01:04.690 INFO:teuthology.orchestra.run.smithi137.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 370 kB/s | 77 kB 00:00 2024-09-12T12:01:04.774 INFO:teuthology.orchestra.run.smithi137.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-09-12T12:01:05.057 INFO:teuthology.orchestra.run.smithi137.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 561 kB/s | 159 kB 00:00 2024-09-12T12:01:05.145 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-12T12:01:05.166 INFO:teuthology.orchestra.run.smithi137.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 566 kB/s | 269 kB 00:00 2024-09-12T12:01:05.266 INFO:teuthology.orchestra.run.smithi137.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 510 kB/s | 51 kB 00:00 2024-09-12T12:01:05.297 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-12T12:01:05.297 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-12T12:01:05.367 INFO:teuthology.orchestra.run.smithi137.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 537 kB/s | 54 kB 00:00 2024-09-12T12:01:05.575 INFO:teuthology.orchestra.run.smithi137.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 539 kB/s | 279 kB 00:00 2024-09-12T12:01:05.675 INFO:teuthology.orchestra.run.smithi137.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-12T12:01:05.809 INFO:teuthology.orchestra.run.smithi137.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 330 kB/s | 44 kB 00:00 2024-09-12T12:01:06.201 INFO:teuthology.orchestra.run.smithi137.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 564 kB/s | 221 kB 00:00 2024-09-12T12:01:06.310 INFO:teuthology.orchestra.run.smithi137.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-09-12T12:01:06.410 INFO:teuthology.orchestra.run.smithi137.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-09-12T12:01:06.877 INFO:teuthology.orchestra.run.smithi137.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 652 kB/s | 304 kB 00:00 2024-09-12T12:01:06.994 INFO:teuthology.orchestra.run.smithi137.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 495 kB/s | 58 kB 00:00 2024-09-12T12:01:07.094 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-12T12:01:07.094 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-12T12:01:07.202 INFO:teuthology.orchestra.run.smithi137.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 460 kB/s | 96 kB 00:00 2024-09-12T12:01:07.403 INFO:teuthology.orchestra.run.smithi137.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 319 kB/s | 64 kB 00:00 2024-09-12T12:01:07.461 INFO:teuthology.orchestra.run.smithi137.stdout:(101/156): python3-numpy-1.20.1-5.el9.x86_64.rp 710 kB/s | 5.1 MB 00:07 2024-09-12T12:01:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-12T12:01:07.512 INFO:teuthology.orchestra.run.smithi137.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 412 kB/s | 45 kB 00:00 2024-09-12T12:01:07.581 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T12:01:07.587 INFO:teuthology.orchestra.run.smithi137.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-09-12T12:01:07.619 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T12:01:07.637 INFO:teuthology.orchestra.run.smithi137.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T12:01:07.662 INFO:teuthology.orchestra.run.smithi137.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-12T12:01:07.803 INFO:teuthology.orchestra.run.smithi137.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-12T12:01:07.803 INFO:teuthology.orchestra.run.smithi137.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T12:01:07.803 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T12:01:07.804 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T12:01:07.804 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T12:01:07.804 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T12:01:07.822 INFO:teuthology.orchestra.run.smithi137.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-09-12T12:01:07.847 INFO:teuthology.orchestra.run.smithi137.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-09-12T12:01:07.881 INFO:teuthology.orchestra.run.smithi137.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-12T12:01:07.906 INFO:teuthology.orchestra.run.smithi137.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-12T12:01:07.931 INFO:teuthology.orchestra.run.smithi137.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 445 kB/s | 49 kB 00:00 2024-09-12T12:01:07.973 INFO:teuthology.orchestra.run.smithi137.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-12T12:01:07.998 INFO:teuthology.orchestra.run.smithi137.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T12:01:08.024 INFO:teuthology.orchestra.run.smithi137.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T12:01:08.049 INFO:teuthology.orchestra.run.smithi137.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-09-12T12:01:08.074 INFO:teuthology.orchestra.run.smithi137.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-12T12:01:08.099 INFO:teuthology.orchestra.run.smithi137.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-12T12:01:08.112 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T12:01:08.125 INFO:teuthology.orchestra.run.smithi137.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-12T12:01:08.150 INFO:teuthology.orchestra.run.smithi137.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-12T12:01:08.175 INFO:teuthology.orchestra.run.smithi137.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-12T12:01:08.209 INFO:teuthology.orchestra.run.smithi137.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T12:01:08.224 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:01:08.234 INFO:teuthology.orchestra.run.smithi137.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-12T12:01:08.260 INFO:teuthology.orchestra.run.smithi137.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-12T12:01:08.285 INFO:teuthology.orchestra.run.smithi137.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-12T12:01:08.310 INFO:teuthology.orchestra.run.smithi137.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-12T12:01:08.335 INFO:teuthology.orchestra.run.smithi137.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-12T12:01:08.346 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:01:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:01:08.386 INFO:teuthology.orchestra.run.smithi137.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-12T12:01:08.411 INFO:teuthology.orchestra.run.smithi137.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T12:01:08.436 INFO:teuthology.orchestra.run.smithi137.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T12:01:08.462 INFO:teuthology.orchestra.run.smithi137.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T12:01:08.467 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:01:08.495 INFO:teuthology.orchestra.run.smithi137.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-12T12:01:08.496 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T12:01:08.512 INFO:teuthology.orchestra.run.smithi137.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 989 kB/s | 16 kB 00:00 2024-09-12T12:01:08.538 INFO:teuthology.orchestra.run.smithi137.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-12T12:01:08.563 INFO:teuthology.orchestra.run.smithi137.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-12T12:01:08.580 INFO:teuthology.orchestra.run.smithi137.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-09-12T12:01:08.597 INFO:teuthology.orchestra.run.smithi137.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 917 kB/s | 31 kB 00:00 2024-09-12T12:01:08.622 INFO:teuthology.orchestra.run.smithi137.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T12:01:08.644 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T12:01:08.655 INFO:teuthology.orchestra.run.smithi137.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T12:01:08.681 INFO:teuthology.orchestra.run.smithi137.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-12T12:01:08.692 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T12:01:08.706 INFO:teuthology.orchestra.run.smithi137.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-12T12:01:08.721 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:01:08.731 INFO:teuthology.orchestra.run.smithi137.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T12:01:08.782 INFO:teuthology.orchestra.run.smithi137.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T12:01:08.807 INFO:teuthology.orchestra.run.smithi137.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-09-12T12:01:08.816 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:01:08.841 INFO:teuthology.orchestra.run.smithi137.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-12T12:01:08.847 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T12:01:08.866 INFO:teuthology.orchestra.run.smithi137.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T12:01:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T12:01:08.916 INFO:teuthology.orchestra.run.smithi137.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T12:01:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T12:01:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T12:01:09.140 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T12:01:09.192 INFO:teuthology.orchestra.run.smithi137.stdout:(148/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-12T12:01:09.269 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T12:01:09.326 INFO:teuthology.orchestra.run.smithi137.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.4 MB/s | 3.4 MB 00:00 2024-09-12T12:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T12:01:09.487 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:09.564 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:09.751 INFO:teuthology.orchestra.run.smithi137.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 4.1 MB/s | 2.3 MB 00:00 2024-09-12T12:01:09.785 INFO:teuthology.orchestra.run.smithi137.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-09-12T12:01:09.785 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:09.827 INFO:teuthology.orchestra.run.smithi137.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 3.6 MB/s | 1.8 MB 00:00 2024-09-12T12:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T12:01:09.902 INFO:teuthology.orchestra.run.smithi137.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-12T12:01:09.989 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T12:01:10.019 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T12:01:10.062 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T12:01:10.150 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T12:01:10.205 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T12:01:10.244 INFO:teuthology.orchestra.run.smithi137.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 3.2 MB/s | 1.4 MB 00:00 2024-09-12T12:01:10.245 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T12:01:10.288 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T12:01:10.324 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T12:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T12:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T12:01:10.475 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T12:01:10.505 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T12:01:10.540 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T12:01:10.586 INFO:teuthology.orchestra.run.smithi137.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-09-12T12:01:10.616 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T12:01:10.651 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T12:01:10.720 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T12:01:10.757 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T12:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T12:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T12:01:10.955 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T12:01:11.088 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T12:01:11.140 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T12:01:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T12:01:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T12:01:11.774 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T12:01:11.816 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T12:01:11.855 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T12:01:11.896 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T12:01:12.052 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T12:01:12.116 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T12:01:12.171 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T12:01:12.252 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T12:01:12.282 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T12:01:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T12:01:12.365 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T12:01:12.410 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T12:01:12.453 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T12:01:12.495 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T12:01:12.640 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T12:01:12.689 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T12:01:12.730 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T12:01:13.045 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T12:01:13.080 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T12:01:13.120 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T12:01:13.146 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T12:01:13.171 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T12:01:13.245 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T12:01:13.380 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T12:01:13.419 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T12:01:13.447 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T12:01:13.449 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-12T12:01:13.481 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T12:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T12:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T12:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T12:01:13.590 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T12:01:13.617 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T12:01:13.650 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T12:01:13.688 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T12:01:13.691 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T12:01:13.726 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T12:01:13.812 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T12:01:13.865 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T12:01:13.893 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T12:01:13.896 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T12:01:13.943 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T12:01:13.968 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T12:01:13.998 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T12:01:14.021 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T12:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T12:01:14.357 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T12:01:14.461 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:01:14.600 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:01:14.629 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:01:14.711 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:01:14.743 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T12:01:14.769 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T12:01:14.844 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T12:01:14.878 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T12:01:14.895 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T12:01:14.948 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T12:01:14.976 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T12:01:14.985 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:01:15.033 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T12:01:15.077 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T12:01:15.089 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:01:15.119 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T12:01:15.172 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T12:01:15.182 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T12:01:15.213 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T12:01:15.213 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T12:01:15.235 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T12:01:15.382 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T12:01:15.399 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T12:01:15.428 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T12:01:15.506 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T12:01:15.508 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T12:01:15.654 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T12:01:15.759 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:15.815 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:16.044 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:16.145 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T12:01:16.154 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T12:01:16.250 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T12:01:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T12:01:16.287 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T12:01:16.322 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T12:01:16.401 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T12:01:16.445 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T12:01:16.477 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T12:01:16.520 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T12:01:16.553 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T12:01:16.582 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T12:01:16.651 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T12:01:16.713 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T12:01:16.743 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T12:01:16.779 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T12:01:16.857 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T12:01:16.890 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T12:01:16.933 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T12:01:16.963 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T12:01:17.033 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T12:01:17.078 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T12:01:17.153 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T12:01:17.296 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T12:01:17.356 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T12:01:17.485 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T12:01:17.575 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T12:01:17.611 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T12:01:17.644 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T12:01:17.894 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T12:01:17.953 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T12:01:17.988 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T12:01:18.023 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T12:01:18.024 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T12:01:18.058 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T12:01:18.097 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T12:01:18.098 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T12:01:18.137 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T12:01:18.172 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T12:01:18.173 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T12:01:18.341 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T12:01:18.417 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T12:01:18.473 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T12:01:18.546 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T12:01:18.578 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T12:01:18.614 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T12:01:18.650 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T12:01:18.661 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T12:01:18.694 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T12:01:18.697 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T12:01:18.740 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T12:01:18.748 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T12:01:18.778 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T12:01:18.785 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T12:01:18.923 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T12:01:19.119 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T12:01:19.170 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T12:01:19.277 INFO:teuthology.orchestra.run.smithi137.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.2 MB/s | 16 MB 00:13 2024-09-12T12:01:19.277 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T12:01:19.282 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-09-12T12:01:19.282 INFO:teuthology.orchestra.run.smithi137.stdout:Total 6.8 MB/s | 226 MB 00:33 2024-09-12T12:01:19.311 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T12:01:19.351 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T12:01:19.377 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T12:01:19.402 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T12:01:19.478 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T12:01:19.606 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T12:01:19.652 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T12:01:19.681 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T12:01:19.715 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T12:01:19.755 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T12:01:19.795 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T12:01:19.825 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T12:01:19.859 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T12:01:19.893 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T12:01:19.935 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T12:01:19.969 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T12:01:20.057 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T12:01:20.100 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T12:01:20.140 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T12:01:20.245 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T12:01:20.309 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T12:01:20.443 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-09-12T12:01:20.593 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-09-12T12:01:20.594 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-09-12T12:01:21.054 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T12:01:21.100 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T12:01:21.124 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T12:01:21.157 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:21.236 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T12:01:21.298 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T12:01:21.345 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T12:01:21.432 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T12:01:21.463 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T12:01:21.649 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T12:01:21.700 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:21.711 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T12:01:21.744 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T12:01:21.788 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T12:01:21.838 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T12:01:21.839 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T12:01:21.839 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T12:01:21.839 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:21.870 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T12:01:22.442 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T12:01:22.556 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T12:01:23.131 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-09-12T12:01:23.131 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-09-12T12:01:23.819 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T12:01:23.896 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T12:01:23.924 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T12:01:23.957 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T12:01:24.268 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T12:01:24.304 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T12:01:24.395 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T12:01:24.426 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T12:01:24.462 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T12:01:24.949 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T12:01:25.000 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T12:01:25.054 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T12:01:25.077 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T12:01:27.323 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:27.375 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:27.919 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:27.963 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T12:01:28.055 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T12:01:28.056 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T12:01:28.056 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T12:01:28.056 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:28.090 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T12:01:29.443 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-09-12T12:01:29.572 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T12:01:29.611 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T12:01:29.664 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T12:01:29.705 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T12:01:29.731 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T12:01:29.779 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T12:01:30.116 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T12:01:30.224 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:01:30.349 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:01:30.375 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:01:30.466 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:01:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T12:01:30.657 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T12:01:30.734 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T12:01:30.762 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:01:30.866 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:01:30.889 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T12:01:30.946 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T12:01:30.973 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T12:01:30.996 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T12:01:31.191 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T12:01:31.368 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T12:01:31.488 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T12:01:31.590 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:31.672 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:31.885 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T12:01:32.000 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T12:01:32.099 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T12:01:32.144 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T12:01:32.188 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T12:01:32.261 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T12:01:32.305 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T12:01:32.346 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T12:01:32.389 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T12:01:32.422 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T12:01:32.449 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T12:01:32.518 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T12:01:32.580 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T12:01:32.611 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T12:01:32.655 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T12:01:32.731 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T12:01:32.774 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T12:01:32.818 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T12:01:32.845 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T12:01:32.914 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T12:01:32.960 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T12:01:33.042 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T12:01:33.182 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T12:01:33.237 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T12:01:33.759 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T12:01:33.832 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T12:01:33.832 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-09-12T12:01:33.832 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-09-12T12:01:33.832 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-09-12T12:01:33.832 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-09-12T12:01:33.833 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-09-12T12:01:33.833 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-09-12T12:01:33.833 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-09-12T12:01:33.833 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:33.840 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T12:01:33.871 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T12:01:33.913 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T12:01:33.951 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T12:01:33.994 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T12:01:34.006 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T12:01:34.022 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T12:01:34.022 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:34.022 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T12:01:34.022 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:34.158 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T12:01:34.221 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T12:01:34.276 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T12:01:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T12:01:34.397 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T12:01:34.434 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T12:01:34.489 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T12:01:34.525 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T12:01:34.568 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T12:01:34.610 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T12:01:34.745 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T12:01:34.795 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T12:01:34.844 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T12:01:34.946 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T12:01:35.004 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T12:01:35.053 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T12:01:35.086 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-12T12:01:35.114 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T12:01:35.211 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T12:01:35.340 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T12:01:35.379 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T12:01:35.413 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T12:01:35.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T12:01:35.497 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T12:01:35.500 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T12:01:35.527 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T12:01:35.527 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:35.527 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T12:01:35.527 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:01:35.528 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:01:35.528 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:35.529 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T12:01:35.559 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T12:01:35.593 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T12:01:35.635 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T12:01:35.676 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T12:01:35.711 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T12:01:35.759 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T12:01:35.783 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T12:01:35.783 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:35.783 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T12:01:35.783 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:01:35.783 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:01:35.783 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:35.797 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T12:01:35.841 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T12:01:35.873 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T12:01:35.976 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T12:01:36.032 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T12:01:36.226 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T12:01:36.250 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T12:01:36.250 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:36.250 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T12:01:36.250 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:01:36.250 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:01:36.250 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:36.356 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T12:01:36.413 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T12:01:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T12:01:36.653 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T12:01:36.711 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T12:01:36.758 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T12:01:36.788 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T12:01:36.804 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T12:01:36.847 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T12:01:36.877 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T12:01:36.945 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T12:01:36.994 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T12:01:37.008 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T12:01:37.039 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T12:01:37.052 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T12:01:37.100 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T12:01:37.128 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T12:01:37.134 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T12:01:37.175 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T12:01:37.360 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T12:01:37.389 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T12:01:37.473 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T12:01:37.588 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T12:01:37.619 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T12:01:37.650 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T12:01:37.713 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T12:01:37.750 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T12:01:38.138 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T12:01:38.254 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T12:01:38.791 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T12:01:38.826 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T12:01:39.489 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T12:01:39.560 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T12:01:39.587 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T12:01:39.620 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T12:01:39.799 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T12:01:39.826 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T12:01:39.930 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T12:01:39.943 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T12:01:40.000 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T12:01:40.078 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T12:01:40.078 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-09-12T12:01:40.078 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-09-12T12:01:40.078 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-09-12T12:01:40.078 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-09-12T12:01:40.078 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-09-12T12:01:40.079 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-09-12T12:01:40.079 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-09-12T12:01:40.079 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T12:01:40.098 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T12:01:40.122 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T12:01:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T12:01:40.158 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T12:01:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T12:01:40.162 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:40.162 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T12:01:40.162 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:01:40.162 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:01:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:40.213 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T12:01:40.225 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T12:01:40.272 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T12:01:40.288 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T12:01:40.288 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:40.288 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T12:01:40.289 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:40.323 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T12:01:40.370 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T12:01:40.638 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T12:01:40.670 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T12:01:40.724 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T12:01:40.755 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T12:01:41.300 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T12:01:41.332 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T12:01:41.356 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T12:01:41.356 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:41.356 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T12:01:41.356 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:01:41.357 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:01:41.357 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:41.399 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T12:01:41.422 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T12:01:41.422 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:41.422 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T12:01:41.422 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:41.475 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T12:01:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T12:01:41.499 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:41.499 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T12:01:41.499 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:01:41.499 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:01:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:41.776 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T12:01:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T12:01:41.803 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T12:01:41.804 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:41.804 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T12:01:41.804 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:01:41.804 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:01:41.804 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T12:01:41.825 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:41.825 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T12:01:41.826 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:01:41.826 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:01:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:42.028 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T12:01:42.050 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T12:01:42.051 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:42.051 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T12:01:42.051 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:01:42.051 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:01:42.051 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:42.486 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T12:01:42.510 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T12:01:42.510 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:42.510 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T12:01:42.510 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:01:42.510 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:01:42.510 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:42.617 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T12:01:42.674 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T12:01:42.730 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T12:01:42.920 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T12:01:42.973 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T12:01:43.052 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T12:01:43.074 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:43.109 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:43.141 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T12:01:43.283 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T12:01:43.332 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T12:01:43.364 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T12:01:43.392 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T12:01:43.659 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T12:01:43.713 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T12:01:43.803 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T12:01:43.803 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T12:01:43.804 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T12:01:43.804 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:01:43.835 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T12:01:43.839 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T12:01:43.867 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T12:01:43.898 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T12:01:43.961 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T12:01:43.990 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T12:01:45.041 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T12:01:45.084 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T12:01:46.049 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T12:01:46.076 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T12:01:46.181 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T12:01:46.339 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T12:01:46.362 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T12:01:46.386 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T12:01:46.386 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:46.387 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T12:01:46.387 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:01:46.387 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:01:46.387 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:46.387 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T12:01:46.429 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T12:01:46.438 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T12:01:46.450 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T12:01:46.469 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T12:01:46.501 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T12:01:46.528 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T12:01:46.547 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T12:01:46.595 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T12:01:46.711 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T12:01:46.759 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T12:01:46.789 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T12:01:46.818 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T12:01:46.865 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T12:01:47.494 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:01:47.518 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T12:01:47.537 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:01:47.550 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T12:01:47.558 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:01:47.574 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T12:01:47.574 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:47.574 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T12:01:47.574 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:01:47.574 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:01:47.574 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:47.618 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T12:01:47.640 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T12:01:47.640 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:47.640 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T12:01:47.640 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:47.685 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T12:01:47.708 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T12:01:47.708 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:47.708 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T12:01:47.708 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:01:47.708 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:01:47.709 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:48.019 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T12:01:48.042 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T12:01:48.043 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:48.043 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T12:01:48.043 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:01:48.043 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:01:48.043 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:01:48.396 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T12:01:48.483 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T12:01:48.523 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T12:01:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T12:01:48.652 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T12:01:48.678 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T12:01:48.704 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T12:01:48.704 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T12:01:48.788 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T12:01:48.788 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T12:01:48.880 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T12:01:48.905 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T12:01:48.911 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:01:48.911 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:01:49.082 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:01:49.082 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T12:01:49.174 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T12:01:49.238 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:01:52.583 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T12:01:52.624 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T12:01:52.667 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T12:01:52.698 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T12:01:52.725 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T12:01:52.920 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T12:01:52.965 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T12:01:52.995 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T12:01:53.025 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T12:01:53.072 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T12:01:53.688 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:01:53.730 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:01:53.750 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:01:54.614 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T12:01:54.722 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T12:01:54.764 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T12:01:54.855 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T12:01:54.895 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T12:01:54.901 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:01:54.901 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T12:01:54.901 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T12:01:54.901 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T12:01:54.901 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T12:01:54.902 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T12:01:54.903 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T12:01:54.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T12:01:54.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T12:01:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T12:01:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T12:01:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T12:01:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T12:01:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T12:01:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T12:01:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T12:01:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T12:01:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T12:01:54.918 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T12:01:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T12:01:54.946 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T12:01:55.029 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T12:01:55.029 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T12:01:55.114 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T12:01:55.139 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T12:01:55.144 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:01:55.144 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:01:55.316 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:01:55.316 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T12:01:55.400 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T12:01:55.472 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:01:55.523 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-09-12T12:01:55.524 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:01:55.703 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T12:01:55.719 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T12:01:55.719 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:55.719 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T12:01:55.719 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:01:57.197 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T12:01:57.224 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T12:01:57.224 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:57.224 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T12:01:57.224 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:01:57.224 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T12:01:57.225 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:01:57.456 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T12:01:57.480 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T12:01:57.480 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:57.480 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T12:01:57.480 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:01:57.480 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T12:01:57.481 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:01:57.733 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:01:57.733 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-12T12:01:57.734 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.735 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.736 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T12:01:57.737 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.738 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T12:01:57.739 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.740 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T12:01:57.741 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T12:01:57.742 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T12:01:57.743 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T12:01:57.744 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T12:01:57.746 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:01:57.747 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-12T12:01:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T12:01:57.947 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T12:01:57.947 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:01:57.947 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T12:01:57.947 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:01:57.947 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T12:01:57.947 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:01:58.054 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T12:01:58.111 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T12:01:58.166 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T12:01:58.171 DEBUG:teuthology.parallel:result is None 2024-09-12T12:01:58.358 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T12:01:58.408 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T12:01:58.485 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T12:01:58.574 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T12:01:58.713 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T12:01:58.802 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T12:01:58.840 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T12:01:58.868 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T12:01:59.325 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T12:01:59.358 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T12:01:59.389 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T12:01:59.452 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T12:01:59.481 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T12:02:00.536 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T12:02:00.574 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:02:01.538 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T12:02:01.539 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T12:02:01.540 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T12:02:01.540 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T12:02:01.540 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T12:02:01.540 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T12:02:01.540 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T12:02:01.540 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T12:02:01.540 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T12:02:01.547 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T12:02:01.552 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T12:02:01.553 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T12:02:01.554 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T12:02:01.555 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T12:02:01.556 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T12:02:01.557 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T12:02:01.558 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T12:02:01.559 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T12:02:01.560 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T12:02:01.561 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T12:02:01.562 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T12:02:01.563 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T12:02:01.583 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T12:02:01.686 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T12:02:01.853 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T12:02:01.885 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T12:02:01.909 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T12:02:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:02:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T12:02:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:02:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T12:02:01.909 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:02:01.961 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T12:02:01.972 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T12:02:02.072 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T12:02:02.134 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T12:02:03.061 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T12:02:03.089 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T12:02:03.112 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T12:02:03.112 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:02:03.113 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T12:02:03.113 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:02:03.113 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T12:02:03.113 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:02:03.164 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T12:02:03.186 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T12:02:03.187 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:02:03.187 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T12:02:03.187 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:02:03.240 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T12:02:03.262 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T12:02:03.262 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:02:03.262 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T12:02:03.263 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:02:03.263 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T12:02:03.263 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:02:03.565 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T12:02:03.588 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T12:02:03.588 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T12:02:03.588 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T12:02:03.589 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:02:03.589 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T12:02:03.589 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:02:04.147 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:02:04.147 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:02:04.147 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-12T12:02:04.148 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.149 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.150 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.150 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.150 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.150 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.150 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.151 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T12:02:04.152 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.153 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T12:02:04.154 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T12:02:04.155 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T12:02:04.156 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T12:02:04.157 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T12:02:04.158 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T12:02:04.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T12:02:04.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.161 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:04.162 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T12:02:04.162 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T12:02:04.162 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T12:02:04.162 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T12:02:04.162 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T12:02:04.162 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T12:02:04.163 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T12:02:04.163 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:02:04.163 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-12T12:02:04.591 DEBUG:teuthology.parallel:result is None 2024-09-12T12:02:08.152 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T12:02:08.202 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T12:02:08.242 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T12:02:08.275 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T12:02:08.302 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T12:02:08.496 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T12:02:08.548 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T12:02:08.579 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T12:02:08.609 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T12:02:08.655 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T12:02:09.231 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:02:09.269 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:02:09.289 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T12:02:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T12:02:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T12:02:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T12:02:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T12:02:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T12:02:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T12:02:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T12:02:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T12:02:10.537 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T12:02:10.538 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T12:02:10.621 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T12:02:10.646 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T12:02:10.652 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:02:10.652 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:02:10.831 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T12:02:10.832 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T12:02:10.907 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T12:02:10.971 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T12:02:16.748 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T12:02:16.749 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T12:02:16.750 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T12:02:16.751 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T12:02:16.752 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T12:02:16.753 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T12:02:16.754 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T12:02:16.755 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-12T12:02:16.756 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T12:02:16.757 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T12:02:16.758 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T12:02:16.759 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T12:02:16.760 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T12:02:16.761 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T12:02:19.024 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T12:02:19.024 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:02:19.024 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-09-12T12:02:19.024 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T12:02:19.025 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.026 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T12:02:19.027 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.028 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T12:02:19.029 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T12:02:19.030 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.031 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T12:02:19.032 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T12:02:19.033 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T12:02:19.034 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T12:02:19.035 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T12:02:19.036 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T12:02:19.037 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T12:02:19.037 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T12:02:19.037 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T12:02:19.037 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T12:02:19.037 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T12:02:19.037 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T12:02:19.037 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:02:19.038 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-09-12T12:02:19.458 DEBUG:teuthology.parallel:result is None 2024-09-12T12:02:19.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:02:19.610 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T12:02:19.633 INFO:teuthology.orchestra.run.smithi137.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T12:02:19.634 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T12:02:19.634 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T12:02:19.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:02:19.788 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T12:02:19.815 INFO:teuthology.orchestra.run.smithi171.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T12:02:19.815 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T12:02:19.815 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T12:02:19.817 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:02:19.965 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T12:02:19.992 INFO:teuthology.orchestra.run.smithi189.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T12:02:19.992 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T12:02:19.992 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T12:02:19.994 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T12:02:19.994 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:02:19.994 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T12:02:20.023 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:02:20.023 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T12:02:20.055 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:02:20.056 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T12:02:20.088 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T12:02:20.089 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:02:20.089 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T12:02:20.115 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T12:02:20.180 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:02:20.181 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T12:02:20.211 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T12:02:20.283 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:02:20.283 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T12:02:20.311 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T12:02:20.377 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T12:02:20.378 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:02:20.378 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T12:02:20.403 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T12:02:20.469 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:02:20.469 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T12:02:20.495 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T12:02:20.561 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:02:20.561 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T12:02:20.586 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T12:02:20.653 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T12:02:20.653 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:02:20.653 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T12:02:20.678 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T12:02:20.743 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:02:20.744 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T12:02:20.770 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T12:02:20.837 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:02:20.837 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T12:02:20.861 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T12:02:20.927 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T12:02:21.023 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-12T12:02:21.023 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:02:21.023 INFO:tasks.cephadm:Cluster fsid is dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:02:21.024 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T12:02:21.024 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.137', 'mon.b': '172.21.15.171', 'mon.c': '172.21.15.189'} 2024-09-12T12:02:21.024 INFO:tasks.cephadm:First mon is mon.a on smithi137 2024-09-12T12:02:21.024 INFO:tasks.cephadm:First mgr is x 2024-09-12T12:02:21.024 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T12:02:21.024 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-09-12T12:02:21.050 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-09-12T12:02:21.076 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-09-12T12:02:21.101 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T12:02:21.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:02:21.253 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T12:02:21.398 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:02:21.399 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T12:02:21.399 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T12:02:21.399 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:21.791 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 12:02 /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:21.791 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:22.179 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 12:02 /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:22.179 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:22.553 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 12:02 /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:22.554 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:22.574 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:22.591 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T12:02:22.629 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T12:02:22.629 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T12:02:22.632 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T12:02:22.634 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T12:02:24.200 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T12:02:24.236 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T12:02:24.281 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T12:02:50.974 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-12T12:02:50.974 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:02:50.974 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T12:02:50.974 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-09-12T12:02:50.975 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T12:02:50.975 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-12T12:02:50.975 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-12T12:03:05.382 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:03:05.382 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:03:05.382 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T12:03:05.382 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-09-12T12:03:05.382 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T12:03:05.382 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:03:05.382 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:03:13.514 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-12T12:03:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T12:03:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T12:03:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-09-12T12:03:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T12:03:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-12T12:03:13.515 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-12T12:03:13.541 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-09-12T12:03:13.575 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-09-12T12:03:13.610 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-09-12T12:03:13.638 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-09-12T12:03:13.666 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-09-12T12:03:13.694 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-09-12T12:03:13.721 INFO:tasks.cephadm:Writing seed config... 2024-09-12T12:03:13.724 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T12:03:13.724 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T12:03:13.724 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T12:03:13.724 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T12:03:13.724 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T12:03:13.724 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T12:03:13.725 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T12:03:13.725 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T12:03:13.725 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T12:03:13.725 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T12:03:13.725 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T12:03:13.725 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T12:03:13.725 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T12:03:13.726 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T12:03:13.727 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T12:03:13.728 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T12:03:13.729 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T12:03:13.729 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T12:03:13.730 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:03:13.730 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T12:03:13.747 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = dd66268a-70fe-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T12:03:13.747 DEBUG:teuthology.orchestra.run.smithi137:mon.a> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T12:03:13.790 DEBUG:teuthology.orchestra.run.smithi137:mgr.x> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T12:03:13.832 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T12:03:13.832 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid dd66268a-70fe-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.137 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:03:14.037 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-09-12T12:03:14.037 INFO:teuthology.orchestra.run.smithi137.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'dd66268a-70fe-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.137', '--skip-admin-label'] 2024-09-12T12:03:14.038 INFO:teuthology.orchestra.run.smithi137.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T12:03:14.038 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying podman|docker is present... 2024-09-12T12:03:14.062 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 5.2.2 2024-09-12T12:03:14.062 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying lvm2 is present... 2024-09-12T12:03:14.062 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying time synchronization is in place... 2024-09-12T12:03:14.070 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T12:03:14.070 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T12:03:14.076 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T12:03:14.076 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-09-12T12:03:14.083 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout enabled 2024-09-12T12:03:14.089 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout active 2024-09-12T12:03:14.089 INFO:teuthology.orchestra.run.smithi137.stdout:Unit chronyd.service is enabled and running 2024-09-12T12:03:14.089 INFO:teuthology.orchestra.run.smithi137.stdout:Repeating the final host check... 2024-09-12T12:03:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 5.2.2 2024-09-12T12:03:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T12:03:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl is present 2024-09-12T12:03:14.112 INFO:teuthology.orchestra.run.smithi137.stdout:lvcreate is present 2024-09-12T12:03:14.118 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T12:03:14.118 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T12:03:14.124 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T12:03:14.125 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-09-12T12:03:14.131 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout enabled 2024-09-12T12:03:14.138 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout active 2024-09-12T12:03:14.138 INFO:teuthology.orchestra.run.smithi137.stdout:Unit chronyd.service is enabled and running 2024-09-12T12:03:14.138 INFO:teuthology.orchestra.run.smithi137.stdout:Host looks OK 2024-09-12T12:03:14.138 INFO:teuthology.orchestra.run.smithi137.stdout:Cluster fsid: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:14.138 INFO:teuthology.orchestra.run.smithi137.stdout:Acquiring lock 140531715306496 on /run/cephadm/dd66268a-70fe-11ef-bcea-c7b262605968.lock 2024-09-12T12:03:14.138 INFO:teuthology.orchestra.run.smithi137.stdout:Lock 140531715306496 acquired on /run/cephadm/dd66268a-70fe-11ef-bcea-c7b262605968.lock 2024-09-12T12:03:14.139 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying IP 172.21.15.137 port 3300 ... 2024-09-12T12:03:14.139 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying IP 172.21.15.137 port 6789 ... 2024-09-12T12:03:14.139 INFO:teuthology.orchestra.run.smithi137.stdout:Base mon IP(s) is [172.21.15.137:3300, 172.21.15.137:6789], mon addrv is [v2:172.21.15.137:3300,v1:172.21.15.137:6789] 2024-09-12T12:03:14.142 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.137 metric 100 2024-09-12T12:03:14.142 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.137 metric 100 2024-09-12T12:03:14.145 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T12:03:14.145 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-12T12:03:14.145 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-12T12:03:14.147 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T12:03:14.147 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T12:03:14.147 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T12:03:14.147 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-12T12:03:14.147 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:8137/64 scope link noprefixroute 2024-09-12T12:03:14.148 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T12:03:14.148 INFO:teuthology.orchestra.run.smithi137.stdout:Mon IP `172.21.15.137` is in CIDR network `172.21.0.0/20` 2024-09-12T12:03:14.148 INFO:teuthology.orchestra.run.smithi137.stdout:Mon IP `172.21.15.137` is in CIDR network `172.21.0.0/20` 2024-09-12T12:03:14.148 INFO:teuthology.orchestra.run.smithi137.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T12:03:14.149 INFO:teuthology.orchestra.run.smithi137.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T12:03:14.149 INFO:teuthology.orchestra.run.smithi137.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T12:03:14.567 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T12:03:14.567 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T12:03:14.568 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T12:03:14.568 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T12:03:14.568 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T12:03:14.568 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T12:03:14.568 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T12:03:15.438 INFO:teuthology.orchestra.run.smithi137.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T12:03:15.438 INFO:teuthology.orchestra.run.smithi137.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T12:03:15.438 INFO:teuthology.orchestra.run.smithi137.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T12:03:16.426 INFO:teuthology.orchestra.run.smithi137.stdout:stat: stdout 167 167 2024-09-12T12:03:16.427 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial keys... 2024-09-12T12:03:17.348 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQCE2OJmX8U+KxAA8dE/HXEjzmqcboJSS81yww== 2024-09-12T12:03:18.288 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQCF2OJmi/ueJxAAD192CMQlpYwR8v0tKJhRqg== 2024-09-12T12:03:19.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQCG2OJm7jSEIxAACAFrGlGWP4lnrb/hMosc0g== 2024-09-12T12:03:19.210 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial monmap... 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:monmaptool for a [v2:172.21.15.137:3300,v1:172.21.15.137:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:setting min_mon_release = quincy 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: set fsid to dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T12:03:20.191 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:03:20.192 INFO:teuthology.orchestra.run.smithi137.stdout:Creating mon... 2024-09-12T12:03:21.180 INFO:teuthology.orchestra.run.smithi137.stdout:create mon.a on 2024-09-12T12:03:21.394 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T12:03:21.548 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T12:03:21.722 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-dd66268a-70fe-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-dd66268a-70fe-11ef-bcea-c7b262605968.target. 2024-09-12T12:03:21.722 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-dd66268a-70fe-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-dd66268a-70fe-11ef-bcea-c7b262605968.target. 2024-09-12T12:03:21.995 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a 2024-09-12T12:03:21.995 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T12:03:22.161 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dd66268a-70fe-11ef-bcea-c7b262605968.target.wants/ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-dd66268a-70fe-11ef-bcea-c7b262605968@.service. 2024-09-12T12:03:22.755 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T12:03:22.755 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-09-12T12:03:22.763 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T12:03:22.763 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-09-12T12:03:22.764 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-09-12T12:03:22.764 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T12:03:22.764 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mon to start... 2024-09-12T12:03:22.764 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mon... 2024-09-12T12:03:23.754 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T12:03:23.754 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout id: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:23.754 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T12:03:23.754 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout services: 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.585685s) 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout data: 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:23.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.212+0000 7fee191cb640 1 Processor -- start 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.212+0000 7fee191cb640 1 -- start start 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.212+0000 7fee191cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee14107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.212+0000 7fee191cb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fee141080c0 con 0x7fee141076f0 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.212+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee14107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.212+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee14107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37546/0 (socket says 172.21.15.137:37546) 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.212+0000 7fee12d9d640 1 -- 172.21.15.137:0/2191298850 learned_addr learned my addr 172.21.15.137:0/2191298850 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.213+0000 7fee12d9d640 1 -- 172.21.15.137:0/2191298850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee141088f0 con 0x7fee141076f0 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.216+0000 7fee12d9d640 1 --2- 172.21.15.137:0/2191298850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee14107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fee04009820 tx=0x7fee0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecfdef947328f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.216+0000 7fee1259c640 1 -- 172.21.15.137:0/2191298850 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee04037070 con 0x7fee141076f0 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.216+0000 7fee1259c640 1 -- 172.21.15.137:0/2191298850 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee04032440 con 0x7fee141076f0 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.216+0000 7fee1259c640 1 -- 172.21.15.137:0/2191298850 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee04032740 con 0x7fee141076f0 2024-09-12T12:03:23.756 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 -- 172.21.15.137:0/2191298850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 msgr2=0x7fee14107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 --2- 172.21.15.137:0/2191298850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee14107af0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fee04009820 tx=0x7fee0402a270 comp rx=0 tx=0).stop 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 -- 172.21.15.137:0/2191298850 shutdown_connections 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 --2- 172.21.15.137:0/2191298850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee14107af0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 -- 172.21.15.137:0/2191298850 >> 172.21.15.137:0/2191298850 conn(0x7fee14102ec0 msgr2=0x7fee141052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 -- 172.21.15.137:0/2191298850 shutdown_connections 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 -- 172.21.15.137:0/2191298850 wait complete. 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 Processor -- start 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.217+0000 7fee191cb640 1 -- start start 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee191cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee1407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee191cb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fee14108e90 con 0x7fee141076f0 2024-09-12T12:03:23.757 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee1407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee12d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee1407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37558/0 (socket says 172.21.15.137:37558) 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee12d9d640 1 -- 172.21.15.137:0/672247896 learned_addr learned my addr 172.21.15.137:0/672247896 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee12d9d640 1 -- 172.21.15.137:0/672247896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1407e0b0 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee12d9d640 1 --2- 172.21.15.137:0/672247896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee1407db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fee040090b0 tx=0x7fee040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee10d99640 1 -- 172.21.15.137:0/672247896 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee04041070 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee10d99640 1 -- 172.21.15.137:0/672247896 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0403c7a0 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1407c2a0 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee10d99640 1 -- 172.21.15.137:0/672247896 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee04037040 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.218+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1407c7e0 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.219+0000 7fee10d99640 1 -- 172.21.15.137:0/672247896 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee04044070 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.219+0000 7fee10d99640 1 -- 172.21.15.137:0/672247896 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee0403d3f0 con 0x7fee141076f0 2024-09-12T12:03:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.219+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee141094e0 con 0x7fee141076f0 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.222+0000 7fee10d99640 1 -- 172.21.15.137:0/672247896 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fee04046450 con 0x7fee141076f0 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.280+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fee1407cee0 con 0x7fee141076f0 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.280+0000 7fee10d99640 1 -- 172.21.15.137:0/672247896 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fee04046630 con 0x7fee141076f0 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.281+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 msgr2=0x7fee1407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.281+0000 7fee191cb640 1 --2- 172.21.15.137:0/672247896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee1407db70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fee040090b0 tx=0x7fee040046c0 comp rx=0 tx=0).stop 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.281+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 shutdown_connections 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.281+0000 7fee191cb640 1 --2- 172.21.15.137:0/672247896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee141076f0 0x7fee1407db70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.281+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 >> 172.21.15.137:0/672247896 conn(0x7fee14102ec0 msgr2=0x7fee1418f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.281+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 shutdown_connections 2024-09-12T12:03:23.759 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:23.281+0000 7fee191cb640 1 -- 172.21.15.137:0/672247896 wait complete. 2024-09-12T12:03:23.760 INFO:teuthology.orchestra.run.smithi137.stdout:mon is available 2024-09-12T12:03:23.760 INFO:teuthology.orchestra.run.smithi137.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T12:03:24.752 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:24 smithi137 ceph-mon[30580]: from='client.? 172.21.15.137:0/486568315' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T12:03:24.755 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:24.763 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T12:03:24.763 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout fsid = dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:24.763 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.137:3300,v1:172.21.15.137:6789] 2024-09-12T12:03:24.764 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T12:03:24.764 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T12:03:24.764 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T12:03:24.765 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.182+0000 7fd2d8d21640 1 Processor -- start 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.182+0000 7fd2d8d21640 1 -- start start 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.182+0000 7fd2d8d21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.182+0000 7fd2d8d21640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d407c8f0 con 0x7fd2d407de70 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.182+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.183+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37560/0 (socket says 172.21.15.137:37560) 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.183+0000 7fd2d259c640 1 -- 172.21.15.137:0/1988373119 learned_addr learned my addr 172.21.15.137:0/1988373119 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.183+0000 7fd2d259c640 1 -- 172.21.15.137:0/1988373119 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d407ca70 con 0x7fd2d407de70 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.184+0000 7fd2d259c640 1 --2- 172.21.15.137:0/1988373119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d407c320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009820 tx=0x7fd2c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8432c06c2f25781 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.184+0000 7fd2d1d9b640 1 -- 172.21.15.137:0/1988373119 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c4037070 con 0x7fd2d407de70 2024-09-12T12:03:24.766 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.184+0000 7fd2d1d9b640 1 -- 172.21.15.137:0/1988373119 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4032440 con 0x7fd2d407de70 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.184+0000 7fd2d1d9b640 1 -- 172.21.15.137:0/1988373119 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c4032740 con 0x7fd2d407de70 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 -- 172.21.15.137:0/1988373119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 msgr2=0x7fd2d407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 --2- 172.21.15.137:0/1988373119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d407c320 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009820 tx=0x7fd2c402a270 comp rx=0 tx=0).stop 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 -- 172.21.15.137:0/1988373119 shutdown_connections 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 --2- 172.21.15.137:0/1988373119 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d407c320 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 -- 172.21.15.137:0/1988373119 >> 172.21.15.137:0/1988373119 conn(0x7fd2d4102ef0 msgr2=0x7fd2d4105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 -- 172.21.15.137:0/1988373119 shutdown_connections 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 -- 172.21.15.137:0/1988373119 wait complete. 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 Processor -- start 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.185+0000 7fd2d8d21640 1 -- start start 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d8d21640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:24.767 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d8d21640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d407d010 con 0x7fd2d407de70 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d419d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37572/0 (socket says 172.21.15.137:37572) 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d259c640 1 -- 172.21.15.137:0/486568315 learned_addr learned my addr 172.21.15.137:0/486568315 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d259c640 1 -- 172.21.15.137:0/486568315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d419d5c0 con 0x7fd2d407de70 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d259c640 1 --2- 172.21.15.137:0/486568315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d419d080 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40090b0 tx=0x7fd2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c4041070 con 0x7fd2d407de70 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c403c7a0 con 0x7fd2d407de70 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2c4037040 con 0x7fd2d407de70 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d419d850 con 0x7fd2d407de70 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.186+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d41a0540 con 0x7fd2d407de70 2024-09-12T12:03:24.768 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.187+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd2c4044070 con 0x7fd2d407de70 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.187+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd2c403d3f0 con 0x7fd2d407de70 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.187+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d407c3a0 con 0x7fd2d407de70 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.189+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd2c4046450 con 0x7fd2d407de70 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.247+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd2d402d910 con 0x7fd2d407de70 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.250+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7fd2c4046630 con 0x7fd2d407de70 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.251+0000 7fd2cb7fe640 1 -- 172.21.15.137:0/486568315 <== mon.0 v2:172.21.15.137:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd2c4056070 con 0x7fd2d407de70 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.251+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 msgr2=0x7fd2d419d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.251+0000 7fd2d8d21640 1 --2- 172.21.15.137:0/486568315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d419d080 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40090b0 tx=0x7fd2c40046c0 comp rx=0 tx=0).stop 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.251+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 shutdown_connections 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.251+0000 7fd2d8d21640 1 --2- 172.21.15.137:0/486568315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2d407de70 0x7fd2d419d080 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.251+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 >> 172.21.15.137:0/486568315 conn(0x7fd2d4102ef0 msgr2=0x7fd2d4103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:24.769 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.252+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 shutdown_connections 2024-09-12T12:03:24.770 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:24.252+0000 7fd2d8d21640 1 -- 172.21.15.137:0/486568315 wait complete. 2024-09-12T12:03:24.770 INFO:teuthology.orchestra.run.smithi137.stdout:Generating new minimal ceph.conf... 2024-09-12T12:03:25.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:25 smithi137 ceph-mon[30580]: from='client.? 172.21.15.137:0/1569981930' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:03:25.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.130+0000 7f23980b9640 1 Processor -- start 2024-09-12T12:03:25.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.130+0000 7f23980b9640 1 -- start start 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.130+0000 7f23980b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f2390107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.130+0000 7f23980b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f23901080c0 con 0x7f23901076f0 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.130+0000 7f2395e55640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f2390107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.130+0000 7f2395e55640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f2390107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37574/0 (socket says 172.21.15.137:37574) 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.130+0000 7f2395e55640 1 -- 172.21.15.137:0/536238983 learned_addr learned my addr 172.21.15.137:0/536238983 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.131+0000 7f2395e55640 1 -- 172.21.15.137:0/536238983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23901088f0 con 0x7f23901076f0 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.131+0000 7f2395e55640 1 --2- 172.21.15.137:0/536238983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f2390107af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f238c009820 tx=0x7f238c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be021549fda3af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.131+0000 7f2394e53640 1 -- 172.21.15.137:0/536238983 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f238c037070 con 0x7f23901076f0 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.131+0000 7f2394e53640 1 -- 172.21.15.137:0/536238983 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f238c032900 con 0x7f23901076f0 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.131+0000 7f2394e53640 1 -- 172.21.15.137:0/536238983 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f238c030850 con 0x7f23901076f0 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 -- 172.21.15.137:0/536238983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 msgr2=0x7f2390107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:25.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 --2- 172.21.15.137:0/536238983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f2390107af0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f238c009820 tx=0x7f238c02a270 comp rx=0 tx=0).stop 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 -- 172.21.15.137:0/536238983 shutdown_connections 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 --2- 172.21.15.137:0/536238983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f2390107af0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 -- 172.21.15.137:0/536238983 >> 172.21.15.137:0/536238983 conn(0x7f2390102ec0 msgr2=0x7f23901052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 -- 172.21.15.137:0/536238983 shutdown_connections 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 -- 172.21.15.137:0/536238983 wait complete. 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.132+0000 7f23980b9640 1 Processor -- start 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f23980b9640 1 -- start start 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f23980b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f239019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f23980b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f239010bca0 con 0x7f23901076f0 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2395e55640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f239019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2395e55640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f239019b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37584/0 (socket says 172.21.15.137:37584) 2024-09-12T12:03:25.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2395e55640 1 -- 172.21.15.137:0/1569981930 learned_addr learned my addr 172.21.15.137:0/1569981930 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2395e55640 1 -- 172.21.15.137:0/1569981930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239019bc20 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2395e55640 1 --2- 172.21.15.137:0/1569981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f239019b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f238c0097f0 tx=0x7f238c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2386ffd640 1 -- 172.21.15.137:0/1569981930 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f238c037040 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2386ffd640 1 -- 172.21.15.137:0/1569981930 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f238c040070 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239019beb0 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f2386ffd640 1 -- 172.21.15.137:0/1569981930 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f238c03a390 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.133+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239019eba0 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.134+0000 7f2386ffd640 1 -- 172.21.15.137:0/1569981930 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f238c03a630 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.134+0000 7f2386ffd640 1 -- 172.21.15.137:0/1569981930 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f238c0459a0 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.134+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2390107b70 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.136+0000 7f2386ffd640 1 -- 172.21.15.137:0/1569981930 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f238c045460 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.193+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f239019ed70 con 0x7f23901076f0 2024-09-12T12:03:25.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.194+0000 7f2386ffd640 1 -- 172.21.15.137:0/1569981930 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f238c03b430 con 0x7f23901076f0 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.194+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 msgr2=0x7f239019b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.194+0000 7f23980b9640 1 --2- 172.21.15.137:0/1569981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f239019b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f238c0097f0 tx=0x7f238c030d80 comp rx=0 tx=0).stop 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.194+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 shutdown_connections 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.194+0000 7f23980b9640 1 --2- 172.21.15.137:0/1569981930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23901076f0 0x7f239019b6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.194+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 >> 172.21.15.137:0/1569981930 conn(0x7f2390102ec0 msgr2=0x7f23901039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.195+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 shutdown_connections 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:25.195+0000 7f23980b9640 1 -- 172.21.15.137:0/1569981930 wait complete. 2024-09-12T12:03:25.820 INFO:teuthology.orchestra.run.smithi137.stdout:Restarting the monitor... 2024-09-12T12:03:26.080 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:25 smithi137 systemd[1]: Stopping Ceph mon.a for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:03:26.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a[30576]: 2024-09-12T12:03:26.079+0000 7f9ad6698640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:03:26.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a[30576]: 2024-09-12T12:03:26.079+0000 7f9ad6698640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T12:03:26.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 podman[30827]: 2024-09-12 12:03:26.141325604 +0000 UTC m=+0.164229860 container died c78bf4c50ef981622da973f9426b09eecfa679c5bf218cd742928d6b6dd21169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T12:03:26.771 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 podman[30827]: 2024-09-12 12:03:26.521187965 +0000 UTC m=+0.544092226 container cleanup c78bf4c50ef981622da973f9426b09eecfa679c5bf218cd742928d6b6dd21169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T12:03:26.771 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 bash[30827]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a 2024-09-12T12:03:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 podman[30841]: 2024-09-12 12:03:26.772124072 +0000 UTC m=+0.623042048 container remove c78bf4c50ef981622da973f9426b09eecfa679c5bf218cd742928d6b6dd21169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:03:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 systemd[1]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T12:03:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 systemd[1]: Stopped Ceph mon.a for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:03:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:26 smithi137 systemd[1]: Starting Ceph mon.a for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:03:27.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 podman[30932]: 2024-09-12 12:03:27.008844517 +0000 UTC m=+0.015967150 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T12:03:27.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 podman[30932]: 2024-09-12 12:03:27.114553004 +0000 UTC m=+0.121675641 container create f9bc5e1ab701817bb623b06976269668ae1f98e19277476c5080fd57b4f4b94a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T12:03:27.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 podman[30932]: 2024-09-12 12:03:27.333349722 +0000 UTC m=+0.340472358 container init f9bc5e1ab701817bb623b06976269668ae1f98e19277476c5080fd57b4f4b94a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T12:03:27.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 podman[30932]: 2024-09-12 12:03:27.336658986 +0000 UTC m=+0.343781620 container start f9bc5e1ab701817bb623b06976269668ae1f98e19277476c5080fd57b4f4b94a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T12:03:27.454 INFO:teuthology.orchestra.run.smithi137.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T12:03:27.661 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T12:03:27.661 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T12:03:27.661 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: pidfile_write: ignore empty --pid-file 2024-09-12T12:03:27.661 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: load: jerasure load: lrc 2024-09-12T12:03:27.661 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: RocksDB version: 7.9.2 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Git sha 0 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: DB SUMMARY 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: DB Session ID: HBCT4CUJI46T5JRFUKNI 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: CURRENT file: CURRENT 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105844 ; 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.error_if_exists: 0 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.create_if_missing: 0 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.paranoid_checks: 1 2024-09-12T12:03:27.662 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.env: 0x55b64e200940 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.info_log: 0x55b64f9836e0 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.statistics: (nil) 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.use_fsync: 0 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_log_file_size: 0 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T12:03:27.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.allow_fallocate: 1 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.use_direct_reads: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.db_log_dir: 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.wal_dir: 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T12:03:27.664 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.write_buffer_manager: 0x55b64f992dc0 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T12:03:27.665 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.unordered_write: 0 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.row_cache: None 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.wal_filter: None 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T12:03:27.666 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.two_write_queues: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.wal_compression: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.atomic_flush: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.log_readahead_size: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T12:03:27.667 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_background_jobs: 2 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_background_compactions: -1 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_subcompactions: 1 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T12:03:27.669 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_open_files: -1 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_background_flushes: -1 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Compression algorithms supported: 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kZSTD supported: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kXpressCompression supported: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kBZip2Compression supported: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T12:03:27.670 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kLZ4Compression supported: 1 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kZlibCompression supported: 1 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: kSnappyCompression supported: 1 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.merge_operator: 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_filter: None 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T12:03:27.671 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b64f983320) 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: index_type: 0 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: data_block_index_type: 0 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: index_shortening: 1 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T12:03:27.672 INFO:journalctl@ceph.mon.a.smithi137.stdout: checksum: 4 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: no_block_cache: 0 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache: 0x55b64f9771f0 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache_options: 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: capacity : 536870912 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: num_shard_bits : 4 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: strict_capacity_limit : 0 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache_compressed: (nil) 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: persistent_cache: (nil) 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_size: 4096 2024-09-12T12:03:27.673 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_size_deviation: 10 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_restart_interval: 16 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: index_block_restart_interval: 1 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: metadata_block_size: 4096 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: partition_filters: 0 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: use_delta_encoding: 1 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: filter_policy: bloomfilter 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: whole_key_filtering: 1 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: verify_compression: 0 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: format_version: 5 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: enable_index_compression: 1 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_align: 0 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: max_auto_readahead_size: 262144 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: prepopulate_block_cache: 0 2024-09-12T12:03:27.674 INFO:journalctl@ceph.mon.a.smithi137.stdout: initial_auto_readahead_size: 8192 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression: NoCompression 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.num_levels: 7 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T12:03:27.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T12:03:27.676 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T12:03:27.677 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T12:03:27.678 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.inplace_update_support: 0 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T12:03:27.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.bloom_locality: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.max_successive_merges: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.ttl: 2592000 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.enable_blob_files: false 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.min_blob_size: 0 2024-09-12T12:03:27.680 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 76447eae-d5de-4eb3-8172-65d4b71381c5 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726142607362853, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726142607363807, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100855, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98740, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14423, "raw_average_key_size": 49, "raw_value_size": 90946, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726142607, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "76447eae-d5de-4eb3-8172-65d4b71381c5", "db_session_id": "HBCT4CUJI46T5JRFUKNI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T12:03:27.681 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726142607363903, "job": 1, "event": "recovery_finished"} 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b64f9a4e00 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: DB pointer 0x55b64faa0000 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** DB Stats ** 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T12:03:27.682 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** Compaction Stats [default] ** 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: L0 2/0 100.35 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: Sum 2/0 100.35 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** Compaction Stats [default] ** 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 167.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T12:03:27.683 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative compaction: 0.00 GB write, 31.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval compaction: 0.00 GB write, 31.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: Block cache BinnedLRUCache@0x55b64f9771f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-09-12T12:03:27.684 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: starting mon.a rank 0 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???) e1 preinit fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???).mds e1 new map 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???).mds e1 print_map 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout: e1 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout: btime 2024-09-12T12:03:22:696421+0000 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout: legacy client fscid: -1 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout: No filesystems configured 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T12:03:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: monmap epoch 1 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: last_changed 2024-09-12T12:03:19.533021+0000 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: election_strategy: 1 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: fsmap 2024-09-12T12:03:27.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T12:03:27.687 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 ceph-mon[30950]: mgrmap e1: no daemons active 2024-09-12T12:03:27.687 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 bash[30932]: f9bc5e1ab701817bb623b06976269668ae1f98e19277476c5080fd57b4f4b94a 2024-09-12T12:03:27.687 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:27 smithi137 systemd[1]: Started Ceph mon.a for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:03:28.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.043+0000 7f43c96b6640 1 Processor -- start 2024-09-12T12:03:28.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.043+0000 7f43c96b6640 1 -- start start 2024-09-12T12:03:28.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.043+0000 7f43c96b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c4106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:28.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.043+0000 7f43c96b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4106fe0 con 0x7f43c4106610 2024-09-12T12:03:28.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.043+0000 7f43c2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c4106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:28.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.043+0000 7f43c2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c4106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37596/0 (socket says 172.21.15.137:37596) 2024-09-12T12:03:28.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.043+0000 7f43c2ffd640 1 -- 172.21.15.137:0/3702527887 learned_addr learned my addr 172.21.15.137:0/3702527887 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.044+0000 7f43c2ffd640 1 -- 172.21.15.137:0/3702527887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c4107810 con 0x7f43c4106610 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.045+0000 7f43c2ffd640 1 --2- 172.21.15.137:0/3702527887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c4106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009820 tx=0x7f43b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51664850dec8882 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.045+0000 7f43c27fc640 1 -- 172.21.15.137:0/3702527887 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4037070 con 0x7f43c4106610 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c27fc640 1 -- 172.21.15.137:0/3702527887 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f43b4032900 con 0x7f43c4106610 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c27fc640 1 -- 172.21.15.137:0/3702527887 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4030850 con 0x7f43c4106610 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c96b6640 1 -- 172.21.15.137:0/3702527887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 msgr2=0x7f43c4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c96b6640 1 --2- 172.21.15.137:0/3702527887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c4106a10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009820 tx=0x7f43b402a270 comp rx=0 tx=0).stop 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c96b6640 1 -- 172.21.15.137:0/3702527887 shutdown_connections 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c96b6640 1 --2- 172.21.15.137:0/3702527887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c4106a10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c96b6640 1 -- 172.21.15.137:0/3702527887 >> 172.21.15.137:0/3702527887 conn(0x7f43c4101da0 msgr2=0x7f43c41041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c96b6640 1 -- 172.21.15.137:0/3702527887 shutdown_connections 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.046+0000 7f43c96b6640 1 -- 172.21.15.137:0/3702527887 wait complete. 2024-09-12T12:03:28.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.047+0000 7f43c96b6640 1 Processor -- start 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.047+0000 7f43c96b6640 1 -- start start 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.047+0000 7f43c96b6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.047+0000 7f43c96b6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f43c410abc0 con 0x7f43c4106610 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.047+0000 7f43c2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.047+0000 7f43c2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37600/0 (socket says 172.21.15.137:37600) 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.047+0000 7f43c2ffd640 1 -- 172.21.15.137:0/4183057638 learned_addr learned my addr 172.21.15.137:0/4183057638 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c2ffd640 1 -- 172.21.15.137:0/4183057638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c419aba0 con 0x7f43c4106610 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c2ffd640 1 --2- 172.21.15.137:0/4183057638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c419a660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009940 tx=0x7f43b40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c0ff9640 1 -- 172.21.15.137:0/4183057638 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4042070 con 0x7f43c4106610 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c0ff9640 1 -- 172.21.15.137:0/4183057638 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f43b4032be0 con 0x7f43c4106610 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c419ae30 con 0x7f43c4106610 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c0ff9640 1 -- 172.21.15.137:0/4183057638 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4037040 con 0x7f43c4106610 2024-09-12T12:03:28.616 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c419db20 con 0x7f43c4106610 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c0ff9640 1 -- 172.21.15.137:0/4183057638 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f43b4045070 con 0x7f43c4106610 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.048+0000 7f43c0ff9640 1 -- 172.21.15.137:0/4183057638 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f43b4047a80 con 0x7f43c4106610 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.049+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c410ad40 con 0x7f43c4106610 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.051+0000 7f43c0ff9640 1 -- 172.21.15.137:0/4183057638 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f43b4032d90 con 0x7f43c4106610 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.108+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f43c4106a90 con 0x7f43c4106610 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.110+0000 7f43c0ff9640 1 -- 172.21.15.137:0/4183057638 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f43b403d430 con 0x7f43c4106610 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.111+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 msgr2=0x7f43c419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.111+0000 7f43c96b6640 1 --2- 172.21.15.137:0/4183057638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c419a660 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009940 tx=0x7f43b40326e0 comp rx=0 tx=0).stop 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.111+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 shutdown_connections 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.111+0000 7f43c96b6640 1 --2- 172.21.15.137:0/4183057638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43c4106610 0x7f43c419a660 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:28.617 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.111+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 >> 172.21.15.137:0/4183057638 conn(0x7f43c4101da0 msgr2=0x7f43c4102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:28.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.111+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 shutdown_connections 2024-09-12T12:03:28.618 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:28.111+0000 7f43c96b6640 1 -- 172.21.15.137:0/4183057638 wait complete. 2024-09-12T12:03:28.618 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T12:03:28.631 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:03:28.632 INFO:teuthology.orchestra.run.smithi137.stdout:Creating mgr... 2024-09-12T12:03:28.632 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T12:03:28.632 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T12:03:28.889 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x 2024-09-12T12:03:28.890 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T12:03:29.039 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dd66268a-70fe-11ef-bcea-c7b262605968.target.wants/ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-dd66268a-70fe-11ef-bcea-c7b262605968@.service. 2024-09-12T12:03:29.386 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:29 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4183057638' entity='client.admin' 2024-09-12T12:03:29.559 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T12:03:29.559 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-09-12T12:03:29.566 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T12:03:29.566 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-09-12T12:03:29.566 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-09-12T12:03:29.566 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T12:03:29.575 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T12:03:29.575 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-09-12T12:03:29.582 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T12:03:29.582 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-09-12T12:03:29.582 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-09-12T12:03:29.583 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T12:03:29.583 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr to start... 2024-09-12T12:03:29.583 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr... 2024-09-12T12:03:30.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:30 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2982657863' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T12:03:30.299 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:30 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:30.194+0000 7ff879516040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:03:30.726 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:30 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:30.514+0000 7ff879516040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:03:30.726 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:30 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:30.576+0000 7ff879516040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:03:30.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:30.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-09-12T12:03:30.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:03:30.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T12:03:30.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T12:03:30.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T12:03:30.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T12:03:30.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T12:03:30.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:30.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T12:03:30.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:30.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T12:03:30.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T12:03:30.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T12:03:30.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T12:03:30.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T12:03:30.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T12:03:30.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:30.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T12:03:30.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T12:03:30.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T12:03:22:696421+0000", 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T12:03:30.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T12:03:22.697055+0000", 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T12:03:30.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.009+0000 7fb0eff07640 1 Processor -- start 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.010+0000 7fb0eff07640 1 -- start start 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.010+0000 7fb0eff07640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e810a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.010+0000 7fb0eff07640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e807beb0 con 0x7fb0e8108160 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.010+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e810a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.010+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e810a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40042/0 (socket says 172.21.15.137:40042) 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.010+0000 7fb0edca3640 1 -- 172.21.15.137:0/418407688 learned_addr learned my addr 172.21.15.137:0/418407688 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.010+0000 7fb0edca3640 1 -- 172.21.15.137:0/418407688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e810aa90 con 0x7fb0e8108160 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.011+0000 7fb0edca3640 1 --2- 172.21.15.137:0/418407688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e810a550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009820 tx=0x7fb0d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc608b327b4cabe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:30.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.011+0000 7fb0ecca1640 1 -- 172.21.15.137:0/418407688 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d8037070 con 0x7fb0e8108160 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.011+0000 7fb0ecca1640 1 -- 172.21.15.137:0/418407688 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0d8032900 con 0x7fb0e8108160 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.011+0000 7fb0ecca1640 1 -- 172.21.15.137:0/418407688 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d8030850 con 0x7fb0e8108160 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.012+0000 7fb0eff07640 1 -- 172.21.15.137:0/418407688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 msgr2=0x7fb0e810a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.012+0000 7fb0eff07640 1 --2- 172.21.15.137:0/418407688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e810a550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009820 tx=0x7fb0d802a270 comp rx=0 tx=0).stop 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.012+0000 7fb0eff07640 1 -- 172.21.15.137:0/418407688 shutdown_connections 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.012+0000 7fb0eff07640 1 --2- 172.21.15.137:0/418407688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e810a550 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.012+0000 7fb0eff07640 1 -- 172.21.15.137:0/418407688 >> 172.21.15.137:0/418407688 conn(0x7fb0e8101e20 msgr2=0x7fb0e8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.012+0000 7fb0eff07640 1 -- 172.21.15.137:0/418407688 shutdown_connections 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.012+0000 7fb0eff07640 1 -- 172.21.15.137:0/418407688 wait complete. 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0eff07640 1 Processor -- start 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0eff07640 1 -- start start 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0eff07640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:30.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0eff07640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810bb30 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0edca3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40058/0 (socket says 172.21.15.137:40058) 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0edca3640 1 -- 172.21.15.137:0/2982657863 learned_addr learned my addr 172.21.15.137:0/2982657863 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0edca3640 1 -- 172.21.15.137:0/2982657863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e819de20 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.013+0000 7fb0edca3640 1 --2- 172.21.15.137:0/2982657863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e819d8e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8031000 tx=0x7fb0d80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.014+0000 7fb0deffd640 1 -- 172.21.15.137:0/2982657863 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d8042070 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.014+0000 7fb0deffd640 1 -- 172.21.15.137:0/2982657863 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0d8032be0 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.014+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e819e0b0 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.014+0000 7fb0deffd640 1 -- 172.21.15.137:0/2982657863 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0d8037040 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.014+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a0da0 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.014+0000 7fb0deffd640 1 -- 172.21.15.137:0/2982657863 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb0d8045070 con 0x7fb0e8108160 2024-09-12T12:03:30.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.014+0000 7fb0deffd640 1 -- 172.21.15.137:0/2982657863 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb0d8047780 con 0x7fb0e8108160 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.015+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e81085c0 con 0x7fb0e8108160 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.017+0000 7fb0deffd640 1 -- 172.21.15.137:0/2982657863 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb0d80328a0 con 0x7fb0e8108160 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.078+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb0e81a1310 con 0x7fb0e8108160 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.079+0000 7fb0deffd640 1 -- 172.21.15.137:0/2982657863 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fb0d803ce10 con 0x7fb0e8108160 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.080+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 msgr2=0x7fb0e819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.080+0000 7fb0eff07640 1 --2- 172.21.15.137:0/2982657863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e819d8e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8031000 tx=0x7fb0d80326e0 comp rx=0 tx=0).stop 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.080+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 shutdown_connections 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.080+0000 7fb0eff07640 1 --2- 172.21.15.137:0/2982657863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0e8108160 0x7fb0e819d8e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.080+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 >> 172.21.15.137:0/2982657863 conn(0x7fb0e8101e20 msgr2=0x7fb0e8102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.080+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 shutdown_connections 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:30.080+0000 7fb0eff07640 1 -- 172.21.15.137:0/2982657863 wait complete. 2024-09-12T12:03:30.747 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (1/15)... 2024-09-12T12:03:31.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:30 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:30.765+0000 7ff879516040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:03:32.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:31 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:31.636+0000 7ff879516040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:03:32.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:31 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:31.704+0000 7ff879516040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:03:32.534 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:32 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:32.420+0000 7ff879516040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:03:32.535 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:32 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:32.534+0000 7ff879516040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:03:32.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:32 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:32.677+0000 7ff879516040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:03:32.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:32 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:32.738+0000 7ff879516040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:03:33.160 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:32 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:32.902+0000 7ff879516040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:03:33.161 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:32 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:32.969+0000 7ff879516040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:03:33.161 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:33 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:33.029+0000 7ff879516040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:03:33.462 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:33 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:33.189+0000 7ff879516040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:03:33.462 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:33 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:33.254+0000 7ff879516040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:03:33.462 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:33 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:33.319+0000 7ff879516040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:03:33.462 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:33 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:33.402+0000 7ff879516040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:03:33.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:33 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:33.461+0000 7ff879516040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:03:33.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:33 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:33.520+0000 7ff879516040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:03:34.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:34.321+0000 7ff879516040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:03:34.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: Activating manager daemon x 2024-09-12T12:03:34.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: mgrmap e2: x(active, starting, since 0.00293251s) 2024-09-12T12:03:34.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T12:03:34.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T12:03:34.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: Manager daemon x is now available 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' 2024-09-12T12:03:34.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:34 smithi137 ceph-mon[30950]: from='mgr.14100 172.21.15.137:0/1527187202' entity='mgr.x' 2024-09-12T12:03:35.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:35 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3696732400' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T12:03:35.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:35 smithi137 ceph-mon[30950]: mgrmap e3: x(active, since 1.00605s) 2024-09-12T12:03:37.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:37 smithi137 ceph-mon[30950]: mgrmap e4: x(active, since 2s) 2024-09-12T12:03:41.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T12:03:41.891 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T12:03:41.892 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T12:03:41.893 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T12:03:22:696421+0000", 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T12:03:41.894 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T12:03:22.697055+0000", 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T12:03:41.895 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.104+0000 7fb781ff0640 1 Processor -- start 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.104+0000 7fb781ff0640 1 -- start start 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.104+0000 7fb781ff0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c10a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.104+0000 7fb781ff0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c07be70 con 0x7fb77c1081a0 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.105+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c10a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.105+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c10a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40118/0 (socket says 172.21.15.137:40118) 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.105+0000 7fb77b7fe640 1 -- 172.21.15.137:0/1163120299 learned_addr learned my addr 172.21.15.137:0/1163120299 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.105+0000 7fb77b7fe640 1 -- 172.21.15.137:0/1163120299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c10aad0 con 0x7fb77c1081a0 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.106+0000 7fb77b7fe640 1 --2- 172.21.15.137:0/1163120299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c10a590 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009820 tx=0x7fb76c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b768778f7ba89e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.106+0000 7fb77a7fc640 1 -- 172.21.15.137:0/1163120299 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c037070 con 0x7fb77c1081a0 2024-09-12T12:03:41.896 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.106+0000 7fb77a7fc640 1 -- 172.21.15.137:0/1163120299 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb76c032900 con 0x7fb77c1081a0 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.106+0000 7fb77a7fc640 1 -- 172.21.15.137:0/1163120299 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c030850 con 0x7fb77c1081a0 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.106+0000 7fb781ff0640 1 -- 172.21.15.137:0/1163120299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 msgr2=0x7fb77c10a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.106+0000 7fb781ff0640 1 --2- 172.21.15.137:0/1163120299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c10a590 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009820 tx=0x7fb76c02a270 comp rx=0 tx=0).stop 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.107+0000 7fb781ff0640 1 -- 172.21.15.137:0/1163120299 shutdown_connections 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.107+0000 7fb781ff0640 1 --2- 172.21.15.137:0/1163120299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c10a590 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.107+0000 7fb781ff0640 1 -- 172.21.15.137:0/1163120299 >> 172.21.15.137:0/1163120299 conn(0x7fb77c101e20 msgr2=0x7fb77c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.107+0000 7fb781ff0640 1 -- 172.21.15.137:0/1163120299 shutdown_connections 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.107+0000 7fb781ff0640 1 -- 172.21.15.137:0/1163120299 wait complete. 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.107+0000 7fb781ff0640 1 Processor -- start 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.107+0000 7fb781ff0640 1 -- start start 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb781ff0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c107a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb781ff0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c10bb10 con 0x7fb77c1081a0 2024-09-12T12:03:41.897 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb77b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40130/0 (socket says 172.21.15.137:40130) 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb77b7fe640 1 -- 172.21.15.137:0/3696732400 learned_addr learned my addr 172.21.15.137:0/3696732400 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb77b7fe640 1 -- 172.21.15.137:0/3696732400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c107f60 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb77b7fe640 1 --2- 172.21.15.137:0/3696732400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c107a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb76c031000 tx=0x7fb76c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb778ff9640 1 -- 172.21.15.137:0/3696732400 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c042070 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb778ff9640 1 -- 172.21.15.137:0/3696732400 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb76c032be0 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb778ff9640 1 -- 172.21.15.137:0/3696732400 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb76c037040 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c106150 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.108+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c106690 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.109+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb77c108600 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.109+0000 7fb778ff9640 1 -- 172.21.15.137:0/3696732400 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 2) ==== 46043+0+0 (secure 0 0 0) 0x7fb76c045070 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.110+0000 7fb778ff9640 1 -- 172.21.15.137:0/3696732400 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb76c06bca0 con 0x7fb77c1081a0 2024-09-12T12:03:41.898 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.112+0000 7fb778ff9640 1 -- 172.21.15.137:0/3696732400 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb76c03c770 con 0x7fb77c1081a0 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.174+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb77c106fe0 con 0x7fb77c1081a0 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.174+0000 7fb778ff9640 1 -- 172.21.15.137:0/3696732400 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fb77c106fe0 con 0x7fb77c1081a0 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.175+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 msgr2=0x7fb77c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.175+0000 7fb781ff0640 1 --2- 172.21.15.137:0/3696732400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c107a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb76c031000 tx=0x7fb76c0326e0 comp rx=0 tx=0).stop 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.175+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 shutdown_connections 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.175+0000 7fb781ff0640 1 --2- 172.21.15.137:0/3696732400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb77c1081a0 0x7fb77c107a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.175+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 >> 172.21.15.137:0/3696732400 conn(0x7fb77c101e20 msgr2=0x7fb77c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.175+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 shutdown_connections 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:35.175+0000 7fb781ff0640 1 -- 172.21.15.137:0/3696732400 wait complete. 2024-09-12T12:03:41.899 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (2/15)... 2024-09-12T12:03:45.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:45 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2542435234' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T12:03:45.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T12:03:45.595 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:45.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T12:03:45.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T12:03:45.596 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T12:03:45.603 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T12:03:45.604 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T12:03:22:696421+0000", 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T12:03:45.605 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T12:03:22.697055+0000", 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T12:03:45.606 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.714+0000 7ffb6f5c0640 1 Processor -- start 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.714+0000 7ffb6f5c0640 1 -- start start 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.714+0000 7ffb6f5c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb68107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.714+0000 7ffb6f5c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb681080c0 con 0x7ffb681076f0 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.714+0000 7ffb6d35c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb68107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.714+0000 7ffb6d35c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb68107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47726/0 (socket says 172.21.15.137:47726) 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.714+0000 7ffb6d35c640 1 -- 172.21.15.137:0/930723575 learned_addr learned my addr 172.21.15.137:0/930723575 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.715+0000 7ffb6d35c640 1 -- 172.21.15.137:0/930723575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb681088f0 con 0x7ffb681076f0 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.715+0000 7ffb6d35c640 1 --2- 172.21.15.137:0/930723575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb68107af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009820 tx=0x7ffb6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2dd771ae7ffd90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:45.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb5ffff640 1 -- 172.21.15.137:0/930723575 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb64037070 con 0x7ffb681076f0 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb5ffff640 1 -- 172.21.15.137:0/930723575 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb64032900 con 0x7ffb681076f0 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb5ffff640 1 -- 172.21.15.137:0/930723575 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb64030850 con 0x7ffb681076f0 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/930723575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 msgr2=0x7ffb68107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb6f5c0640 1 --2- 172.21.15.137:0/930723575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb68107af0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009820 tx=0x7ffb6402a270 comp rx=0 tx=0).stop 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/930723575 shutdown_connections 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb6f5c0640 1 --2- 172.21.15.137:0/930723575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb68107af0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/930723575 >> 172.21.15.137:0/930723575 conn(0x7ffb68102ec0 msgr2=0x7ffb681052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/930723575 shutdown_connections 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.716+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/930723575 wait complete. 2024-09-12T12:03:45.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.717+0000 7ffb6f5c0640 1 Processor -- start 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.717+0000 7ffb6f5c0640 1 -- start start 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.717+0000 7ffb6f5c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb6819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.717+0000 7ffb6f5c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810bca0 con 0x7ffb681076f0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.717+0000 7ffb6d35c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb6819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.717+0000 7ffb6d35c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb6819d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47730/0 (socket says 172.21.15.137:47730) 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.717+0000 7ffb6d35c640 1 -- 172.21.15.137:0/2542435234 learned_addr learned my addr 172.21.15.137:0/2542435234 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.718+0000 7ffb6d35c640 1 -- 172.21.15.137:0/2542435234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6819de50 con 0x7ffb681076f0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.718+0000 7ffb6d35c640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb6819d910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb640097f0 tx=0x7ffb64030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.718+0000 7ffb5e7fc640 1 -- 172.21.15.137:0/2542435234 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb64037040 con 0x7ffb681076f0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.718+0000 7ffb5e7fc640 1 -- 172.21.15.137:0/2542435234 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb64040070 con 0x7ffb681076f0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.718+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6819e0e0 con 0x7ffb681076f0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.718+0000 7ffb5e7fc640 1 -- 172.21.15.137:0/2542435234 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb6403a390 con 0x7ffb681076f0 2024-09-12T12:03:45.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.718+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb681a0dd0 con 0x7ffb681076f0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.719+0000 7ffb5e7fc640 1 -- 172.21.15.137:0/2542435234 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ffb6403a630 con 0x7ffb681076f0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.719+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb68107b70 con 0x7ffb681076f0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.719+0000 7ffb5e7fc640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ffb54038f20 0x7ffb5403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.719+0000 7ffb5e7fc640 1 -- 172.21.15.137:0/2542435234 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb6406c250 con 0x7ffb681076f0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.719+0000 7ffb6cb5b640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ffb54038f20 0x7ffb5403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.720+0000 7ffb6cb5b640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ffb54038f20 0x7ffb5403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098c0 tx=0x7ffb58006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.724+0000 7ffb5e7fc640 1 -- 172.21.15.137:0/2542435234 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb6403a9d0 con 0x7ffb681076f0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.952+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffb6819e560 con 0x7ffb681076f0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.953+0000 7ffb5e7fc640 1 -- 172.21.15.137:0/2542435234 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ffb6406bfb0 con 0x7ffb681076f0 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ffb54038f20 msgr2=0x7ffb5403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ffb54038f20 0x7ffb5403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffb580098c0 tx=0x7ffb58006db0 comp rx=0 tx=0).stop 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 msgr2=0x7ffb6819d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb6819d910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffb640097f0 tx=0x7ffb64030d80 comp rx=0 tx=0).stop 2024-09-12T12:03:45.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 shutdown_connections 2024-09-12T12:03:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ffb54038f20 0x7ffb5403b3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 --2- 172.21.15.137:0/2542435234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb681076f0 0x7ffb6819d910 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 >> 172.21.15.137:0/2542435234 conn(0x7ffb68102ec0 msgr2=0x7ffb68103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 shutdown_connections 2024-09-12T12:03:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:44.955+0000 7ffb6f5c0640 1 -- 172.21.15.137:0/2542435234 wait complete. 2024-09-12T12:03:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:mgr is available 2024-09-12T12:03:46.453 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:46 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4216540638' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T12:03:46.721 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:46.721 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T12:03:46.721 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout fsid = dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:46.721 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.137:3300,v1:172.21.15.137:6789] 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T12:03:46.722 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.977+0000 7f9ddc556640 1 Processor -- start 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.977+0000 7f9ddc556640 1 -- start start 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.978+0000 7f9ddc556640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd41069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.978+0000 7f9ddc556640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4106fc0 con 0x7f9dd41065f0 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.978+0000 7f9dda2f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd41069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.978+0000 7f9dda2f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd41069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47734/0 (socket says 172.21.15.137:47734) 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.978+0000 7f9dda2f2640 1 -- 172.21.15.137:0/3690971145 learned_addr learned my addr 172.21.15.137:0/3690971145 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.978+0000 7f9dda2f2640 1 -- 172.21.15.137:0/3690971145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd41077f0 con 0x7f9dd41065f0 2024-09-12T12:03:46.723 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.979+0000 7f9dda2f2640 1 --2- 172.21.15.137:0/3690971145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd41069f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009820 tx=0x7f9dc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ad41409a7982f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.979+0000 7f9dd92f0640 1 -- 172.21.15.137:0/3690971145 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc8037070 con 0x7f9dd41065f0 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.979+0000 7f9dd92f0640 1 -- 172.21.15.137:0/3690971145 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9dc8032900 con 0x7f9dd41065f0 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.979+0000 7f9dd92f0640 1 -- 172.21.15.137:0/3690971145 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc8030850 con 0x7f9dd41065f0 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.979+0000 7f9ddc556640 1 -- 172.21.15.137:0/3690971145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 msgr2=0x7f9dd41069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.979+0000 7f9ddc556640 1 --2- 172.21.15.137:0/3690971145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd41069f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009820 tx=0x7f9dc802a270 comp rx=0 tx=0).stop 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 -- 172.21.15.137:0/3690971145 shutdown_connections 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 --2- 172.21.15.137:0/3690971145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd41069f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 -- 172.21.15.137:0/3690971145 >> 172.21.15.137:0/3690971145 conn(0x7f9dd4101e20 msgr2=0x7f9dd4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 -- 172.21.15.137:0/3690971145 shutdown_connections 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 -- 172.21.15.137:0/3690971145 wait complete. 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 Processor -- start 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 -- start start 2024-09-12T12:03:46.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9ddc556640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd410aba0 con 0x7f9dd41065f0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9dda2f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9dda2f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47738/0 (socket says 172.21.15.137:47738) 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.980+0000 7f9dda2f2640 1 -- 172.21.15.137:0/4216540638 learned_addr learned my addr 172.21.15.137:0/4216540638 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.981+0000 7f9dda2f2640 1 -- 172.21.15.137:0/4216540638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd419ab60 con 0x7f9dd41065f0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.981+0000 7f9dda2f2640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd419a620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80097f0 tx=0x7f9dc8030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.981+0000 7f9dc77fe640 1 -- 172.21.15.137:0/4216540638 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc8042070 con 0x7f9dd41065f0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.981+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd419adf0 con 0x7f9dd41065f0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.981+0000 7f9dc77fe640 1 -- 172.21.15.137:0/4216540638 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9dc8032ca0 con 0x7f9dd41065f0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.981+0000 7f9dc77fe640 1 -- 172.21.15.137:0/4216540638 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dc8037040 con 0x7f9dd41065f0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.981+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd419dae0 con 0x7f9dd41065f0 2024-09-12T12:03:46.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.982+0000 7f9dc77fe640 1 -- 172.21.15.137:0/4216540638 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f9dc8045070 con 0x7f9dd41065f0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.982+0000 7f9dc77fe640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7f9dc0038f20 0x7f9dc003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.982+0000 7f9dc77fe640 1 -- 172.21.15.137:0/4216540638 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9dc806be60 con 0x7f9dd41065f0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.982+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd410ad20 con 0x7f9dd41065f0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.982+0000 7f9dd9af1640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7f9dc0038f20 0x7f9dc003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.982+0000 7f9dd9af1640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7f9dc0038f20 0x7f9dc003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00098c0 tx=0x7f9dd0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:45.987+0000 7f9dc77fe640 1 -- 172.21.15.137:0/4216540638 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9dc8066070 con 0x7f9dd41065f0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.150+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9dd4106a70 con 0x7f9dd41065f0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.151+0000 7f9dc77fe640 1 -- 172.21.15.137:0/4216540638 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f9dc804b170 con 0x7f9dd41065f0 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7f9dc0038f20 msgr2=0x7f9dc003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7f9dc0038f20 0x7f9dc003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00098c0 tx=0x7f9dd0006db0 comp rx=0 tx=0).stop 2024-09-12T12:03:46.726 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 msgr2=0x7f9dd419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd419a620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80097f0 tx=0x7f9dc8030ec0 comp rx=0 tx=0).stop 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 shutdown_connections 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7f9dc0038f20 0x7f9dc003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 --2- 172.21.15.137:0/4216540638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd41065f0 0x7f9dd419a620 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.153+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 >> 172.21.15.137:0/4216540638 conn(0x7f9dd4101e20 msgr2=0x7f9dd4102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.154+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 shutdown_connections 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:46.154+0000 7f9ddc556640 1 -- 172.21.15.137:0/4216540638 wait complete. 2024-09-12T12:03:46.727 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling cephadm module... 2024-09-12T12:03:47.722 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/781535926' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T12:03:47.723 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ignoring --setuser ceph since I am not root 2024-09-12T12:03:47.723 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ignoring --setgroup ceph since I am not root 2024-09-12T12:03:47.723 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:47.519+0000 7f7008b73040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:03:47.723 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:47.575+0000 7f7008b73040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:03:47.723 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:47.722+0000 7f7008b73040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:03:48.026 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:47.790+0000 7f7008b73040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:03:48.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:47.938+0000 7f7008b73040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:03:48.034 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.105+0000 7fac42b83640 1 Processor -- start 2024-09-12T12:03:48.034 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.105+0000 7fac42b83640 1 -- start start 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.106+0000 7fac42b83640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.106+0000 7fac42b83640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1080f0 con 0x7fac3c107720 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.106+0000 7fac4091f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.106+0000 7fac4091f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47746/0 (socket says 172.21.15.137:47746) 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.106+0000 7fac4091f640 1 -- 172.21.15.137:0/153100057 learned_addr learned my addr 172.21.15.137:0/153100057 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.106+0000 7fac4091f640 1 -- 172.21.15.137:0/153100057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c1088d0 con 0x7fac3c107720 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.107+0000 7fac4091f640 1 --2- 172.21.15.137:0/153100057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c107b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac30009820 tx=0x7fac3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f92f466538d885b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.107+0000 7fac3b7fe640 1 -- 172.21.15.137:0/153100057 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac30037070 con 0x7fac3c107720 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.107+0000 7fac3b7fe640 1 -- 172.21.15.137:0/153100057 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac30032900 con 0x7fac3c107720 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.107+0000 7fac3b7fe640 1 -- 172.21.15.137:0/153100057 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac30030850 con 0x7fac3c107720 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.107+0000 7fac42b83640 1 -- 172.21.15.137:0/153100057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 msgr2=0x7fac3c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:48.035 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.107+0000 7fac42b83640 1 --2- 172.21.15.137:0/153100057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c107b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fac30009820 tx=0x7fac3002a270 comp rx=0 tx=0).stop 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 -- 172.21.15.137:0/153100057 shutdown_connections 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 --2- 172.21.15.137:0/153100057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c107b20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 -- 172.21.15.137:0/153100057 >> 172.21.15.137:0/153100057 conn(0x7fac3c102ef0 msgr2=0x7fac3c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 -- 172.21.15.137:0/153100057 shutdown_connections 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 -- 172.21.15.137:0/153100057 wait complete. 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 Processor -- start 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 -- start start 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac42b83640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c10bc80 con 0x7fac3c107720 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac4091f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac4091f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c19b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47752/0 (socket says 172.21.15.137:47752) 2024-09-12T12:03:48.036 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.108+0000 7fac4091f640 1 -- 172.21.15.137:0/781535926 learned_addr learned my addr 172.21.15.137:0/781535926 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.109+0000 7fac4091f640 1 -- 172.21.15.137:0/781535926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c19bbd0 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.109+0000 7fac4091f640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c19b690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fac30009940 tx=0x7fac30032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.109+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac30042070 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.109+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c19be60 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.109+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac300308c0 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.109+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac30037040 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.109+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c19eb50 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.110+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fac30045070 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.110+0000 7fac39ffb640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7fac2c038f70 0x7fac2c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.110+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c10be00 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.110+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac3006c160 con 0x7fac3c107720 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.110+0000 7fac3bfff640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7fac2c038f70 0x7fac2c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:48.037 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.111+0000 7fac3bfff640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7fac2c038f70 0x7fac2c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac280098c0 tx=0x7fac28006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.115+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac30045360 con 0x7fac3c107720 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.328+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fac3c19ed20 con 0x7fac3c107720 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.380+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fac3006bbc0 con 0x7fac3c107720 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.381+0000 7fac39ffb640 1 -- 172.21.15.137:0/781535926 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fac3003c760 con 0x7fac3c107720 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7fac2c038f70 msgr2=0x7fac2c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7fac2c038f70 0x7fac2c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fac280098c0 tx=0x7fac28006db0 comp rx=0 tx=0).stop 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 msgr2=0x7fac3c19b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c19b690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fac30009940 tx=0x7fac30032ce0 comp rx=0 tx=0).stop 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 shutdown_connections 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7fac2c038f70 0x7fac2c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 --2- 172.21.15.137:0/781535926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac3c107720 0x7fac3c19b690 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 >> 172.21.15.137:0/781535926 conn(0x7fac3c102ef0 msgr2=0x7fac3c103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:48.038 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 shutdown_connections 2024-09-12T12:03:48.039 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:47.383+0000 7fac42b83640 1 -- 172.21.15.137:0/781535926 wait complete. 2024-09-12T12:03:48.281 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:48 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:48.083+0000 7f7008b73040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:03:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:48 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/781535926' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T12:03:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:48 smithi137 ceph-mon[30950]: mgrmap e5: x(active, since 13s) 2024-09-12T12:03:48.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:48 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:48.401+0000 7f7008b73040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:03:48.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:48 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:48.464+0000 7f7008b73040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:03:49.015 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:48 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:48.659+0000 7f7008b73040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:03:49.466 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.483+0000 7ff5b97d8640 1 Processor -- start 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.483+0000 7ff5b97d8640 1 -- start start 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.484+0000 7ff5b97d8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b4107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.484+0000 7ff5b97d8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b41080f0 con 0x7ff5b4107720 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.484+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b4107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:49.467 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.484+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b4107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47784/0 (socket says 172.21.15.137:47784) 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.484+0000 7ff5b2ffd640 1 -- 172.21.15.137:0/2902579216 learned_addr learned my addr 172.21.15.137:0/2902579216 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.484+0000 7ff5b2ffd640 1 -- 172.21.15.137:0/2902579216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b41088d0 con 0x7ff5b4107720 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.485+0000 7ff5b2ffd640 1 --2- 172.21.15.137:0/2902579216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b4107b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009820 tx=0x7ff5a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bd6691323e9330 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.485+0000 7ff5b1ffb640 1 -- 172.21.15.137:0/2902579216 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4037070 con 0x7ff5b4107720 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.485+0000 7ff5b1ffb640 1 -- 172.21.15.137:0/2902579216 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5a4032900 con 0x7ff5b4107720 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.485+0000 7ff5b1ffb640 1 -- 172.21.15.137:0/2902579216 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4030850 con 0x7ff5b4107720 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.485+0000 7ff5b97d8640 1 -- 172.21.15.137:0/2902579216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 msgr2=0x7ff5b4107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 --2- 172.21.15.137:0/2902579216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b4107b20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009820 tx=0x7ff5a402a270 comp rx=0 tx=0).stop 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 -- 172.21.15.137:0/2902579216 shutdown_connections 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 --2- 172.21.15.137:0/2902579216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b4107b20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 -- 172.21.15.137:0/2902579216 >> 172.21.15.137:0/2902579216 conn(0x7ff5b4102ef0 msgr2=0x7ff5b4105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 -- 172.21.15.137:0/2902579216 shutdown_connections 2024-09-12T12:03:49.468 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 -- 172.21.15.137:0/2902579216 wait complete. 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 Processor -- start 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.486+0000 7ff5b97d8640 1 -- start start 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b97d8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b97d8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b410bc80 con 0x7ff5b4107720 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47796/0 (socket says 172.21.15.137:47796) 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b2ffd640 1 -- 172.21.15.137:0/3766738303 learned_addr learned my addr 172.21.15.137:0/3766738303 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b2ffd640 1 -- 172.21.15.137:0/3766738303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b419bc00 con 0x7ff5b4107720 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b2ffd640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b419b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009940 tx=0x7ff5a4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff59bfff640 1 -- 172.21.15.137:0/3766738303 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4042070 con 0x7ff5b4107720 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff59bfff640 1 -- 172.21.15.137:0/3766738303 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5a40308c0 con 0x7ff5b4107720 2024-09-12T12:03:49.469 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b419be90 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff59bfff640 1 -- 172.21.15.137:0/3766738303 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5a4037040 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.487+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b419eb80 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.488+0000 7ff59bfff640 1 -- 172.21.15.137:0/3766738303 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff5a4045070 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.488+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b410be00 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.489+0000 7ff59bfff640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 0x7ff59c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.489+0000 7ff59bfff640 1 -- 172.21.15.137:0/3766738303 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5a406bfc0 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.489+0000 7ff5b27fc640 1 -- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 msgr2=0x7ff59c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1104802518 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.489+0000 7ff5b27fc640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 0x7ff59c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.493+0000 7ff59bfff640 1 -- 172.21.15.137:0/3766738303 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5a4045520 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.690+0000 7ff5b27fc640 1 -- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 msgr2=0x7ff59c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1104802518 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.690+0000 7ff5b27fc640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 0x7ff59c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.712+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff5b419f210 con 0x7ff5b4107720 2024-09-12T12:03:49.470 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.713+0000 7ff59bfff640 1 -- 172.21.15.137:0/3766738303 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7ff5a403cd90 con 0x7ff5b4107720 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 msgr2=0x7ff59c03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 0x7ff59c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 msgr2=0x7ff5b419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b419b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009940 tx=0x7ff5a4032ce0 comp rx=0 tx=0).stop 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 shutdown_connections 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff59c038f70 0x7ff59c03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 --2- 172.21.15.137:0/3766738303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5b4107720 0x7ff5b419b6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 >> 172.21.15.137:0/3766738303 conn(0x7ff5b4102ef0 msgr2=0x7ff5b4103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 shutdown_connections 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:48.715+0000 7ff5b97d8640 1 -- 172.21.15.137:0/3766738303 wait complete. 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for the mgr to restart... 2024-09-12T12:03:49.471 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr epoch 5... 2024-09-12T12:03:49.679 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:49 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3766738303' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T12:03:49.680 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:49 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:49.556+0000 7f7008b73040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:03:49.680 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:49 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:49.625+0000 7f7008b73040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:03:50.599 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:50.341+0000 7f7008b73040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:03:50.599 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:50.457+0000 7f7008b73040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:03:50.875 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:50.599+0000 7f7008b73040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:03:50.875 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:50.661+0000 7f7008b73040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:03:50.875 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:50.807+0000 7f7008b73040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:03:50.875 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:50.875+0000 7f7008b73040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:03:51.155 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:50.935+0000 7f7008b73040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:03:51.155 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:51.090+0000 7f7008b73040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:03:51.155 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:51.155+0000 7f7008b73040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:03:51.421 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:51.221+0000 7f7008b73040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:03:51.421 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:51.304+0000 7f7008b73040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:03:51.421 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:51.363+0000 7f7008b73040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:03:51.421 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:51.421+0000 7f7008b73040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:03:52.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:52.203+0000 7f7008b73040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:03:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: Active manager daemon x restarted 2024-09-12T12:03:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: Activating manager daemon x 2024-09-12T12:03:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T12:03:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: mgrmap e6: x(active, starting, since 0.00334885s) 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: Manager daemon x is now available 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:03:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T12:03:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:52 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T12:03:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:53 smithi137 ceph-mon[30950]: Found migration_current of "None". Setting to last migration. 2024-09-12T12:03:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:53 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:03:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:53 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:03:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:53 smithi137 ceph-mon[30950]: mgrmap e7: x(active, since 1.00554s) 2024-09-12T12:03:53.814 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.865+0000 7ff390414640 1 Processor -- start 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.865+0000 7ff390414640 1 -- start start 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.865+0000 7ff390414640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff388106f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.865+0000 7ff390414640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3881074e0 con 0x7ff388106b10 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.866+0000 7ff38e1b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff388106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:53.815 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.866+0000 7ff38e1b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff388106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42182/0 (socket says 172.21.15.137:42182) 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.866+0000 7ff38e1b0640 1 -- 172.21.15.137:0/1743590433 learned_addr learned my addr 172.21.15.137:0/1743590433 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.866+0000 7ff38e1b0640 1 -- 172.21.15.137:0/1743590433 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff388107830 con 0x7ff388106b10 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff38e1b0640 1 --2- 172.21.15.137:0/1743590433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff388106f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff378009820 tx=0x7ff37802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca23845e9b03fea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff38d1ae640 1 -- 172.21.15.137:0/1743590433 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff378037070 con 0x7ff388106b10 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff38d1ae640 1 -- 172.21.15.137:0/1743590433 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff378032900 con 0x7ff388106b10 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff38d1ae640 1 -- 172.21.15.137:0/1743590433 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff378030850 con 0x7ff388106b10 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff390414640 1 -- 172.21.15.137:0/1743590433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 msgr2=0x7ff388106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff390414640 1 --2- 172.21.15.137:0/1743590433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff388106f10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff378009820 tx=0x7ff37802a270 comp rx=0 tx=0).stop 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff390414640 1 -- 172.21.15.137:0/1743590433 shutdown_connections 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff390414640 1 --2- 172.21.15.137:0/1743590433 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff388106f10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.867+0000 7ff390414640 1 -- 172.21.15.137:0/1743590433 >> 172.21.15.137:0/1743590433 conn(0x7ff3881014e0 msgr2=0x7ff388103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff390414640 1 -- 172.21.15.137:0/1743590433 shutdown_connections 2024-09-12T12:03:53.816 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff390414640 1 -- 172.21.15.137:0/1743590433 wait complete. 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff390414640 1 Processor -- start 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff390414640 1 -- start start 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff390414640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff38819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff390414640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff38810abe0 con 0x7ff388106b10 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff38e1b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff38819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff38e1b0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff38819b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42192/0 (socket says 172.21.15.137:42192) 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.868+0000 7ff38e1b0640 1 -- 172.21.15.137:0/3513984469 learned_addr learned my addr 172.21.15.137:0/3513984469 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.869+0000 7ff38e1b0640 1 -- 172.21.15.137:0/3513984469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38819bcd0 con 0x7ff388106b10 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.869+0000 7ff38e1b0640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff38819b790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff3780097f0 tx=0x7ff378030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.869+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff378042070 con 0x7ff388106b10 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.869+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff378032ca0 con 0x7ff388106b10 2024-09-12T12:03:53.817 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.869+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38819bf60 con 0x7ff388106b10 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.869+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff378037040 con 0x7ff388106b10 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.869+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38819ec50 con 0x7ff388106b10 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.870+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7ff378045070 con 0x7ff388106b10 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.870+0000 7ff37f7fe640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 0x7ff37403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.870+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 --> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff37403bb40 con 0x7ff374038f70 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.870+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff37806bea0 con 0x7ff388106b10 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.870+0000 7ff38d9af640 1 -- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 msgr2=0x7ff37403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1104802518 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:49.870+0000 7ff38d9af640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 0x7ff37403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:50.071+0000 7ff38d9af640 1 -- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 msgr2=0x7ff37403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1104802518 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:50.071+0000 7ff38d9af640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 0x7ff37403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:50.472+0000 7ff38d9af640 1 -- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 msgr2=0x7ff37403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1104802518 2024-09-12T12:03:53.818 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:50.472+0000 7ff38d9af640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 0x7ff37403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:51.273+0000 7ff38d9af640 1 -- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 msgr2=0x7ff37403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1104802518 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:51.273+0000 7ff38d9af640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 0x7ff37403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:52.206+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7ff37806a960 con 0x7ff388106b10 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:52.207+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 msgr2=0x7ff37403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:52.207+0000 7ff37f7fe640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 0x7ff37403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.208+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7ff37803d430 con 0x7ff388106b10 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.208+0000 7ff37f7fe640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff37403c8a0 0x7ff37403ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.208+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff37403bb40 con 0x7ff37403c8a0 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.208+0000 7ff38d9af640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff37403c8a0 0x7ff37403ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.209+0000 7ff38d9af640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff37403c8a0 0x7ff37403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff384003980 tx=0x7ff384006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.209+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff37403bb40 con 0x7ff37403c8a0 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.216+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff388106f90 con 0x7ff37403c8a0 2024-09-12T12:03:53.819 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff37f7fe640 1 -- 172.21.15.137:0/3513984469 <== mgr.14118 v2:172.21.15.137:6800/1596672143 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff388106f90 con 0x7ff37403c8a0 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff37403c8a0 msgr2=0x7ff37403ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff37403c8a0 0x7ff37403ec90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff384003980 tx=0x7ff384006e10 comp rx=0 tx=0).stop 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 msgr2=0x7ff38819b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff38819b790 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff3780097f0 tx=0x7ff378030ec0 comp rx=0 tx=0).stop 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 shutdown_connections 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff37403c8a0 0x7ff37403ec90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:6800/1104802518,v1:172.21.15.137:6801/1104802518] conn(0x7ff374038f70 0x7ff37403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 --2- 172.21.15.137:0/3513984469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff388106b10 0x7ff38819b790 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 >> 172.21.15.137:0/3513984469 conn(0x7ff3881014e0 msgr2=0x7ff388102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 shutdown_connections 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:53.217+0000 7ff390414640 1 -- 172.21.15.137:0/3513984469 wait complete. 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:mgr epoch 5 is available 2024-09-12T12:03:53.820 INFO:teuthology.orchestra.run.smithi137.stdout:Setting orchestrator backend to cephadm... 2024-09-12T12:03:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-mon[30950]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T12:03:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-mon[30950]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T12:03:54.309 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:54.303+0000 7f6f6fecd640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:12:03:54] ENGINE Error in HTTPServer.serve 2024-09-12T12:03:54.309 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Traceback (most recent call last): 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._connections.run(self.expiration_interval) 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._run(expiration_interval) 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: s = self.context.wrap_socket( 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: return self.sslsocket_class._create( 2024-09-12T12:03:54.310 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:03:54.311 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self.do_handshake() 2024-09-12T12:03:54.311 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:03:54.311 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._sslobj.do_handshake() 2024-09-12T12:03:54.311 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:03:54.311 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:03:55.070 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.278+0000 7f3ed5168640 1 Processor -- start 2024-09-12T12:03:55.070 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.278+0000 7f3ed5168640 1 -- start start 2024-09-12T12:03:55.070 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.278+0000 7f3ed5168640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed010a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:55.070 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.279+0000 7f3ed5168640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed007be50 con 0x7f3ed01081c0 2024-09-12T12:03:55.070 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.279+0000 7f3eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed010a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:55.070 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.279+0000 7f3eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed010a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42272/0 (socket says 172.21.15.137:42272) 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.279+0000 7f3eced9d640 1 -- 172.21.15.137:0/4041217526 learned_addr learned my addr 172.21.15.137:0/4041217526 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.279+0000 7f3eced9d640 1 -- 172.21.15.137:0/4041217526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed010aaf0 con 0x7f3ed01081c0 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3eced9d640 1 --2- 172.21.15.137:0/4041217526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed010a5b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009820 tx=0x7f3ec002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4597a628ba2369 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ecdd9b640 1 -- 172.21.15.137:0/4041217526 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec0037070 con 0x7f3ed01081c0 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ecdd9b640 1 -- 172.21.15.137:0/4041217526 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ec0032900 con 0x7f3ed01081c0 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ecdd9b640 1 -- 172.21.15.137:0/4041217526 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec0030850 con 0x7f3ed01081c0 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ed5168640 1 -- 172.21.15.137:0/4041217526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 msgr2=0x7f3ed010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ed5168640 1 --2- 172.21.15.137:0/4041217526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed010a5b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0009820 tx=0x7f3ec002a270 comp rx=0 tx=0).stop 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ed5168640 1 -- 172.21.15.137:0/4041217526 shutdown_connections 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ed5168640 1 --2- 172.21.15.137:0/4041217526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed010a5b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:55.071 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ed5168640 1 -- 172.21.15.137:0/4041217526 >> 172.21.15.137:0/4041217526 conn(0x7f3ed0101e20 msgr2=0x7f3ed0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ed5168640 1 -- 172.21.15.137:0/4041217526 shutdown_connections 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.280+0000 7f3ed5168640 1 -- 172.21.15.137:0/4041217526 wait complete. 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.281+0000 7f3ed5168640 1 Processor -- start 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.281+0000 7f3ed5168640 1 -- start start 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.281+0000 7f3ed5168640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed0107a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.281+0000 7f3ed5168640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed010bb30 con 0x7f3ed01081c0 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.281+0000 7f3eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed0107a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.281+0000 7f3eced9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed0107a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42288/0 (socket says 172.21.15.137:42288) 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.281+0000 7f3eced9d640 1 -- 172.21.15.137:0/2217135845 learned_addr learned my addr 172.21.15.137:0/2217135845 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.282+0000 7f3eced9d640 1 -- 172.21.15.137:0/2217135845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed0107f80 con 0x7f3ed01081c0 2024-09-12T12:03:55.072 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.282+0000 7f3eced9d640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed0107a40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0031000 tx=0x7f3ec00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.282+0000 7f3eb7fff640 1 -- 172.21.15.137:0/2217135845 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec0042070 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.282+0000 7f3eb7fff640 1 -- 172.21.15.137:0/2217135845 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ec0032be0 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.282+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed0106170 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.282+0000 7f3eb7fff640 1 -- 172.21.15.137:0/2217135845 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ec0037040 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.282+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed01066b0 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.283+0000 7f3eb7fff640 1 -- 172.21.15.137:0/2217135845 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3ec0045070 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.283+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed0108620 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.283+0000 7f3eb7fff640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f3eb8038f70 0x7f3eb803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.283+0000 7f3eb7fff640 1 -- 172.21.15.137:0/2217135845 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ec006c1a0 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.283+0000 7f3ece59c640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f3eb8038f70 0x7f3eb803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.284+0000 7f3ece59c640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f3eb8038f70 0x7f3eb803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098c0 tx=0x7f3ec4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.288+0000 7f3eb7fff640 1 -- 172.21.15.137:0/2217135845 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ec0032d90 con 0x7f3ed01081c0 2024-09-12T12:03:55.073 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.471+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3ed0103060 con 0x7f3eb8038f70 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.476+0000 7f3eb7fff640 1 -- 172.21.15.137:0/2217135845 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ed0103060 con 0x7f3eb8038f70 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f3eb8038f70 msgr2=0x7f3eb803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f3eb8038f70 0x7f3eb803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40098c0 tx=0x7f3ec4006db0 comp rx=0 tx=0).stop 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 msgr2=0x7f3ed0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed0107a40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0031000 tx=0x7f3ec00326e0 comp rx=0 tx=0).stop 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 shutdown_connections 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f3eb8038f70 0x7f3eb803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 --2- 172.21.15.137:0/2217135845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ed01081c0 0x7f3ed0107a40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 >> 172.21.15.137:0/2217135845 conn(0x7f3ed0101e20 msgr2=0x7f3ed0102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 shutdown_connections 2024-09-12T12:03:55.074 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:54.479+0000 7f3ed5168640 1 -- 172.21.15.137:0/2217135845 wait complete. 2024-09-12T12:03:55.356 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: [12/Sep/2024:12:03:54] ENGINE Bus STARTING 2024-09-12T12:03:55.356 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: [12/Sep/2024:12:03:54] ENGINE Serving on http://172.21.15.137:8765 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: mgrmap e8: x(active, since 2s) 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: [12/Sep/2024:12:03:54] ENGINE Serving on https://172.21.15.137:7150 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: [12/Sep/2024:12:03:54] ENGINE Bus STARTED 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: [12/Sep/2024:12:03:54] ENGINE Error in HTTPServer.serve 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout: Traceback (most recent call last): 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._connections.run(self.expiration_interval) 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._run(expiration_interval) 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:03:55.357 INFO:journalctl@ceph.mon.a.smithi137.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: s = self.context.wrap_socket( 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: return self.sslsocket_class._create( 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: self.do_handshake() 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._sslobj.do_handshake() 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:03:55.358 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:55 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:03:56.167 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33cf183640 1 Processor -- start 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33cf183640 1 -- start start 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33cf183640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c8107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33cf183640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33c8108060 con 0x7f33c8107720 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33ccf1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c8107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33ccf1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c8107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42304/0 (socket says 172.21.15.137:42304) 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33ccf1f640 1 -- 172.21.15.137:0/170216228 learned_addr learned my addr 172.21.15.137:0/170216228 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.433+0000 7f33ccf1f640 1 -- 172.21.15.137:0/170216228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c81081e0 con 0x7f33c8107720 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.434+0000 7f33ccf1f640 1 --2- 172.21.15.137:0/170216228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c8107b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009820 tx=0x7f33bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfad5733bbbdf4e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.434+0000 7f33c77fe640 1 -- 172.21.15.137:0/170216228 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc037070 con 0x7f33c8107720 2024-09-12T12:03:56.168 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.434+0000 7f33c77fe640 1 -- 172.21.15.137:0/170216228 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33bc032900 con 0x7f33c8107720 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.434+0000 7f33c77fe640 1 -- 172.21.15.137:0/170216228 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc030850 con 0x7f33c8107720 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 -- 172.21.15.137:0/170216228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 msgr2=0x7f33c8107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 --2- 172.21.15.137:0/170216228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c8107b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009820 tx=0x7f33bc02a270 comp rx=0 tx=0).stop 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 -- 172.21.15.137:0/170216228 shutdown_connections 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 --2- 172.21.15.137:0/170216228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c8107b20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 -- 172.21.15.137:0/170216228 >> 172.21.15.137:0/170216228 conn(0x7f33c8102f10 msgr2=0x7f33c8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 -- 172.21.15.137:0/170216228 shutdown_connections 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 -- 172.21.15.137:0/170216228 wait complete. 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 Processor -- start 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 -- start start 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c819b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.435+0000 7f33cf183640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f33c810bb90 con 0x7f33c8107720 2024-09-12T12:03:56.169 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33ccf1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c819b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33ccf1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c819b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42316/0 (socket says 172.21.15.137:42316) 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33ccf1f640 1 -- 172.21.15.137:0/573660115 learned_addr learned my addr 172.21.15.137:0/573660115 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33ccf1f640 1 -- 172.21.15.137:0/573660115 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c819bc60 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33ccf1f640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c819b720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33bc031000 tx=0x7f33bc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33c5ffb640 1 -- 172.21.15.137:0/573660115 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc042070 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33c5ffb640 1 -- 172.21.15.137:0/573660115 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33bc032be0 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c819bef0 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33c5ffb640 1 -- 172.21.15.137:0/573660115 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33bc037040 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.436+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c819ebe0 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.437+0000 7f33c5ffb640 1 -- 172.21.15.137:0/573660115 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f33bc045070 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.437+0000 7f33c5ffb640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f33b8038fc0 0x7f33b803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.437+0000 7f33c5ffb640 1 -- 172.21.15.137:0/573660115 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f33bc06c0d0 con 0x7f33c8107720 2024-09-12T12:03:56.170 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.437+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c8107ba0 con 0x7f33c8107720 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.437+0000 7f33c7fff640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f33b8038fc0 0x7f33b803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.438+0000 7f33c7fff640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f33b8038fc0 0x7f33b803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f33b40098c0 tx=0x7f33b4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.442+0000 7f33c5ffb640 1 -- 172.21.15.137:0/573660115 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33bc04c030 con 0x7f33c8107720 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.611+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f33c8104210 con 0x7f33b8038fc0 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.612+0000 7f33c5ffb640 1 -- 172.21.15.137:0/573660115 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f33c8104210 con 0x7f33b8038fc0 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f33b8038fc0 msgr2=0x7f33b803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f33b8038fc0 0x7f33b803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f33b40098c0 tx=0x7f33b4006db0 comp rx=0 tx=0).stop 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 msgr2=0x7f33c819b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c819b720 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33bc031000 tx=0x7f33bc0326e0 comp rx=0 tx=0).stop 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 shutdown_connections 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f33b8038fc0 0x7f33b803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:56.171 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 --2- 172.21.15.137:0/573660115 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c8107720 0x7f33c819b720 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:56.172 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 >> 172.21.15.137:0/573660115 conn(0x7f33c8102f10 msgr2=0x7f33c8103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:56.172 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.614+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 shutdown_connections 2024-09-12T12:03:56.172 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:55.615+0000 7f33cf183640 1 -- 172.21.15.137:0/573660115 wait complete. 2024-09-12T12:03:56.172 INFO:teuthology.orchestra.run.smithi137.stdout:Generating ssh key... 2024-09-12T12:03:56.416 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-mon[30950]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:03:57.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Generating public/private rsa key pair. 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Your identification has been saved in /tmp/tmpmj33h7vb/key 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Your public key has been saved in /tmp/tmpmj33h7vb/key.pub 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: The key fingerprint is: 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: SHA256:ErmU+7h1f70GnkpyrkedogE8Eb0Tn26okjft8ae4KF4 ceph-dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: The key's randomart image is: 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: +---[RSA 3072]----+ 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | .o | 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | + o | 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | * . + . | 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | . B o o | 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | + S + . . | 2024-09-12T12:03:57.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | + o = + | 2024-09-12T12:03:57.051 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | oE=o*oo o. | 2024-09-12T12:03:57.051 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | +.*.+Oo +...| 2024-09-12T12:03:57.051 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: | ..=.o=+==....| 2024-09-12T12:03:57.051 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:03:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: +----[SHA256]-----+ 2024-09-12T12:03:57.475 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:57 smithi137 ceph-mon[30950]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:03:57.475 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:57 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:03:57.475 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:57 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:03:57.482 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.623+0000 7fbca4456640 1 Processor -- start 2024-09-12T12:03:57.482 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.623+0000 7fbca4456640 1 -- start start 2024-09-12T12:03:57.482 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.623+0000 7fbca4456640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:57.482 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.623+0000 7fbca4456640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1080c0 con 0x7fbc9c1076f0 2024-09-12T12:03:57.482 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.623+0000 7fbca21f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.623+0000 7fbca21f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42326/0 (socket says 172.21.15.137:42326) 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.623+0000 7fbca21f2640 1 -- 172.21.15.137:0/1757280247 learned_addr learned my addr 172.21.15.137:0/1757280247 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.624+0000 7fbca21f2640 1 -- 172.21.15.137:0/1757280247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c1088f0 con 0x7fbc9c1076f0 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.624+0000 7fbca21f2640 1 --2- 172.21.15.137:0/1757280247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c107af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009820 tx=0x7fbc9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287bae367de6ec14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.624+0000 7fbca11f0640 1 -- 172.21.15.137:0/1757280247 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc98037070 con 0x7fbc9c1076f0 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.624+0000 7fbca11f0640 1 -- 172.21.15.137:0/1757280247 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc98032900 con 0x7fbc9c1076f0 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.624+0000 7fbca11f0640 1 -- 172.21.15.137:0/1757280247 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc98030850 con 0x7fbc9c1076f0 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.625+0000 7fbca4456640 1 -- 172.21.15.137:0/1757280247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 msgr2=0x7fbc9c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.625+0000 7fbca4456640 1 --2- 172.21.15.137:0/1757280247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c107af0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009820 tx=0x7fbc9802a270 comp rx=0 tx=0).stop 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.625+0000 7fbca4456640 1 -- 172.21.15.137:0/1757280247 shutdown_connections 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.625+0000 7fbca4456640 1 --2- 172.21.15.137:0/1757280247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c107af0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.625+0000 7fbca4456640 1 -- 172.21.15.137:0/1757280247 >> 172.21.15.137:0/1757280247 conn(0x7fbc9c102ec0 msgr2=0x7fbc9c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:57.483 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.625+0000 7fbca4456640 1 -- 172.21.15.137:0/1757280247 shutdown_connections 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.625+0000 7fbca4456640 1 -- 172.21.15.137:0/1757280247 wait complete. 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca4456640 1 Processor -- start 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca4456640 1 -- start start 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca4456640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca4456640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c10bca0 con 0x7fbc9c1076f0 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca21f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca21f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42342/0 (socket says 172.21.15.137:42342) 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca21f2640 1 -- 172.21.15.137:0/1905511085 learned_addr learned my addr 172.21.15.137:0/1905511085 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca21f2640 1 -- 172.21.15.137:0/1905511085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c19bc20 con 0x7fbc9c1076f0 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.626+0000 7fbca21f2640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c19b6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009940 tx=0x7fbc98032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.627+0000 7fbc937fe640 1 -- 172.21.15.137:0/1905511085 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc98042070 con 0x7fbc9c1076f0 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.627+0000 7fbc937fe640 1 -- 172.21.15.137:0/1905511085 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc980308c0 con 0x7fbc9c1076f0 2024-09-12T12:03:57.484 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.627+0000 7fbc937fe640 1 -- 172.21.15.137:0/1905511085 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc98037040 con 0x7fbc9c1076f0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.627+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9c19beb0 con 0x7fbc9c1076f0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.627+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9c19eba0 con 0x7fbc9c1076f0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.627+0000 7fbc937fe640 1 -- 172.21.15.137:0/1905511085 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbc98045070 con 0x7fbc9c1076f0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.628+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc9c10be20 con 0x7fbc9c1076f0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.628+0000 7fbc937fe640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbc88038fc0 0x7fbc8803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.628+0000 7fbc937fe640 1 -- 172.21.15.137:0/1905511085 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc9806bff0 con 0x7fbc9c1076f0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.628+0000 7fbca19f1640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbc88038fc0 0x7fbc8803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.628+0000 7fbca19f1640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbc88038fc0 0x7fbc8803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098c0 tx=0x7fbc8c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.632+0000 7fbc937fe640 1 -- 172.21.15.137:0/1905511085 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc9804b170 con 0x7fbc9c1076f0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.797+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fbc9c1040e0 con 0x7fbc88038fc0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.864+0000 7fbc937fe640 1 -- 172.21.15.137:0/1905511085 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc9c1040e0 con 0x7fbc88038fc0 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbc88038fc0 msgr2=0x7fbc8803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:57.485 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbc88038fc0 0x7fbc8803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0098c0 tx=0x7fbc8c006db0 comp rx=0 tx=0).stop 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 msgr2=0x7fbc9c19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c19b6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009940 tx=0x7fbc98032ce0 comp rx=0 tx=0).stop 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 shutdown_connections 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbc88038fc0 0x7fbc8803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 --2- 172.21.15.137:0/1905511085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc9c1076f0 0x7fbc9c19b6e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.866+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 >> 172.21.15.137:0/1905511085 conn(0x7fbc9c102ec0 msgr2=0x7fbc9c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.867+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 shutdown_connections 2024-09-12T12:03:57.486 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:56.867+0000 7fbca4456640 1 -- 172.21.15.137:0/1905511085 wait complete. 2024-09-12T12:03:58.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:58 smithi137 ceph-mon[30950]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:03:58.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:58 smithi137 ceph-mon[30950]: Generating ssh key... 2024-09-12T12:03:58.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwBOjGHkW9ilto0ip4eQAM0pq5xXoHEio1dnmOOLmK6o6aJt6zs3w1csOWPRUSjQKw0l+5+tX4GjQ/R185dCgi3OslScx8u8r65Qwe2G9It1pHooU57Omqb4dcAwn9cZ0/5S2t3pcpGr1GFIux1MEO4GkCvZa1m9SF7j7Q8c5t2eTal0SX5eTzivdKZEDrQM4XH+hh7CD9F+apB2MyjQfk18p/iuBmWsiZxg1V1LxSBQ+3CL0w7DHRjKH3iRYD0+P+FttsauxKy/Fq0r4inAgLESnWVetX8w4/AQKQQwehTwV0sxGB6096xw/WdYOUXeJ164ZiMmrDLsFhbSmLJHCdZ8iIzGohEKcGnN7kH3SRIBvNFybhzOagzLcJIhYXUlETh4xyIikQFDd6RGPA24pOtwdRnx65arRAVDte6H8MY6tTHJlvcZZllFHvoPcjFbvNmyK1lrL4tkgF5TjAe112j+3BENF8MUT4b/2PBUEsIWFuSGo6SmuuGOCiEkTdM4M= ceph-dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.869+0000 7fbd5284f640 1 Processor -- start 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.869+0000 7fbd5284f640 1 -- start start 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.869+0000 7fbd5284f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c107af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.869+0000 7fbd5284f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c1080c0 con 0x7fbd4c1076f0 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.869+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c107af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.869+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c107af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42358/0 (socket says 172.21.15.137:42358) 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.869+0000 7fbd4bfff640 1 -- 172.21.15.137:0/121789820 learned_addr learned my addr 172.21.15.137:0/121789820 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.870+0000 7fbd4bfff640 1 -- 172.21.15.137:0/121789820 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c1088f0 con 0x7fbd4c1076f0 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.870+0000 7fbd4bfff640 1 --2- 172.21.15.137:0/121789820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c107af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009820 tx=0x7fbd3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57d6550811a31b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:58.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.870+0000 7fbd4affd640 1 -- 172.21.15.137:0/121789820 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c037070 con 0x7fbd4c1076f0 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd4affd640 1 -- 172.21.15.137:0/121789820 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd3c032900 con 0x7fbd4c1076f0 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd4affd640 1 -- 172.21.15.137:0/121789820 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c030850 con 0x7fbd4c1076f0 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd5284f640 1 -- 172.21.15.137:0/121789820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 msgr2=0x7fbd4c107af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd5284f640 1 --2- 172.21.15.137:0/121789820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c107af0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009820 tx=0x7fbd3c02a270 comp rx=0 tx=0).stop 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd5284f640 1 -- 172.21.15.137:0/121789820 shutdown_connections 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd5284f640 1 --2- 172.21.15.137:0/121789820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c107af0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd5284f640 1 -- 172.21.15.137:0/121789820 >> 172.21.15.137:0/121789820 conn(0x7fbd4c102ec0 msgr2=0x7fbd4c1052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd5284f640 1 -- 172.21.15.137:0/121789820 shutdown_connections 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.871+0000 7fbd5284f640 1 -- 172.21.15.137:0/121789820 wait complete. 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd5284f640 1 Processor -- start 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd5284f640 1 -- start start 2024-09-12T12:03:58.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd5284f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd5284f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c10bca0 con 0x7fbd4c1076f0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42366/0 (socket says 172.21.15.137:42366) 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd4bfff640 1 -- 172.21.15.137:0/3489922676 learned_addr learned my addr 172.21.15.137:0/3489922676 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.872+0000 7fbd4bfff640 1 -- 172.21.15.137:0/3489922676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c19bc20 con 0x7fbd4c1076f0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.873+0000 7fbd4bfff640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c19b6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009940 tx=0x7fbd3c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.873+0000 7fbd497fa640 1 -- 172.21.15.137:0/3489922676 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c042070 con 0x7fbd4c1076f0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.873+0000 7fbd497fa640 1 -- 172.21.15.137:0/3489922676 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd3c0308c0 con 0x7fbd4c1076f0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.873+0000 7fbd497fa640 1 -- 172.21.15.137:0/3489922676 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c037040 con 0x7fbd4c1076f0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.873+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c19beb0 con 0x7fbd4c1076f0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.873+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c19eba0 con 0x7fbd4c1076f0 2024-09-12T12:03:58.658 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.874+0000 7fbd497fa640 1 -- 172.21.15.137:0/3489922676 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbd3c045070 con 0x7fbd4c1076f0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.874+0000 7fbd497fa640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbd34038fc0 0x7fbd3403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.874+0000 7fbd497fa640 1 -- 172.21.15.137:0/3489922676 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd3c06c010 con 0x7fbd4c1076f0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.874+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c10be20 con 0x7fbd4c1076f0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.874+0000 7fbd4b7fe640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbd34038fc0 0x7fbd3403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.874+0000 7fbd4b7fe640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbd34038fc0 0x7fbd3403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd400098c0 tx=0x7fbd40006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:57.879+0000 7fbd497fa640 1 -- 172.21.15.137:0/3489922676 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd3c045360 con 0x7fbd4c1076f0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.046+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fbd4c1040e0 con 0x7fbd34038fc0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.047+0000 7fbd497fa640 1 -- 172.21.15.137:0/3489922676 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fbd4c1040e0 con 0x7fbd34038fc0 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.049+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbd34038fc0 msgr2=0x7fbd3403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.049+0000 7fbd5284f640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbd34038fc0 0x7fbd3403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbd400098c0 tx=0x7fbd40006db0 comp rx=0 tx=0).stop 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.049+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 msgr2=0x7fbd4c19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.049+0000 7fbd5284f640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c19b6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009940 tx=0x7fbd3c032ce0 comp rx=0 tx=0).stop 2024-09-12T12:03:58.659 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.050+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 shutdown_connections 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.050+0000 7fbd5284f640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fbd34038fc0 0x7fbd3403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.050+0000 7fbd5284f640 1 --2- 172.21.15.137:0/3489922676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd4c1076f0 0x7fbd4c19b6e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.050+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 >> 172.21.15.137:0/3489922676 conn(0x7fbd4c102ec0 msgr2=0x7fbd4c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.050+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 shutdown_connections 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:58.050+0000 7fbd5284f640 1 -- 172.21.15.137:0/3489922676 wait complete. 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T12:03:58.660 INFO:teuthology.orchestra.run.smithi137.stdout:Adding host smithi137... 2024-09-12T12:03:59.540 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:03:59 smithi137 ceph-mon[30950]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:00.320 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:00 smithi137 ceph-mon[30950]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "addr": "172.21.15.137", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:01 smithi137 ceph-mon[30950]: Deploying cephadm binary to smithi137 2024-09-12T12:04:03.055 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Added host 'smithi137' with addr '172.21.15.137' 2024-09-12T12:04:03.055 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.096+0000 7fbab2ca2640 1 Processor -- start 2024-09-12T12:04:03.055 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.096+0000 7fbab2ca2640 1 -- start start 2024-09-12T12:04:03.055 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.096+0000 7fbab2ca2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:03.055 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.096+0000 7fbab2ca2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1080f0 con 0x7fbaac107720 2024-09-12T12:04:03.055 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.096+0000 7fbab0a3e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.096+0000 7fbab0a3e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42376/0 (socket says 172.21.15.137:42376) 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.096+0000 7fbab0a3e640 1 -- 172.21.15.137:0/3661242558 learned_addr learned my addr 172.21.15.137:0/3661242558 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.097+0000 7fbab0a3e640 1 -- 172.21.15.137:0/3661242558 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac1088d0 con 0x7fbaac107720 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.097+0000 7fbab0a3e640 1 --2- 172.21.15.137:0/3661242558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac107b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009820 tx=0x7fbaa002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d54522be06e90e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.097+0000 7fbaab7fe640 1 -- 172.21.15.137:0/3661242558 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa0037070 con 0x7fbaac107720 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.097+0000 7fbaab7fe640 1 -- 172.21.15.137:0/3661242558 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbaa0032900 con 0x7fbaac107720 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.097+0000 7fbaab7fe640 1 -- 172.21.15.137:0/3661242558 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa0030850 con 0x7fbaac107720 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.098+0000 7fbab2ca2640 1 -- 172.21.15.137:0/3661242558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 msgr2=0x7fbaac107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.098+0000 7fbab2ca2640 1 --2- 172.21.15.137:0/3661242558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac107b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009820 tx=0x7fbaa002a270 comp rx=0 tx=0).stop 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.098+0000 7fbab2ca2640 1 -- 172.21.15.137:0/3661242558 shutdown_connections 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.098+0000 7fbab2ca2640 1 --2- 172.21.15.137:0/3661242558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac107b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:03.056 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.098+0000 7fbab2ca2640 1 -- 172.21.15.137:0/3661242558 >> 172.21.15.137:0/3661242558 conn(0x7fbaac102ef0 msgr2=0x7fbaac105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.098+0000 7fbab2ca2640 1 -- 172.21.15.137:0/3661242558 shutdown_connections 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.098+0000 7fbab2ca2640 1 -- 172.21.15.137:0/3661242558 wait complete. 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab2ca2640 1 Processor -- start 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab2ca2640 1 -- start start 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab2ca2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab2ca2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac10bc80 con 0x7fbaac107720 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab0a3e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab0a3e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42384/0 (socket says 172.21.15.137:42384) 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab0a3e640 1 -- 172.21.15.137:0/725157556 learned_addr learned my addr 172.21.15.137:0/725157556 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab0a3e640 1 -- 172.21.15.137:0/725157556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac07e0b0 con 0x7fbaac107720 2024-09-12T12:04:03.057 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab0a3e640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac07db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009940 tx=0x7fbaa0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbaa9ffb640 1 -- 172.21.15.137:0/725157556 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa0042070 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbaa9ffb640 1 -- 172.21.15.137:0/725157556 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbaa00308c0 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbaa9ffb640 1 -- 172.21.15.137:0/725157556 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbaa0037040 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.099+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac07c2a0 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.100+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac07c7e0 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.101+0000 7fbaa9ffb640 1 -- 172.21.15.137:0/725157556 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbaa0045070 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.101+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaac107ba0 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.101+0000 7fbaa9ffb640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fba9c038f70 0x7fba9c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.101+0000 7fbaabfff640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fba9c038f70 0x7fba9c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.101+0000 7fbaa9ffb640 1 -- 172.21.15.137:0/725157556 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbaa006c030 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.102+0000 7fbaabfff640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fba9c038f70 0x7fba9c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fba980098c0 tx=0x7fba98006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.106+0000 7fbaa9ffb640 1 -- 172.21.15.137:0/725157556 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbaa003cd90 con 0x7fbaac107720 2024-09-12T12:04:03.058 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:03:59.278+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "addr": "172.21.15.137", "target": ["mon-mgr", ""]}) -- 0x7fbaac104140 con 0x7fba9c038f70 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.442+0000 7fbaa9ffb640 1 -- 172.21.15.137:0/725157556 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbaac104140 con 0x7fba9c038f70 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.445+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fba9c038f70 msgr2=0x7fba9c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.445+0000 7fbab2ca2640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fba9c038f70 0x7fba9c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fba980098c0 tx=0x7fba98006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 msgr2=0x7fbaac07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac07db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009940 tx=0x7fbaa0032ce0 comp rx=0 tx=0).stop 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 shutdown_connections 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fba9c038f70 0x7fba9c03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 --2- 172.21.15.137:0/725157556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbaac107720 0x7fbaac07db70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 >> 172.21.15.137:0/725157556 conn(0x7fbaac102ef0 msgr2=0x7fbaac103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 shutdown_connections 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:02.446+0000 7fbab2ca2640 1 -- 172.21.15.137:0/725157556 wait complete. 2024-09-12T12:04:03.059 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying unmanaged mon service... 2024-09-12T12:04:03.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:03 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:04:03.686 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:03 smithi137 ceph-mon[30950]: Added host smithi137 2024-09-12T12:04:03.687 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:03 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:04.497 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.479+0000 7f09b67a3640 1 Processor -- start 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.480+0000 7f09b67a3640 1 -- start start 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.480+0000 7f09b67a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b0107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.480+0000 7f09b67a3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01080f0 con 0x7f09b0107720 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.480+0000 7f09affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b0107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.480+0000 7f09affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b0107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60326/0 (socket says 172.21.15.137:60326) 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.480+0000 7f09affff640 1 -- 172.21.15.137:0/224042927 learned_addr learned my addr 172.21.15.137:0/224042927 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.480+0000 7f09affff640 1 -- 172.21.15.137:0/224042927 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b01088d0 con 0x7f09b0107720 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.481+0000 7f09affff640 1 --2- 172.21.15.137:0/224042927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b0107b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009820 tx=0x7f09a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8e6461e889977c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.481+0000 7f09aeffd640 1 -- 172.21.15.137:0/224042927 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09a0037070 con 0x7f09b0107720 2024-09-12T12:04:04.498 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.481+0000 7f09aeffd640 1 -- 172.21.15.137:0/224042927 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09a0032900 con 0x7f09b0107720 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.481+0000 7f09aeffd640 1 -- 172.21.15.137:0/224042927 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09a0030850 con 0x7f09b0107720 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 -- 172.21.15.137:0/224042927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 msgr2=0x7f09b0107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 --2- 172.21.15.137:0/224042927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b0107b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009820 tx=0x7f09a002a270 comp rx=0 tx=0).stop 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 -- 172.21.15.137:0/224042927 shutdown_connections 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 --2- 172.21.15.137:0/224042927 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b0107b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 -- 172.21.15.137:0/224042927 >> 172.21.15.137:0/224042927 conn(0x7f09b0102ef0 msgr2=0x7f09b0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 -- 172.21.15.137:0/224042927 shutdown_connections 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 -- 172.21.15.137:0/224042927 wait complete. 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 Processor -- start 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.482+0000 7f09b67a3640 1 -- start start 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09b67a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:04.499 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09b67a3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09b010bc80 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60340/0 (socket says 172.21.15.137:60340) 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09affff640 1 -- 172.21.15.137:0/565489783 learned_addr learned my addr 172.21.15.137:0/565489783 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09affff640 1 -- 172.21.15.137:0/565489783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b007e0b0 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09affff640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b007db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009940 tx=0x7f09a0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09ad7fa640 1 -- 172.21.15.137:0/565489783 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09a0042070 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09ad7fa640 1 -- 172.21.15.137:0/565489783 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09a00308c0 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.483+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b007c2a0 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.484+0000 7f09ad7fa640 1 -- 172.21.15.137:0/565489783 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09a0037040 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.484+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b007c7e0 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.484+0000 7f09ad7fa640 1 -- 172.21.15.137:0/565489783 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f09a0045070 con 0x7f09b0107720 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.484+0000 7f09ad7fa640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f099c038fc0 0x7f099c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:04.500 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.485+0000 7f09ad7fa640 1 -- 172.21.15.137:0/565489783 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f09a006d090 con 0x7f09b0107720 2024-09-12T12:04:04.501 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.485+0000 7f09af7fe640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f099c038fc0 0x7f099c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:04.501 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.485+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b010be00 con 0x7f09b0107720 2024-09-12T12:04:04.501 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.485+0000 7f09af7fe640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f099c038fc0 0x7f099c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f09980098c0 tx=0x7f0998006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:04.501 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.490+0000 7f09ad7fa640 1 -- 172.21.15.137:0/565489783 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09a0045520 con 0x7f09b0107720 2024-09-12T12:04:04.501 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.685+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f09b007cf40 con 0x7f099c038fc0 2024-09-12T12:04:04.501 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.690+0000 7f09ad7fa640 1 -- 172.21.15.137:0/565489783 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f09b007cf40 con 0x7f099c038fc0 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f099c038fc0 msgr2=0x7f099c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f099c038fc0 0x7f099c03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f09980098c0 tx=0x7f0998006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 msgr2=0x7f09b007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b007db70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009940 tx=0x7f09a0032ce0 comp rx=0 tx=0).stop 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 shutdown_connections 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f099c038fc0 0x7f099c03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 --2- 172.21.15.137:0/565489783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f09b0107720 0x7f09b007db70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 >> 172.21.15.137:0/565489783 conn(0x7f09b0102ef0 msgr2=0x7f09b0103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 shutdown_connections 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:03.692+0000 7f09b67a3640 1 -- 172.21.15.137:0/565489783 wait complete. 2024-09-12T12:04:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying unmanaged mgr service... 2024-09-12T12:04:04.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:04 smithi137 ceph-mon[30950]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:04.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:04 smithi137 ceph-mon[30950]: Saving service mon spec with placement count:5 2024-09-12T12:04:04.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:04 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:04:05.724 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T12:04:05.725 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.892+0000 7ff1f448c640 1 Processor -- start 2024-09-12T12:04:05.729 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.892+0000 7ff1f448c640 1 -- start start 2024-09-12T12:04:05.729 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.893+0000 7ff1f448c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:05.729 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.893+0000 7ff1f448c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec108110 con 0x7ff1ec107740 2024-09-12T12:04:05.729 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.893+0000 7ff1f2228640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:05.729 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.893+0000 7ff1f2228640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60348/0 (socket says 172.21.15.137:60348) 2024-09-12T12:04:05.729 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.893+0000 7ff1f2228640 1 -- 172.21.15.137:0/974533131 learned_addr learned my addr 172.21.15.137:0/974533131 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:05.729 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.893+0000 7ff1f2228640 1 -- 172.21.15.137:0/974533131 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1088f0 con 0x7ff1ec107740 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.894+0000 7ff1f2228640 1 --2- 172.21.15.137:0/974533131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec107b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009820 tx=0x7ff1e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef329fcc0c7b011 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.894+0000 7ff1f1226640 1 -- 172.21.15.137:0/974533131 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e8037070 con 0x7ff1ec107740 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.895+0000 7ff1f1226640 1 -- 172.21.15.137:0/974533131 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1e8032900 con 0x7ff1ec107740 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.895+0000 7ff1f1226640 1 -- 172.21.15.137:0/974533131 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e8030850 con 0x7ff1ec107740 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.895+0000 7ff1f448c640 1 -- 172.21.15.137:0/974533131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 msgr2=0x7ff1ec107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.895+0000 7ff1f448c640 1 --2- 172.21.15.137:0/974533131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec107b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009820 tx=0x7ff1e802a270 comp rx=0 tx=0).stop 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.895+0000 7ff1f448c640 1 -- 172.21.15.137:0/974533131 shutdown_connections 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.895+0000 7ff1f448c640 1 --2- 172.21.15.137:0/974533131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec107b40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.895+0000 7ff1f448c640 1 -- 172.21.15.137:0/974533131 >> 172.21.15.137:0/974533131 conn(0x7ff1ec102ef0 msgr2=0x7ff1ec105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f448c640 1 -- 172.21.15.137:0/974533131 shutdown_connections 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f448c640 1 -- 172.21.15.137:0/974533131 wait complete. 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f448c640 1 Processor -- start 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f448c640 1 -- start start 2024-09-12T12:04:05.730 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f448c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f448c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec10bca0 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f2228640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f2228640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60352/0 (socket says 172.21.15.137:60352) 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.896+0000 7ff1f2228640 1 -- 172.21.15.137:0/4091724271 learned_addr learned my addr 172.21.15.137:0/4091724271 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.897+0000 7ff1f2228640 1 -- 172.21.15.137:0/4091724271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec19bca0 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.897+0000 7ff1f2228640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec19b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009940 tx=0x7ff1e80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.897+0000 7ff1e37fe640 1 -- 172.21.15.137:0/4091724271 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e8042070 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.897+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec19bf30 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.897+0000 7ff1e37fe640 1 -- 172.21.15.137:0/4091724271 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff1e8032be0 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.897+0000 7ff1e37fe640 1 -- 172.21.15.137:0/4091724271 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1e8037040 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.897+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec19ec20 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.898+0000 7ff1e37fe640 1 -- 172.21.15.137:0/4091724271 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff1e8045070 con 0x7ff1ec107740 2024-09-12T12:04:05.731 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.898+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ec10be20 con 0x7ff1ec107740 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.899+0000 7ff1e37fe640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff1d8038f70 0x7ff1d803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.899+0000 7ff1e37fe640 1 -- 172.21.15.137:0/4091724271 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff1e806bef0 con 0x7ff1ec107740 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.899+0000 7ff1f1a27640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff1d8038f70 0x7ff1d803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.899+0000 7ff1f1a27640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff1d8038f70 0x7ff1d803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0098c0 tx=0x7ff1dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:04.904+0000 7ff1e37fe640 1 -- 172.21.15.137:0/4091724271 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1e804b170 con 0x7ff1ec107740 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.088+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff1ec104190 con 0x7ff1d8038f70 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.095+0000 7ff1e37fe640 1 -- 172.21.15.137:0/4091724271 <== mgr.14118 v2:172.21.15.137:6800/1596672143 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff1ec104190 con 0x7ff1d8038f70 2024-09-12T12:04:05.732 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff1d8038f70 msgr2=0x7ff1d803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff1d8038f70 0x7ff1d803b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0098c0 tx=0x7ff1dc006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 msgr2=0x7ff1ec19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec19b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009940 tx=0x7ff1e80326e0 comp rx=0 tx=0).stop 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 shutdown_connections 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7ff1d8038f70 0x7ff1d803b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 --2- 172.21.15.137:0/4091724271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1ec107740 0x7ff1ec19b760 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 >> 172.21.15.137:0/4091724271 conn(0x7ff1ec102ef0 msgr2=0x7ff1ec103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 shutdown_connections 2024-09-12T12:04:05.733 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:05.097+0000 7ff1f448c640 1 -- 172.21.15.137:0/4091724271 wait complete. 2024-09-12T12:04:06.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:06 smithi137 ceph-mon[30950]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:06.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:06 smithi137 ceph-mon[30950]: Saving service mgr spec with placement count:2 2024-09-12T12:04:06.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:06 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:04:07.645 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:07 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1912996135' entity='client.admin' 2024-09-12T12:04:07.645 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:07 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.177+0000 7fcaccdfe640 1 Processor -- start 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.177+0000 7fcaccdfe640 1 -- start start 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.177+0000 7fcaccdfe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac81090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.177+0000 7fcaccdfe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcac81096c0 con 0x7fcac8108cf0 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.178+0000 7fcac659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac81090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.178+0000 7fcac659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac81090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60354/0 (socket says 172.21.15.137:60354) 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.178+0000 7fcac659c640 1 -- 172.21.15.137:0/2113138950 learned_addr learned my addr 172.21.15.137:0/2113138950 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:07.653 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.178+0000 7fcac659c640 1 -- 172.21.15.137:0/2113138950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac8109840 con 0x7fcac8108cf0 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcac659c640 1 --2- 172.21.15.137:0/2113138950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac81090f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009820 tx=0x7fcab802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596b74f955ecdef1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcac5d9b640 1 -- 172.21.15.137:0/2113138950 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab8037070 con 0x7fcac8108cf0 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcac5d9b640 1 -- 172.21.15.137:0/2113138950 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcab8032900 con 0x7fcac8108cf0 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcac5d9b640 1 -- 172.21.15.137:0/2113138950 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab8030850 con 0x7fcac8108cf0 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcaccdfe640 1 -- 172.21.15.137:0/2113138950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 msgr2=0x7fcac81090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcaccdfe640 1 --2- 172.21.15.137:0/2113138950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac81090f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009820 tx=0x7fcab802a270 comp rx=0 tx=0).stop 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcaccdfe640 1 -- 172.21.15.137:0/2113138950 shutdown_connections 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcaccdfe640 1 --2- 172.21.15.137:0/2113138950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac81090f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.179+0000 7fcaccdfe640 1 -- 172.21.15.137:0/2113138950 >> 172.21.15.137:0/2113138950 conn(0x7fcac81014e0 msgr2=0x7fcac8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.180+0000 7fcaccdfe640 1 -- 172.21.15.137:0/2113138950 shutdown_connections 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.180+0000 7fcaccdfe640 1 -- 172.21.15.137:0/2113138950 wait complete. 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.180+0000 7fcaccdfe640 1 Processor -- start 2024-09-12T12:04:07.654 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.180+0000 7fcaccdfe640 1 -- start start 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.180+0000 7fcaccdfe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.180+0000 7fcaccdfe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcac810bcc0 con 0x7fcac8108cf0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcac659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcac659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac819b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60370/0 (socket says 172.21.15.137:60370) 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcac659c640 1 -- 172.21.15.137:0/1912996135 learned_addr learned my addr 172.21.15.137:0/1912996135 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcac659c640 1 -- 172.21.15.137:0/1912996135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac819bc70 con 0x7fcac8108cf0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcac659c640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac819b730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009940 tx=0x7fcab80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcabf7fe640 1 -- 172.21.15.137:0/1912996135 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab8042070 con 0x7fcac8108cf0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac819bf00 con 0x7fcac8108cf0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcabf7fe640 1 -- 172.21.15.137:0/1912996135 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcab8032be0 con 0x7fcac8108cf0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcabf7fe640 1 -- 172.21.15.137:0/1912996135 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcab8037040 con 0x7fcac8108cf0 2024-09-12T12:04:07.655 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.181+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac819ebf0 con 0x7fcac8108cf0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.182+0000 7fcabf7fe640 1 -- 172.21.15.137:0/1912996135 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcab8045070 con 0x7fcac8108cf0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.182+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac810be40 con 0x7fcac8108cf0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.182+0000 7fcabf7fe640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fcab4038f70 0x7fcab403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.183+0000 7fcabf7fe640 1 -- 172.21.15.137:0/1912996135 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcab806c0c0 con 0x7fcac8108cf0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.183+0000 7fcabffff640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fcab4038f70 0x7fcab403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.183+0000 7fcabffff640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fcab4038f70 0x7fcab403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098c0 tx=0x7fcab0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.187+0000 7fcabf7fe640 1 -- 172.21.15.137:0/1912996135 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcab8032e00 con 0x7fcac8108cf0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.359+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fcac8109170 con 0x7fcac8108cf0 2024-09-12T12:04:07.656 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.365+0000 7fcabf7fe640 1 -- 172.21.15.137:0/1912996135 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7fcab8045360 con 0x7fcac8108cf0 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fcab4038f70 msgr2=0x7fcab403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fcab4038f70 0x7fcab403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098c0 tx=0x7fcab0006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 msgr2=0x7fcac819b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac819b730 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009940 tx=0x7fcab80326e0 comp rx=0 tx=0).stop 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 shutdown_connections 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fcab4038f70 0x7fcab403b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 --2- 172.21.15.137:0/1912996135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcac8108cf0 0x7fcac819b730 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 >> 172.21.15.137:0/1912996135 conn(0x7fcac81014e0 msgr2=0x7fcac8102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 shutdown_connections 2024-09-12T12:04:07.657 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:06.367+0000 7fcaccdfe640 1 -- 172.21.15.137:0/1912996135 wait complete. 2024-09-12T12:04:08.851 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.060+0000 7fdea38eb640 1 Processor -- start 2024-09-12T12:04:08.851 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.060+0000 7fdea38eb640 1 -- start start 2024-09-12T12:04:08.851 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.060+0000 7fdea38eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c10a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:08.851 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.060+0000 7fdea38eb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c07be50 con 0x7fde9c1081e0 2024-09-12T12:04:08.851 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.060+0000 7fdea1687640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c10a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:08.851 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.060+0000 7fdea1687640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c10a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60378/0 (socket says 172.21.15.137:60378) 2024-09-12T12:04:08.851 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.060+0000 7fdea1687640 1 -- 172.21.15.137:0/3770058603 learned_addr learned my addr 172.21.15.137:0/3770058603 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.061+0000 7fdea1687640 1 -- 172.21.15.137:0/3770058603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c10ab10 con 0x7fde9c1081e0 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.061+0000 7fdea1687640 1 --2- 172.21.15.137:0/3770058603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c10a5d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009e80 tx=0x7fde8c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5290c0dc7089f087 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.061+0000 7fde9bfff640 1 -- 172.21.15.137:0/3770058603 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c036070 con 0x7fde9c1081e0 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.061+0000 7fde9bfff640 1 -- 172.21.15.137:0/3770058603 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c0318e0 con 0x7fde9c1081e0 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.061+0000 7fde9bfff640 1 -- 172.21.15.137:0/3770058603 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c007a00 con 0x7fde9c1081e0 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.062+0000 7fdea38eb640 1 -- 172.21.15.137:0/3770058603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 msgr2=0x7fde9c10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.062+0000 7fdea38eb640 1 --2- 172.21.15.137:0/3770058603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c10a5d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009e80 tx=0x7fde8c02a7a0 comp rx=0 tx=0).stop 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.062+0000 7fdea38eb640 1 -- 172.21.15.137:0/3770058603 shutdown_connections 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.062+0000 7fdea38eb640 1 --2- 172.21.15.137:0/3770058603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c10a5d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.062+0000 7fdea38eb640 1 -- 172.21.15.137:0/3770058603 >> 172.21.15.137:0/3770058603 conn(0x7fde9c101da0 msgr2=0x7fde9c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.062+0000 7fdea38eb640 1 -- 172.21.15.137:0/3770058603 shutdown_connections 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.062+0000 7fdea38eb640 1 -- 172.21.15.137:0/3770058603 wait complete. 2024-09-12T12:04:08.852 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea38eb640 1 Processor -- start 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea38eb640 1 -- start start 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea38eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea38eb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c10bb90 con 0x7fde9c1081e0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea1687640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea1687640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c19b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60386/0 (socket says 172.21.15.137:60386) 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea1687640 1 -- 172.21.15.137:0/219716186 learned_addr learned my addr 172.21.15.137:0/219716186 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea1687640 1 -- 172.21.15.137:0/219716186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c19bc70 con 0x7fde9c1081e0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.063+0000 7fdea1687640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c19b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde8c031cf0 tx=0x7fde8c031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.064+0000 7fde9a7fc640 1 -- 172.21.15.137:0/219716186 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c036050 con 0x7fde9c1081e0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.064+0000 7fde9a7fc640 1 -- 172.21.15.137:0/219716186 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c042070 con 0x7fde9c1081e0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.064+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c19bf00 con 0x7fde9c1081e0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.064+0000 7fde9a7fc640 1 -- 172.21.15.137:0/219716186 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8c03d9e0 con 0x7fde9c1081e0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.064+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c19ebf0 con 0x7fde9c1081e0 2024-09-12T12:04:08.853 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.064+0000 7fde9a7fc640 1 -- 172.21.15.137:0/219716186 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fde8c03dce0 con 0x7fde9c1081e0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.065+0000 7fde9a7fc640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fde84038f70 0x7fde8403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.065+0000 7fde9a7fc640 1 -- 172.21.15.137:0/219716186 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fde8c06d410 con 0x7fde9c1081e0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.065+0000 7fdea0e86640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fde84038f70 0x7fde8403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.065+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c108640 con 0x7fde9c1081e0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.065+0000 7fdea0e86640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fde84038f70 0x7fde8403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde900098c0 tx=0x7fde90006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.070+0000 7fde9a7fc640 1 -- 172.21.15.137:0/219716186 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde8c04c200 con 0x7fde9c1081e0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.245+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fde9c19c3e0 con 0x7fde9c1081e0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.249+0000 7fde9a7fc640 1 -- 172.21.15.137:0/219716186 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fde9c19c3e0 con 0x7fde9c1081e0 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fde84038f70 msgr2=0x7fde8403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fde84038f70 0x7fde8403b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fde900098c0 tx=0x7fde90006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 msgr2=0x7fde9c19b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:08.854 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c19b730 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fde8c031cf0 tx=0x7fde8c031d20 comp rx=0 tx=0).stop 2024-09-12T12:04:08.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 shutdown_connections 2024-09-12T12:04:08.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fde84038f70 0x7fde8403b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:08.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 --2- 172.21.15.137:0/219716186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde9c1081e0 0x7fde9c19b730 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:08.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 >> 172.21.15.137:0/219716186 conn(0x7fde9c101da0 msgr2=0x7fde9c102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:08.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 shutdown_connections 2024-09-12T12:04:08.855 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:08.252+0000 7fdea38eb640 1 -- 172.21.15.137:0/219716186 wait complete. 2024-09-12T12:04:08.855 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling the dashboard module... 2024-09-12T12:04:09.482 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:09 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:04:09.483 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:09 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/219716186' entity='client.admin' 2024-09-12T12:04:10.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/488787288' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T12:04:10.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-mon[30950]: from='mgr.14118 172.21.15.137:0/2093501006' entity='mgr.x' 2024-09-12T12:04:10.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ignoring --setuser ceph since I am not root 2024-09-12T12:04:10.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ignoring --setgroup ceph since I am not root 2024-09-12T12:04:10.528 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:10.353+0000 7f638ec53040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:04:10.528 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:10.408+0000 7f638ec53040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:04:10.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:10.557+0000 7f638ec53040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:04:10.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:10.623+0000 7f638ec53040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:04:10.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:10.770+0000 7f638ec53040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:04:10.877 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.256+0000 7f5b717df640 1 Processor -- start 2024-09-12T12:04:10.877 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.256+0000 7f5b717df640 1 -- start start 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.256+0000 7f5b717df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.256+0000 7f5b717df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1080f0 con 0x7f5b6c107720 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.256+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.256+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56194/0 (socket says 172.21.15.137:56194) 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.256+0000 7f5b6affd640 1 -- 172.21.15.137:0/3012639854 learned_addr learned my addr 172.21.15.137:0/3012639854 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.257+0000 7f5b6affd640 1 -- 172.21.15.137:0/3012639854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c1088d0 con 0x7f5b6c107720 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.257+0000 7f5b6affd640 1 --2- 172.21.15.137:0/3012639854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c107b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009820 tx=0x7f5b5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7353b4482b74b35a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.257+0000 7f5b6a7fc640 1 -- 172.21.15.137:0/3012639854 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b5c037070 con 0x7f5b6c107720 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.257+0000 7f5b6a7fc640 1 -- 172.21.15.137:0/3012639854 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c032920 con 0x7f5b6c107720 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b6a7fc640 1 -- 172.21.15.137:0/3012639854 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b5c030810 con 0x7f5b6c107720 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b717df640 1 -- 172.21.15.137:0/3012639854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 msgr2=0x7f5b6c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b717df640 1 --2- 172.21.15.137:0/3012639854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c107b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009820 tx=0x7f5b5c02a270 comp rx=0 tx=0).stop 2024-09-12T12:04:10.878 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b717df640 1 -- 172.21.15.137:0/3012639854 shutdown_connections 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b717df640 1 --2- 172.21.15.137:0/3012639854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c107b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b717df640 1 -- 172.21.15.137:0/3012639854 >> 172.21.15.137:0/3012639854 conn(0x7f5b6c102ef0 msgr2=0x7f5b6c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b717df640 1 -- 172.21.15.137:0/3012639854 shutdown_connections 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.258+0000 7f5b717df640 1 -- 172.21.15.137:0/3012639854 wait complete. 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.259+0000 7f5b717df640 1 Processor -- start 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.259+0000 7f5b717df640 1 -- start start 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.259+0000 7f5b717df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c19b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.259+0000 7f5b717df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10bc80 con 0x7f5b6c107720 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.259+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c19b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.259+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c19b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56204/0 (socket says 172.21.15.137:56204) 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.259+0000 7f5b6affd640 1 -- 172.21.15.137:0/488787288 learned_addr learned my addr 172.21.15.137:0/488787288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.260+0000 7f5b6affd640 1 -- 172.21.15.137:0/488787288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c19bc00 con 0x7f5b6c107720 2024-09-12T12:04:10.879 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.260+0000 7f5b6affd640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c19b6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c030560 tx=0x7f5b5c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.260+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b5c037040 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.260+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c040070 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.260+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b5c03a740 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.260+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c19be90 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.260+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c19eb80 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.261+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5b5c03a9b0 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.261+0000 7f5b68ff9640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f5b58038f70 0x7f5b5803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.261+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c10be00 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.261+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5b5c06c5a0 con 0x7f5b6c107720 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.261+0000 7f5b627fc640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f5b58038f70 0x7f5b5803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:10.880 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.261+0000 7f5b627fc640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f5b58038f70 0x7f5b5803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5b540098c0 tx=0x7f5b54006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.266+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b5c041270 con 0x7f5b6c107720 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:09.482+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5b6c19ed50 con 0x7f5b6c107720 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.183+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5b5c0328f0 con 0x7f5b6c107720 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.184+0000 7f5b68ff9640 1 -- 172.21.15.137:0/488787288 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5b5c06b750 con 0x7f5b6c107720 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f5b58038f70 msgr2=0x7f5b5803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f5b58038f70 0x7f5b5803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5b540098c0 tx=0x7f5b54006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 msgr2=0x7f5b6c19b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c19b6c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c030560 tx=0x7f5b5c032670 comp rx=0 tx=0).stop 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 shutdown_connections 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f5b58038f70 0x7f5b5803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 --2- 172.21.15.137:0/488787288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b6c107720 0x7f5b6c19b6c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:10.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 >> 172.21.15.137:0/488787288 conn(0x7f5b6c102ef0 msgr2=0x7f5b6c103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:10.882 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 shutdown_connections 2024-09-12T12:04:10.882 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:10.186+0000 7f5b717df640 1 -- 172.21.15.137:0/488787288 wait complete. 2024-09-12T12:04:11.151 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:10.920+0000 7f638ec53040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:04:11.498 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:11 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/488787288' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T12:04:11.498 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:11 smithi137 ceph-mon[30950]: mgrmap e9: x(active, since 17s) 2024-09-12T12:04:11.498 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:11.237+0000 7f638ec53040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:04:11.498 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:11.299+0000 7f638ec53040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:04:11.498 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:11.496+0000 7f638ec53040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.351+0000 7fc2e7f82640 1 Processor -- start 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.352+0000 7fc2e7f82640 1 -- start start 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.352+0000 7fc2e7f82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e01069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.352+0000 7fc2e7f82640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e0106fb0 con 0x7fc2e01065e0 2024-09-12T12:04:12.210 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.352+0000 7fc2e5d1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e01069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.352+0000 7fc2e5d1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e01069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56212/0 (socket says 172.21.15.137:56212) 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.352+0000 7fc2e5d1e640 1 -- 172.21.15.137:0/265572226 learned_addr learned my addr 172.21.15.137:0/265572226 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.352+0000 7fc2e5d1e640 1 -- 172.21.15.137:0/265572226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e01077e0 con 0x7fc2e01065e0 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.353+0000 7fc2e5d1e640 1 --2- 172.21.15.137:0/265572226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e01069e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009820 tx=0x7fc2dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8a60f6af9edbbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.353+0000 7fc2e4d1c640 1 -- 172.21.15.137:0/265572226 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc037070 con 0x7fc2e01065e0 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.353+0000 7fc2e4d1c640 1 -- 172.21.15.137:0/265572226 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2dc032920 con 0x7fc2e01065e0 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.353+0000 7fc2e4d1c640 1 -- 172.21.15.137:0/265572226 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc030810 con 0x7fc2e01065e0 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.354+0000 7fc2e7f82640 1 -- 172.21.15.137:0/265572226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 msgr2=0x7fc2e01069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.354+0000 7fc2e7f82640 1 --2- 172.21.15.137:0/265572226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e01069e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009820 tx=0x7fc2dc02a270 comp rx=0 tx=0).stop 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.354+0000 7fc2e7f82640 1 -- 172.21.15.137:0/265572226 shutdown_connections 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.354+0000 7fc2e7f82640 1 --2- 172.21.15.137:0/265572226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e01069e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:12.211 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.354+0000 7fc2e7f82640 1 -- 172.21.15.137:0/265572226 >> 172.21.15.137:0/265572226 conn(0x7fc2e0101d90 msgr2=0x7fc2e01041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.354+0000 7fc2e7f82640 1 -- 172.21.15.137:0/265572226 shutdown_connections 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.354+0000 7fc2e7f82640 1 -- 172.21.15.137:0/265572226 wait complete. 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e7f82640 1 Processor -- start 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e7f82640 1 -- start start 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e7f82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e7f82640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e010ab90 con 0x7fc2e01065e0 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e5d1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e5d1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56226/0 (socket says 172.21.15.137:56226) 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e5d1e640 1 -- 172.21.15.137:0/2369844690 learned_addr learned my addr 172.21.15.137:0/2369844690 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.355+0000 7fc2e5d1e640 1 -- 172.21.15.137:0/2369844690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e019deb0 con 0x7fc2e01065e0 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.356+0000 7fc2e5d1e640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e019d970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc030560 tx=0x7fc2dc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.356+0000 7fc2d6ffd640 1 -- 172.21.15.137:0/2369844690 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc037040 con 0x7fc2e01065e0 2024-09-12T12:04:12.212 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.356+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e019e140 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.356+0000 7fc2d6ffd640 1 -- 172.21.15.137:0/2369844690 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2dc040070 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.356+0000 7fc2d6ffd640 1 -- 172.21.15.137:0/2369844690 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc03a6d0 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.356+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e01a0e30 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.357+0000 7fc2d6ffd640 1 -- 172.21.15.137:0/2369844690 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc2dc03a940 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.357+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e010ad10 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.357+0000 7fc2d6ffd640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 0x7fc2cc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.357+0000 7fc2d6ffd640 1 -- 172.21.15.137:0/2369844690 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc2dc06c5c0 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.357+0000 7fc2e551d640 1 -- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 msgr2=0x7fc2cc03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1596672143 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.357+0000 7fc2e551d640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 0x7fc2cc03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.361+0000 7fc2d6ffd640 1 -- 172.21.15.137:0/2369844690 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2dc03add0 con 0x7fc2e01065e0 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.558+0000 7fc2e551d640 1 -- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 msgr2=0x7fc2cc03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1596672143 2024-09-12T12:04:12.213 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.558+0000 7fc2e551d640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 0x7fc2cc03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.598+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc2e002d980 con 0x7fc2e01065e0 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.599+0000 7fc2d6ffd640 1 -- 172.21.15.137:0/2369844690 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc2dc045980 con 0x7fc2e01065e0 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 msgr2=0x7fc2cc03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 0x7fc2cc03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 msgr2=0x7fc2e019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e019d970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc030560 tx=0x7fc2dc032670 comp rx=0 tx=0).stop 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 shutdown_connections 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7fc2cc038fc0 0x7fc2cc03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 --2- 172.21.15.137:0/2369844690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2e01065e0 0x7fc2e019d970 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 >> 172.21.15.137:0/2369844690 conn(0x7fc2e0101d90 msgr2=0x7fc2e0102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 shutdown_connections 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:11.601+0000 7fc2e7f82640 1 -- 172.21.15.137:0/2369844690 wait complete. 2024-09-12T12:04:12.214 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for the mgr to restart... 2024-09-12T12:04:12.215 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr epoch 9... 2024-09-12T12:04:12.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:12 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2369844690' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T12:04:12.580 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:12.398+0000 7f638ec53040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:04:12.581 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:12.465+0000 7f638ec53040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:04:13.298 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.183+0000 7f638ec53040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:04:13.298 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.298+0000 7f638ec53040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:04:13.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.441+0000 7f638ec53040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:04:13.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.502+0000 7f638ec53040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:04:13.932 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.650+0000 7f638ec53040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:04:13.932 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.717+0000 7f638ec53040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:04:13.932 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.776+0000 7f638ec53040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:04:13.932 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.931+0000 7f638ec53040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:04:14.202 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:13.995+0000 7f638ec53040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:04:14.203 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:14.061+0000 7f638ec53040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:04:14.203 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:14.142+0000 7f638ec53040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:04:14.203 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:14.201+0000 7f638ec53040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:04:14.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:14.260+0000 7f638ec53040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:04:15.299 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:15.036+0000 7f638ec53040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:04:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: Active manager daemon x restarted 2024-09-12T12:04:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: Activating manager daemon x 2024-09-12T12:04:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T12:04:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: mgrmap e10: x(active, starting, since 0.00333071s) 2024-09-12T12:04:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:04:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T12:04:15.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T12:04:15.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T12:04:15.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T12:04:15.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: Manager daemon x is now available 2024-09-12T12:04:15.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:15.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T12:04:15.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T12:04:16.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:16.401+0000 7f62f9f67640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:12:04:16] ENGINE Error in HTTPServer.serve 2024-09-12T12:04:16.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Traceback (most recent call last): 2024-09-12T12:04:16.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._connections.run(self.expiration_interval) 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._run(expiration_interval) 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: s = self.context.wrap_socket( 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: return self.sslsocket_class._create( 2024-09-12T12:04:16.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:04:16.551 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self.do_handshake() 2024-09-12T12:04:16.551 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:04:16.551 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._sslobj.do_handshake() 2024-09-12T12:04:16.551 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:04:16.551 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:04:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.793+0000 7f9fd3ea1640 1 Processor -- start 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.793+0000 7f9fd3ea1640 1 -- start start 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.794+0000 7f9fd3ea1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:16.937 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.794+0000 7f9fd3ea1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc07beb0 con 0x7f9fcc108180 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.794+0000 7f9fd1c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.794+0000 7f9fd1c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56232/0 (socket says 172.21.15.137:56232) 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.794+0000 7f9fd1c3d640 1 -- 172.21.15.137:0/2390099240 learned_addr learned my addr 172.21.15.137:0/2390099240 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.794+0000 7f9fd1c3d640 1 -- 172.21.15.137:0/2390099240 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc10aab0 con 0x7f9fcc108180 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.795+0000 7f9fd1c3d640 1 --2- 172.21.15.137:0/2390099240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc10a570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009820 tx=0x7f9fc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d59381a23812d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.795+0000 7f9fd0c3b640 1 -- 172.21.15.137:0/2390099240 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc8037070 con 0x7f9fcc108180 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.795+0000 7f9fd0c3b640 1 -- 172.21.15.137:0/2390099240 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8032920 con 0x7f9fcc108180 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.795+0000 7f9fd0c3b640 1 -- 172.21.15.137:0/2390099240 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc8030810 con 0x7f9fcc108180 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/2390099240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 msgr2=0x7f9fcc10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 --2- 172.21.15.137:0/2390099240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc10a570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009820 tx=0x7f9fc802a270 comp rx=0 tx=0).stop 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/2390099240 shutdown_connections 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 --2- 172.21.15.137:0/2390099240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc10a570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:16.938 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/2390099240 >> 172.21.15.137:0/2390099240 conn(0x7f9fcc101e60 msgr2=0x7f9fcc104280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/2390099240 shutdown_connections 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/2390099240 wait complete. 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 Processor -- start 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.796+0000 7f9fd3ea1640 1 -- start start 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd3ea1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd3ea1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc10bb90 con 0x7f9fcc108180 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd1c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd1c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56236/0 (socket says 172.21.15.137:56236) 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd1c3d640 1 -- 172.21.15.137:0/3880703543 learned_addr learned my addr 172.21.15.137:0/3880703543 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd1c3d640 1 -- 172.21.15.137:0/3880703543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc19de60 con 0x7f9fcc108180 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd1c3d640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc19d920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8030420 tx=0x7f9fc8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:16.939 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc8037040 con 0x7f9fcc108180 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc8040070 con 0x7f9fcc108180 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc19e0f0 con 0x7f9fcc108180 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc803a6e0 con 0x7f9fcc108180 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.797+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc1a0de0 con 0x7f9fcc108180 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.798+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f9fc803a9e0 con 0x7f9fcc108180 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.798+0000 7f9fc2ffd640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.798+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 --> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9fb803bb90 con 0x7f9fb8038fc0 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.798+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9fc806c890 con 0x7f9fcc108180 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.798+0000 7f9fd143c640 1 -- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 msgr2=0x7f9fb803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1596672143 2024-09-12T12:04:16.940 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:12.799+0000 7f9fd143c640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:13.000+0000 7f9fd143c640 1 -- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 msgr2=0x7f9fb803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1596672143 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:13.000+0000 7f9fd143c640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:13.401+0000 7f9fd143c640 1 -- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 msgr2=0x7f9fb803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1596672143 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:13.401+0000 7f9fd143c640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:14.202+0000 7f9fd143c640 1 -- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 msgr2=0x7f9fb803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1596672143 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:14.202+0000 7f9fd143c640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:15.040+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f9fc806b340 con 0x7f9fcc108180 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:15.040+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 msgr2=0x7f9fb803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:15.040+0000 7f9fc2ffd640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.041+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f9fc8045980 con 0x7f9fcc108180 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.041+0000 7f9fc2ffd640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9fb803ca80 0x7f9fb803ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.041+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9fb803bb90 con 0x7f9fb803ca80 2024-09-12T12:04:16.941 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.042+0000 7f9fd143c640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9fb803ca80 0x7f9fb803ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.042+0000 7f9fd143c640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9fb803ca80 0x7f9fb803ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc003980 tx=0x7f9fbc006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.043+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f9fb803bb90 con 0x7f9fb803ca80 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.055+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9fcc1085e0 con 0x7f9fb803ca80 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.055+0000 7f9fc2ffd640 1 -- 172.21.15.137:0/3880703543 <== mgr.14150 v2:172.21.15.137:6800/1789953576 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f9fcc1085e0 con 0x7f9fb803ca80 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9fb803ca80 msgr2=0x7f9fb803ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9fb803ca80 0x7f9fb803ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc003980 tx=0x7f9fbc006e10 comp rx=0 tx=0).stop 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 msgr2=0x7f9fcc19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc19d920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8030420 tx=0x7f9fc8032670 comp rx=0 tx=0).stop 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 shutdown_connections 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9fb803ca80 0x7f9fb803ee70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:6800/1596672143,v1:172.21.15.137:6801/1596672143] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 --2- 172.21.15.137:0/3880703543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9fcc108180 0x7f9fcc19d920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:16.942 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 >> 172.21.15.137:0/3880703543 conn(0x7f9fcc101e60 msgr2=0x7f9fcc102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:16.943 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 shutdown_connections 2024-09-12T12:04:16.943 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:16.056+0000 7f9fd3ea1640 1 -- 172.21.15.137:0/3880703543 wait complete. 2024-09-12T12:04:16.943 INFO:teuthology.orchestra.run.smithi137.stdout:mgr epoch 9 is available 2024-09-12T12:04:16.943 INFO:teuthology.orchestra.run.smithi137.stdout:Generating a dashboard self-signed certificate... 2024-09-12T12:04:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:17 smithi137 ceph-mon[30950]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T12:04:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:17 smithi137 ceph-mon[30950]: mgrmap e11: x(active, since 1.00615s) 2024-09-12T12:04:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:17 smithi137 ceph-mon[30950]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T12:04:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: [12/Sep/2024:12:04:16] ENGINE Bus STARTING 2024-09-12T12:04:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: [12/Sep/2024:12:04:16] ENGINE Serving on https://172.21.15.137:7150 2024-09-12T12:04:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: [12/Sep/2024:12:04:16] ENGINE Error in HTTPServer.serve 2024-09-12T12:04:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout: Traceback (most recent call last): 2024-09-12T12:04:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._connections.run(self.expiration_interval) 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._run(expiration_interval) 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:04:18.300 INFO:journalctl@ceph.mon.a.smithi137.stdout: s = self.context.wrap_socket( 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout: return self.sslsocket_class._create( 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout: self.do_handshake() 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._sslobj.do_handshake() 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: [12/Sep/2024:12:04:16] ENGINE Serving on http://172.21.15.137:8765 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: [12/Sep/2024:12:04:16] ENGINE Bus STARTED 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: mgrmap e12: x(active, since 2s) 2024-09-12T12:04:18.301 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:18.302 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:18 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.426+0000 7fef7de60640 1 Processor -- start 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.426+0000 7fef7de60640 1 -- start start 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.426+0000 7fef7de60640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef78107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.426+0000 7fef7de60640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef781080a0 con 0x7fef781076d0 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.426+0000 7fef777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef78107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.426+0000 7fef777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef78107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56298/0 (socket says 172.21.15.137:56298) 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.426+0000 7fef777fe640 1 -- 172.21.15.137:0/2825772924 learned_addr learned my addr 172.21.15.137:0/2825772924 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.427+0000 7fef777fe640 1 -- 172.21.15.137:0/2825772924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef781088d0 con 0x7fef781076d0 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.427+0000 7fef777fe640 1 --2- 172.21.15.137:0/2825772924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef78107ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef68009e80 tx=0x7fef6802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f75747952c2dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:18.311 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.427+0000 7fef767fc640 1 -- 172.21.15.137:0/2825772924 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef68036070 con 0x7fef781076d0 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.427+0000 7fef767fc640 1 -- 172.21.15.137:0/2825772924 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef680318e0 con 0x7fef781076d0 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.427+0000 7fef767fc640 1 -- 172.21.15.137:0/2825772924 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef68007a00 con 0x7fef781076d0 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.428+0000 7fef7de60640 1 -- 172.21.15.137:0/2825772924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 msgr2=0x7fef78107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.428+0000 7fef7de60640 1 --2- 172.21.15.137:0/2825772924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef78107ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef68009e80 tx=0x7fef6802a7a0 comp rx=0 tx=0).stop 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.428+0000 7fef7de60640 1 -- 172.21.15.137:0/2825772924 shutdown_connections 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.428+0000 7fef7de60640 1 --2- 172.21.15.137:0/2825772924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef78107ad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.428+0000 7fef7de60640 1 -- 172.21.15.137:0/2825772924 >> 172.21.15.137:0/2825772924 conn(0x7fef78102e60 msgr2=0x7fef781052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.428+0000 7fef7de60640 1 -- 172.21.15.137:0/2825772924 shutdown_connections 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.428+0000 7fef7de60640 1 -- 172.21.15.137:0/2825772924 wait complete. 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef7de60640 1 Processor -- start 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef7de60640 1 -- start start 2024-09-12T12:04:18.312 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef7de60640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef7819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef7de60640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fef7810bc80 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef7819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef7819b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56300/0 (socket says 172.21.15.137:56300) 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef777fe640 1 -- 172.21.15.137:0/2453540158 learned_addr learned my addr 172.21.15.137:0/2453540158 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.429+0000 7fef777fe640 1 -- 172.21.15.137:0/2453540158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7819bc30 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.430+0000 7fef777fe640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef7819b6f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef68031cc0 tx=0x7fef68031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.430+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef68036050 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.430+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7819bec0 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.430+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef68042070 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.430+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef6803d8e0 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.430+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7819ebb0 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.431+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fef6803dbe0 con 0x7fef781076d0 2024-09-12T12:04:18.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.431+0000 7fef74ff9640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fef60038eb0 0x7fef6003b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.431+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef6806d4d0 con 0x7fef781076d0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.431+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7810be00 con 0x7fef781076d0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.431+0000 7fef76ffd640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fef60038eb0 0x7fef6003b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.431+0000 7fef76ffd640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fef60038eb0 0x7fef6003b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098c0 tx=0x7fef6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.436+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef68044310 con 0x7fef781076d0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.544+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fef68047d00 con 0x7fef781076d0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.617+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fef781040d0 con 0x7fef60038eb0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.656+0000 7fef74ff9640 1 -- 172.21.15.137:0/2453540158 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fef781040d0 con 0x7fef60038eb0 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fef60038eb0 msgr2=0x7fef6003b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fef60038eb0 0x7fef6003b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fef6c0098c0 tx=0x7fef6c006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 msgr2=0x7fef7819b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef7819b6f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef68031cc0 tx=0x7fef68031cf0 comp rx=0 tx=0).stop 2024-09-12T12:04:18.314 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 shutdown_connections 2024-09-12T12:04:18.315 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fef60038eb0 0x7fef6003b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:18.315 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 --2- 172.21.15.137:0/2453540158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef781076d0 0x7fef7819b6f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:18.315 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 >> 172.21.15.137:0/2453540158 conn(0x7fef78102e60 msgr2=0x7fef781039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:18.315 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 shutdown_connections 2024-09-12T12:04:18.315 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:17.659+0000 7fef7de60640 1 -- 172.21.15.137:0/2453540158 wait complete. 2024-09-12T12:04:18.315 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial admin user... 2024-09-12T12:04:19.168 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:19 smithi137 ceph-mon[30950]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:20 smithi137 ceph-mon[30950]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$w/PxHRbogkcd5wDxbapARuYk36qNljFTDA5YxB8.TRBwyddU/ntD6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726142659, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcae5982640 1 Processor -- start 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcae5982640 1 -- start start 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcae5982640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae0107b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcae5982640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01080f0 con 0x7fcae0107720 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcadeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae0107b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcadeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae0107b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56302/0 (socket says 172.21.15.137:56302) 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcadeffd640 1 -- 172.21.15.137:0/3542092622 learned_addr learned my addr 172.21.15.137:0/3542092622 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:20.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.745+0000 7fcadeffd640 1 -- 172.21.15.137:0/3542092622 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae01088d0 con 0x7fcae0107720 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.746+0000 7fcadeffd640 1 --2- 172.21.15.137:0/3542092622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae0107b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009820 tx=0x7fcad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6247bea89f4f0343 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcaddffb640 1 -- 172.21.15.137:0/3542092622 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad0037070 con 0x7fcae0107720 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcaddffb640 1 -- 172.21.15.137:0/3542092622 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad0032920 con 0x7fcae0107720 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcaddffb640 1 -- 172.21.15.137:0/3542092622 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad0030810 con 0x7fcae0107720 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcae5982640 1 -- 172.21.15.137:0/3542092622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 msgr2=0x7fcae0107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcae5982640 1 --2- 172.21.15.137:0/3542092622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae0107b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009820 tx=0x7fcad002a270 comp rx=0 tx=0).stop 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcae5982640 1 -- 172.21.15.137:0/3542092622 shutdown_connections 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcae5982640 1 --2- 172.21.15.137:0/3542092622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae0107b20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcae5982640 1 -- 172.21.15.137:0/3542092622 >> 172.21.15.137:0/3542092622 conn(0x7fcae0102ef0 msgr2=0x7fcae0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:20.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcae5982640 1 -- 172.21.15.137:0/3542092622 shutdown_connections 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.747+0000 7fcae5982640 1 -- 172.21.15.137:0/3542092622 wait complete. 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.748+0000 7fcae5982640 1 Processor -- start 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.748+0000 7fcae5982640 1 -- start start 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.748+0000 7fcae5982640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.748+0000 7fcae5982640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcae010bc80 con 0x7fcae0107720 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.748+0000 7fcadeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.748+0000 7fcadeffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae019b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56316/0 (socket says 172.21.15.137:56316) 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcadeffd640 1 -- 172.21.15.137:0/325550592 learned_addr learned my addr 172.21.15.137:0/325550592 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcadeffd640 1 -- 172.21.15.137:0/325550592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae019bc60 con 0x7fcae0107720 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcadeffd640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae019b720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcad00097f0 tx=0x7fcad00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:20.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcae4980640 1 -- 172.21.15.137:0/325550592 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad0037040 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcae4980640 1 -- 172.21.15.137:0/325550592 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad0030cc0 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae019bef0 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcae4980640 1 -- 172.21.15.137:0/325550592 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad003a640 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.749+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae019ebe0 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.750+0000 7fcae4980640 1 -- 172.21.15.137:0/325550592 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fcad003a940 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.750+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae010be00 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.750+0000 7fcae4980640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcac8038fc0 0x7fcac803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.750+0000 7fcae4980640 1 -- 172.21.15.137:0/325550592 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcad006c6f0 con 0x7fcae0107720 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.750+0000 7fcade7fc640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcac8038fc0 0x7fcac803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:20.584 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.751+0000 7fcade7fc640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcac8038fc0 0x7fcac803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcad40098c0 tx=0x7fcad4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.755+0000 7fcae4980640 1 -- 172.21.15.137:0/325550592 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcad003ac30 con 0x7fcae0107720 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:18.940+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fcae0107ba0 con 0x7fcac8038fc0 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.165+0000 7fcae4980640 1 -- 172.21.15.137:0/325550592 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fcae0107ba0 con 0x7fcac8038fc0 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcac8038fc0 msgr2=0x7fcac803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcac8038fc0 0x7fcac803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcad40098c0 tx=0x7fcad4006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 msgr2=0x7fcae019b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae019b720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcad00097f0 tx=0x7fcad00327b0 comp rx=0 tx=0).stop 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 shutdown_connections 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcac8038fc0 0x7fcac803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:20.585 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 --2- 172.21.15.137:0/325550592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0107720 0x7fcae019b720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:20.586 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.167+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 >> 172.21.15.137:0/325550592 conn(0x7fcae0102ef0 msgr2=0x7fcae0103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:20.586 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.168+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 shutdown_connections 2024-09-12T12:04:20.586 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:19.168+0000 7fcae5982640 1 -- 172.21.15.137:0/325550592 wait complete. 2024-09-12T12:04:20.586 INFO:teuthology.orchestra.run.smithi137.stdout:Fetching dashboard port number... 2024-09-12T12:04:21.454 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:21 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2519688868' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T12:04:21.736 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f226239e640 1 Processor -- start 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f226239e640 1 -- start start 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f226239e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c10a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f226239e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f225c07be30 con 0x7f225c1081e0 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f225bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c10a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f225bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c10a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45208/0 (socket says 172.21.15.137:45208) 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f225bfff640 1 -- 172.21.15.137:0/1848548374 learned_addr learned my addr 172.21.15.137:0/1848548374 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.972+0000 7f225bfff640 1 -- 172.21.15.137:0/1848548374 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c10ab10 con 0x7f225c1081e0 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.973+0000 7f225bfff640 1 --2- 172.21.15.137:0/1848548374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c10a5d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f224c009820 tx=0x7f224c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1c016d72d7398e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.973+0000 7f225affd640 1 -- 172.21.15.137:0/1848548374 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f224c037070 con 0x7f225c1081e0 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.973+0000 7f225affd640 1 -- 172.21.15.137:0/1848548374 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c032920 con 0x7f225c1081e0 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.973+0000 7f225affd640 1 -- 172.21.15.137:0/1848548374 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f224c030810 con 0x7f225c1081e0 2024-09-12T12:04:21.737 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 -- 172.21.15.137:0/1848548374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 msgr2=0x7f225c10a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 --2- 172.21.15.137:0/1848548374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c10a5d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f224c009820 tx=0x7f224c02a270 comp rx=0 tx=0).stop 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 -- 172.21.15.137:0/1848548374 shutdown_connections 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 --2- 172.21.15.137:0/1848548374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c10a5d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 -- 172.21.15.137:0/1848548374 >> 172.21.15.137:0/1848548374 conn(0x7f225c101e20 msgr2=0x7f225c104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 -- 172.21.15.137:0/1848548374 shutdown_connections 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 -- 172.21.15.137:0/1848548374 wait complete. 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 Processor -- start 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.974+0000 7f226239e640 1 -- start start 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f226239e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f226239e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f225c10bb30 con 0x7f225c1081e0 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f225bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f225bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c19b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45220/0 (socket says 172.21.15.137:45220) 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f225bfff640 1 -- 172.21.15.137:0/2519688868 learned_addr learned my addr 172.21.15.137:0/2519688868 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:21.738 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f225bfff640 1 -- 172.21.15.137:0/2519688868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c19bc60 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f225bfff640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c19b720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f224c030420 tx=0x7f224c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f22597fa640 1 -- 172.21.15.137:0/2519688868 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f224c037040 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c19bef0 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f22597fa640 1 -- 172.21.15.137:0/2519688868 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f224c040070 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f22597fa640 1 -- 172.21.15.137:0/2519688868 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f224c03a6e0 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.975+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c19ebe0 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.976+0000 7f22597fa640 1 -- 172.21.15.137:0/2519688868 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f224c03a9e0 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.976+0000 7f22597fa640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2244038fc0 0x7f224403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.976+0000 7f22597fa640 1 -- 172.21.15.137:0/2519688868 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f224c06c880 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.976+0000 7f225b7fe640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2244038fc0 0x7f224403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.976+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c108640 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.977+0000 7f225b7fe640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2244038fc0 0x7f224403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22500098c0 tx=0x7f2250006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:20.981+0000 7f22597fa640 1 -- 172.21.15.137:0/2519688868 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f224c041270 con 0x7f225c1081e0 2024-09-12T12:04:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.147+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f225c19f0c0 con 0x7f225c1081e0 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.147+0000 7f22597fa640 1 -- 172.21.15.137:0/2519688868 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f224c043bf0 con 0x7f225c1081e0 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.149+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2244038fc0 msgr2=0x7f224403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.149+0000 7f226239e640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2244038fc0 0x7f224403b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22500098c0 tx=0x7f2250006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.149+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 msgr2=0x7f225c19b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.149+0000 7f226239e640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c19b720 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f224c030420 tx=0x7f224c032670 comp rx=0 tx=0).stop 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.150+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 shutdown_connections 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.150+0000 7f226239e640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2244038fc0 0x7f224403b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.150+0000 7f226239e640 1 --2- 172.21.15.137:0/2519688868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f225c1081e0 0x7f225c19b720 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.150+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 >> 172.21.15.137:0/2519688868 conn(0x7f225c101e20 msgr2=0x7f225c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.150+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 shutdown_connections 2024-09-12T12:04:21.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:21.150+0000 7f226239e640 1 -- 172.21.15.137:0/2519688868 wait complete. 2024-09-12T12:04:21.744 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T12:04:21.744 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-09-12T12:04:21.752 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T12:04:21.752 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-09-12T12:04:21.752 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-09-12T12:04:21.752 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T12:04:21.754 INFO:teuthology.orchestra.run.smithi137.stdout:Ceph Dashboard is now available at: 2024-09-12T12:04:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: URL: https://smithi137.front.sepia.ceph.com:8443/ 2024-09-12T12:04:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: User: admin 2024-09-12T12:04:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: Password: udqhdt76zg 2024-09-12T12:04:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:21.755 INFO:teuthology.orchestra.run.smithi137.stdout:Saving cluster configuration to /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config directory 2024-09-12T12:04:22.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:22 smithi137 ceph-mon[30950]: mgrmap e13: x(active, since 6s) 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.178+0000 7fcd18988640 1 Processor -- start 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.178+0000 7fcd18988640 1 -- start start 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.179+0000 7fcd18988640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd141069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.179+0000 7fcd18988640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd14106fc0 con 0x7fcd141065f0 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.179+0000 7fcd1259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd141069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.179+0000 7fcd1259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd141069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45236/0 (socket says 172.21.15.137:45236) 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.179+0000 7fcd1259c640 1 -- 172.21.15.137:0/1943145493 learned_addr learned my addr 172.21.15.137:0/1943145493 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.179+0000 7fcd1259c640 1 -- 172.21.15.137:0/1943145493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd141077f0 con 0x7fcd141065f0 2024-09-12T12:04:23.061 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.180+0000 7fcd1259c640 1 --2- 172.21.15.137:0/1943145493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd141069f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009820 tx=0x7fcd0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c750df1e034dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.180+0000 7fcd1159a640 1 -- 172.21.15.137:0/1943145493 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x7fcd141065f0 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.180+0000 7fcd1159a640 1 -- 172.21.15.137:0/1943145493 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd04032920 con 0x7fcd141065f0 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.180+0000 7fcd1159a640 1 -- 172.21.15.137:0/1943145493 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd04030810 con 0x7fcd141065f0 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.181+0000 7fcd18988640 1 -- 172.21.15.137:0/1943145493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 msgr2=0x7fcd141069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.181+0000 7fcd18988640 1 --2- 172.21.15.137:0/1943145493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd141069f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009820 tx=0x7fcd0402a270 comp rx=0 tx=0).stop 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.181+0000 7fcd18988640 1 -- 172.21.15.137:0/1943145493 shutdown_connections 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.181+0000 7fcd18988640 1 --2- 172.21.15.137:0/1943145493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd141069f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.181+0000 7fcd18988640 1 -- 172.21.15.137:0/1943145493 >> 172.21.15.137:0/1943145493 conn(0x7fcd14101e20 msgr2=0x7fcd14104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.181+0000 7fcd18988640 1 -- 172.21.15.137:0/1943145493 shutdown_connections 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.181+0000 7fcd18988640 1 -- 172.21.15.137:0/1943145493 wait complete. 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd18988640 1 Processor -- start 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd18988640 1 -- start start 2024-09-12T12:04:23.062 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd18988640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd1419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd18988640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1410aba0 con 0x7fcd141065f0 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd1259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd1419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd1259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd1419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45242/0 (socket says 172.21.15.137:45242) 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd1259c640 1 -- 172.21.15.137:0/2095969488 learned_addr learned my addr 172.21.15.137:0/2095969488 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd1259c640 1 -- 172.21.15.137:0/2095969488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1419bcc0 con 0x7fcd141065f0 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.182+0000 7fcd1259c640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd1419b780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097f0 tx=0x7fcd040327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.183+0000 7fcd037fe640 1 -- 172.21.15.137:0/2095969488 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd04037040 con 0x7fcd141065f0 2024-09-12T12:04:23.063 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.183+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1419bf50 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.183+0000 7fcd037fe640 1 -- 172.21.15.137:0/2095969488 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd04030c40 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.183+0000 7fcd037fe640 1 -- 172.21.15.137:0/2095969488 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd0403a5b0 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.183+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1419ec40 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.184+0000 7fcd037fe640 1 -- 172.21.15.137:0/2095969488 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcd0403a7f0 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.184+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1410ad20 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.184+0000 7fcd037fe640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fccfc039060 0x7fccfc03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.184+0000 7fcd037fe640 1 -- 172.21.15.137:0/2095969488 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcd0406c5b0 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.184+0000 7fcd11d9b640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fccfc039060 0x7fccfc03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.184+0000 7fcd11d9b640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fccfc039060 0x7fccfc03b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd080098c0 tx=0x7fcd08006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.188+0000 7fcd037fe640 1 -- 172.21.15.137:0/2095969488 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd04039950 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.444+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fcd1402d9b0 con 0x7fcd141065f0 2024-09-12T12:04:23.064 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.446+0000 7fcd037fe640 1 -- 172.21.15.137:0/2095969488 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fcd040371e0 con 0x7fcd141065f0 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fccfc039060 msgr2=0x7fccfc03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fccfc039060 0x7fccfc03b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcd080098c0 tx=0x7fcd08006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 msgr2=0x7fcd1419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd1419b780 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097f0 tx=0x7fcd040327b0 comp rx=0 tx=0).stop 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 shutdown_connections 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fccfc039060 0x7fccfc03b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 --2- 172.21.15.137:0/2095969488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd141065f0 0x7fcd1419b780 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 >> 172.21.15.137:0/2095969488 conn(0x7fcd14101e20 msgr2=0x7fcd14102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 shutdown_connections 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-09-12T12:04:22.448+0000 7fcd18988640 1 -- 172.21.15.137:0/2095969488 wait complete. 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T12:04:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: ceph telemetry on 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout:For more information see: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:23.066 INFO:teuthology.orchestra.run.smithi137.stdout:Bootstrap complete. 2024-09-12T12:04:23.100 INFO:tasks.cephadm:Fetching config... 2024-09-12T12:04:23.101 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:04:23.101 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T12:04:23.116 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T12:04:23.117 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:04:23.117 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T12:04:23.173 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T12:04:23.174 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:04:23.174 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T12:04:23.238 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T12:04:23.239 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:04:23.239 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T12:04:23.295 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T12:04:23.295 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwBOjGHkW9ilto0ip4eQAM0pq5xXoHEio1dnmOOLmK6o6aJt6zs3w1csOWPRUSjQKw0l+5+tX4GjQ/R185dCgi3OslScx8u8r65Qwe2G9It1pHooU57Omqb4dcAwn9cZ0/5S2t3pcpGr1GFIux1MEO4GkCvZa1m9SF7j7Q8c5t2eTal0SX5eTzivdKZEDrQM4XH+hh7CD9F+apB2MyjQfk18p/iuBmWsiZxg1V1LxSBQ+3CL0w7DHRjKH3iRYD0+P+FttsauxKy/Fq0r4inAgLESnWVetX8w4/AQKQQwehTwV0sxGB6096xw/WdYOUXeJ164ZiMmrDLsFhbSmLJHCdZ8iIzGohEKcGnN7kH3SRIBvNFybhzOagzLcJIhYXUlETh4xyIikQFDd6RGPA24pOtwdRnx65arRAVDte6H8MY6tTHJlvcZZllFHvoPcjFbvNmyK1lrL4tkgF5TjAe112j+3BENF8MUT4b/2PBUEsIWFuSGo6SmuuGOCiEkTdM4M= ceph-dd66268a-70fe-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T12:04:23.369 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwBOjGHkW9ilto0ip4eQAM0pq5xXoHEio1dnmOOLmK6o6aJt6zs3w1csOWPRUSjQKw0l+5+tX4GjQ/R185dCgi3OslScx8u8r65Qwe2G9It1pHooU57Omqb4dcAwn9cZ0/5S2t3pcpGr1GFIux1MEO4GkCvZa1m9SF7j7Q8c5t2eTal0SX5eTzivdKZEDrQM4XH+hh7CD9F+apB2MyjQfk18p/iuBmWsiZxg1V1LxSBQ+3CL0w7DHRjKH3iRYD0+P+FttsauxKy/Fq0r4inAgLESnWVetX8w4/AQKQQwehTwV0sxGB6096xw/WdYOUXeJ164ZiMmrDLsFhbSmLJHCdZ8iIzGohEKcGnN7kH3SRIBvNFybhzOagzLcJIhYXUlETh4xyIikQFDd6RGPA24pOtwdRnx65arRAVDte6H8MY6tTHJlvcZZllFHvoPcjFbvNmyK1lrL4tkgF5TjAe112j+3BENF8MUT4b/2PBUEsIWFuSGo6SmuuGOCiEkTdM4M= ceph-dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:23.380 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwBOjGHkW9ilto0ip4eQAM0pq5xXoHEio1dnmOOLmK6o6aJt6zs3w1csOWPRUSjQKw0l+5+tX4GjQ/R185dCgi3OslScx8u8r65Qwe2G9It1pHooU57Omqb4dcAwn9cZ0/5S2t3pcpGr1GFIux1MEO4GkCvZa1m9SF7j7Q8c5t2eTal0SX5eTzivdKZEDrQM4XH+hh7CD9F+apB2MyjQfk18p/iuBmWsiZxg1V1LxSBQ+3CL0w7DHRjKH3iRYD0+P+FttsauxKy/Fq0r4inAgLESnWVetX8w4/AQKQQwehTwV0sxGB6096xw/WdYOUXeJ164ZiMmrDLsFhbSmLJHCdZ8iIzGohEKcGnN7kH3SRIBvNFybhzOagzLcJIhYXUlETh4xyIikQFDd6RGPA24pOtwdRnx65arRAVDte6H8MY6tTHJlvcZZllFHvoPcjFbvNmyK1lrL4tkgF5TjAe112j+3BENF8MUT4b/2PBUEsIWFuSGo6SmuuGOCiEkTdM4M= ceph-dd66268a-70fe-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T12:04:23.422 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwBOjGHkW9ilto0ip4eQAM0pq5xXoHEio1dnmOOLmK6o6aJt6zs3w1csOWPRUSjQKw0l+5+tX4GjQ/R185dCgi3OslScx8u8r65Qwe2G9It1pHooU57Omqb4dcAwn9cZ0/5S2t3pcpGr1GFIux1MEO4GkCvZa1m9SF7j7Q8c5t2eTal0SX5eTzivdKZEDrQM4XH+hh7CD9F+apB2MyjQfk18p/iuBmWsiZxg1V1LxSBQ+3CL0w7DHRjKH3iRYD0+P+FttsauxKy/Fq0r4inAgLESnWVetX8w4/AQKQQwehTwV0sxGB6096xw/WdYOUXeJ164ZiMmrDLsFhbSmLJHCdZ8iIzGohEKcGnN7kH3SRIBvNFybhzOagzLcJIhYXUlETh4xyIikQFDd6RGPA24pOtwdRnx65arRAVDte6H8MY6tTHJlvcZZllFHvoPcjFbvNmyK1lrL4tkgF5TjAe112j+3BENF8MUT4b/2PBUEsIWFuSGo6SmuuGOCiEkTdM4M= ceph-dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:23.434 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwBOjGHkW9ilto0ip4eQAM0pq5xXoHEio1dnmOOLmK6o6aJt6zs3w1csOWPRUSjQKw0l+5+tX4GjQ/R185dCgi3OslScx8u8r65Qwe2G9It1pHooU57Omqb4dcAwn9cZ0/5S2t3pcpGr1GFIux1MEO4GkCvZa1m9SF7j7Q8c5t2eTal0SX5eTzivdKZEDrQM4XH+hh7CD9F+apB2MyjQfk18p/iuBmWsiZxg1V1LxSBQ+3CL0w7DHRjKH3iRYD0+P+FttsauxKy/Fq0r4inAgLESnWVetX8w4/AQKQQwehTwV0sxGB6096xw/WdYOUXeJ164ZiMmrDLsFhbSmLJHCdZ8iIzGohEKcGnN7kH3SRIBvNFybhzOagzLcJIhYXUlETh4xyIikQFDd6RGPA24pOtwdRnx65arRAVDte6H8MY6tTHJlvcZZllFHvoPcjFbvNmyK1lrL4tkgF5TjAe112j+3BENF8MUT4b/2PBUEsIWFuSGo6SmuuGOCiEkTdM4M= ceph-dd66268a-70fe-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T12:04:23.476 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwBOjGHkW9ilto0ip4eQAM0pq5xXoHEio1dnmOOLmK6o6aJt6zs3w1csOWPRUSjQKw0l+5+tX4GjQ/R185dCgi3OslScx8u8r65Qwe2G9It1pHooU57Omqb4dcAwn9cZ0/5S2t3pcpGr1GFIux1MEO4GkCvZa1m9SF7j7Q8c5t2eTal0SX5eTzivdKZEDrQM4XH+hh7CD9F+apB2MyjQfk18p/iuBmWsiZxg1V1LxSBQ+3CL0w7DHRjKH3iRYD0+P+FttsauxKy/Fq0r4inAgLESnWVetX8w4/AQKQQwehTwV0sxGB6096xw/WdYOUXeJ164ZiMmrDLsFhbSmLJHCdZ8iIzGohEKcGnN7kH3SRIBvNFybhzOagzLcJIhYXUlETh4xyIikQFDd6RGPA24pOtwdRnx65arRAVDte6H8MY6tTHJlvcZZllFHvoPcjFbvNmyK1lrL4tkgF5TjAe112j+3BENF8MUT4b/2PBUEsIWFuSGo6SmuuGOCiEkTdM4M= ceph-dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:23.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T12:04:23.639 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:23 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2095969488' entity='client.admin' 2024-09-12T12:04:23.736 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 -- 172.21.15.137:0/4085285773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 msgr2=0x7f378c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 --2- 172.21.15.137:0/4085285773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c101a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3784009880 tx=0x7f378402a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 -- 172.21.15.137:0/4085285773 shutdown_connections 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 --2- 172.21.15.137:0/4085285773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c101a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 -- 172.21.15.137:0/4085285773 >> 172.21.15.137:0/4085285773 conn(0x7f378c0fcdf0 msgr2=0x7f378c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 -- 172.21.15.137:0/4085285773 shutdown_connections 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 -- 172.21.15.137:0/4085285773 wait complete. 2024-09-12T12:04:24.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 Processor -- start 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 -- start start 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.238+0000 7f3793be1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f378c107750 con 0x7f378c101640 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f379197d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f379197d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45250/0 (socket says 172.21.15.137:45250) 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f379197d640 1 -- 172.21.15.137:0/976658019 learned_addr learned my addr 172.21.15.137:0/976658019 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f379197d640 1 -- 172.21.15.137:0/976658019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c197390 con 0x7f378c101640 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f379197d640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c196e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3784009230 tx=0x7f37840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f378affd640 1 -- 172.21.15.137:0/976658019 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f378403f070 con 0x7f378c101640 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c197620 con 0x7f378c101640 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f378affd640 1 -- 172.21.15.137:0/976658019 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3784003ad0 con 0x7f378c101640 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f378affd640 1 -- 172.21.15.137:0/976658019 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3784035030 con 0x7f378c101640 2024-09-12T12:04:24.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.239+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c19a310 con 0x7f378c101640 2024-09-12T12:04:24.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.240+0000 7f378affd640 1 -- 172.21.15.137:0/976658019 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3784042070 con 0x7f378c101640 2024-09-12T12:04:24.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.240+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378c107ed0 con 0x7f378c101640 2024-09-12T12:04:24.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.241+0000 7f378affd640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3774039060 0x7f377403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:24.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.241+0000 7f378affd640 1 -- 172.21.15.137:0/976658019 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f378406cd20 con 0x7f378c101640 2024-09-12T12:04:24.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.241+0000 7f379117c640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3774039060 0x7f377403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:24.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.241+0000 7f379117c640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3774039060 0x7f377403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37800098c0 tx=0x7f3780006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:24.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.249+0000 7f378affd640 1 -- 172.21.15.137:0/976658019 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f378404c350 con 0x7f378c101640 2024-09-12T12:04:24.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.425+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f378c02d830 con 0x7f378c101640 2024-09-12T12:04:24.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.428+0000 7f378affd640 1 -- 172.21.15.137:0/976658019 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3784030d90 con 0x7f378c101640 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3774039060 msgr2=0x7f377403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3774039060 0x7f377403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37800098c0 tx=0x7f3780006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 msgr2=0x7f378c196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c196e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3784009230 tx=0x7f37840048f0 comp rx=0 tx=0).stop 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 shutdown_connections 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3774039060 0x7f377403b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 --2- 172.21.15.137:0/976658019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f378c101640 0x7f378c196e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 >> 172.21.15.137:0/976658019 conn(0x7f378c0fcdf0 msgr2=0x7f378c0fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 shutdown_connections 2024-09-12T12:04:24.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:24.431+0000 7f3793be1640 1 -- 172.21.15.137:0/976658019 wait complete. 2024-09-12T12:04:25.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:25 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/976658019' entity='client.admin' 2024-09-12T12:04:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:04:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:25.536 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T12:04:25.536 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T12:04:25.776 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:04:26.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 -- 172.21.15.137:0/2190365643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 msgr2=0x7f98fc100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:26.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 --2- 172.21.15.137:0/2190365643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc100920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009880 tx=0x7f98f802a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:26.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 -- 172.21.15.137:0/2190365643 shutdown_connections 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 --2- 172.21.15.137:0/2190365643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc100920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 -- 172.21.15.137:0/2190365643 >> 172.21.15.137:0/2190365643 conn(0x7f98fc0fbcb0 msgr2=0x7f98fc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 -- 172.21.15.137:0/2190365643 shutdown_connections 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 -- 172.21.15.137:0/2190365643 wait complete. 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.239+0000 7f99045d3640 1 Processor -- start 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f99045d3640 1 -- start start 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f99045d3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f99045d3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc106630 con 0x7f98fc100520 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f990236f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f990236f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45262/0 (socket says 172.21.15.137:45262) 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f990236f640 1 -- 172.21.15.137:0/1820836149 learned_addr learned my addr 172.21.15.137:0/1820836149 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f990236f640 1 -- 172.21.15.137:0/1820836149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc1972f0 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f990236f640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f8004f60 tx=0x7f98f80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f98f37fe640 1 -- 172.21.15.137:0/1820836149 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f803f070 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f98f37fe640 1 -- 172.21.15.137:0/1820836149 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98f8004d40 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc197580 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f98f37fe640 1 -- 172.21.15.137:0/1820836149 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98f8035030 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.240+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc19a230 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.241+0000 7f98f37fe640 1 -- 172.21.15.137:0/1820836149 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f98f8042070 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.241+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc106db0 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.241+0000 7f98f37fe640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98e8039060 0x7f98e803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.241+0000 7f98f37fe640 1 -- 172.21.15.137:0/1820836149 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f98f806cd50 con 0x7f98fc100520 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.242+0000 7f9901b6e640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98e8039060 0x7f98e803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.242+0000 7f9901b6e640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98e8039060 0x7f98e803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0098c0 tx=0x7f98ec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:26.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.246+0000 7f98f37fe640 1 -- 172.21.15.137:0/1820836149 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98f8070030 con 0x7f98fc100520 2024-09-12T12:04:26.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.416+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f98fc0fd4c0 con 0x7f98e8039060 2024-09-12T12:04:26.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.419+0000 7f98f37fe640 1 -- 172.21.15.137:0/1820836149 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98fc0fd4c0 con 0x7f98e8039060 2024-09-12T12:04:26.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98e8039060 msgr2=0x7f98e803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98e8039060 0x7f98e803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0098c0 tx=0x7f98ec006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 msgr2=0x7f98fc196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f8004f60 tx=0x7f98f80048f0 comp rx=0 tx=0).stop 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 shutdown_connections 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98e8039060 0x7f98e803b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 --2- 172.21.15.137:0/1820836149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f98fc100520 0x7f98fc196db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 >> 172.21.15.137:0/1820836149 conn(0x7f98fc0fbcb0 msgr2=0x7f98fc0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 shutdown_connections 2024-09-12T12:04:26.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:26.422+0000 7f99045d3640 1 -- 172.21.15.137:0/1820836149 wait complete. 2024-09-12T12:04:27.326 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-09-12T12:04:27.327 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:04:27.327 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-09-12T12:04:27.346 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:04:27.346 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:27.404 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-09-12T12:04:27.404 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch host add smithi171 2024-09-12T12:04:27.641 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:04:27.761 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:27.762 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 -- 172.21.15.137:0/2851187208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 msgr2=0x7f506c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 --2- 172.21.15.137:0/2851187208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c106970 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5068009880 tx=0x7f506802a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 -- 172.21.15.137:0/2851187208 shutdown_connections 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 --2- 172.21.15.137:0/2851187208 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c106970 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 -- 172.21.15.137:0/2851187208 >> 172.21.15.137:0/2851187208 conn(0x7f506c0fc1a0 msgr2=0x7f506c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 -- 172.21.15.137:0/2851187208 shutdown_connections 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 -- 172.21.15.137:0/2851187208 wait complete. 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 Processor -- start 2024-09-12T12:04:28.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.094+0000 7f50743d6640 1 -- start start 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f50743d6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f50743d6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f506c10abc0 con 0x7f506c106590 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f5072172640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f5072172640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45286/0 (socket says 172.21.15.137:45286) 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f5072172640 1 -- 172.21.15.137:0/1395778406 learned_addr learned my addr 172.21.15.137:0/1395778406 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f5072172640 1 -- 172.21.15.137:0/1395778406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506c196530 con 0x7f506c106590 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f5072172640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c195ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5068009230 tx=0x7f5068003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f50637fe640 1 -- 172.21.15.137:0/1395778406 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f506803f070 con 0x7f506c106590 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506c1967c0 con 0x7f506c106590 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f50637fe640 1 -- 172.21.15.137:0/1395778406 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506803bc80 con 0x7f506c106590 2024-09-12T12:04:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f50637fe640 1 -- 172.21.15.137:0/1395778406 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5068035030 con 0x7f506c106590 2024-09-12T12:04:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.095+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506c19a8c0 con 0x7f506c106590 2024-09-12T12:04:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.096+0000 7f50637fe640 1 -- 172.21.15.137:0/1395778406 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5068042070 con 0x7f506c106590 2024-09-12T12:04:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.096+0000 7f50637fe640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f5058039060 0x7f505803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.096+0000 7f50637fe640 1 -- 172.21.15.137:0/1395778406 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f506806cc90 con 0x7f506c106590 2024-09-12T12:04:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.096+0000 7f5071971640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f5058039060 0x7f505803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.096+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506c101c00 con 0x7f506c106590 2024-09-12T12:04:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.097+0000 7f5071971640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f5058039060 0x7f505803b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f505c0098c0 tx=0x7f505c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.101+0000 7f50637fe640 1 -- 172.21.15.137:0/1395778406 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f506804c350 con 0x7f506c106590 2024-09-12T12:04:28.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:28.273+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) -- 0x7f506c106a20 con 0x7f5058039060 2024-09-12T12:04:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:29 smithi137 ceph-mon[30950]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:29 smithi137 ceph-mon[30950]: Deploying cephadm binary to smithi171 2024-09-12T12:04:31.446 INFO:teuthology.orchestra.run.smithi137.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-09-12T12:04:31.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.445+0000 7f50637fe640 1 -- 172.21.15.137:0/1395778406 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f506c106a20 con 0x7f5058039060 2024-09-12T12:04:31.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f5058039060 msgr2=0x7f505803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f5058039060 0x7f505803b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f505c0098c0 tx=0x7f505c006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 msgr2=0x7f506c195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c195ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5068009230 tx=0x7f5068003d40 comp rx=0 tx=0).stop 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 shutdown_connections 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f5058039060 0x7f505803b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 --2- 172.21.15.137:0/1395778406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f506c106590 0x7f506c195ff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.448+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 >> 172.21.15.137:0/1395778406 conn(0x7f506c0fc1a0 msgr2=0x7f506c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.449+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 shutdown_connections 2024-09-12T12:04:31.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:31.449+0000 7f50743d6640 1 -- 172.21.15.137:0/1395778406 wait complete. 2024-09-12T12:04:32.070 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T12:04:32.309 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:04:32.703 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:32.704 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:32 smithi137 ceph-mon[30950]: Added host smithi171 2024-09-12T12:04:32.704 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:32.704 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:32.704 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:32.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.784+0000 7f4928fe7640 1 -- 172.21.15.137:0/337643846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 msgr2=0x7f4924100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:32.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.784+0000 7f4928fe7640 1 --2- 172.21.15.137:0/337643846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f4924100d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4914009880 tx=0x7f491402a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:32.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.784+0000 7f4928fe7640 1 -- 172.21.15.137:0/337643846 shutdown_connections 2024-09-12T12:04:32.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.784+0000 7f4928fe7640 1 --2- 172.21.15.137:0/337643846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f4924100d70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:32.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.784+0000 7f4928fe7640 1 -- 172.21.15.137:0/337643846 >> 172.21.15.137:0/337643846 conn(0x7f4924076250 msgr2=0x7f4924076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:32.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.784+0000 7f4928fe7640 1 -- 172.21.15.137:0/337643846 shutdown_connections 2024-09-12T12:04:32.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.784+0000 7f4928fe7640 1 -- 172.21.15.137:0/337643846 wait complete. 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f4928fe7640 1 Processor -- start 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f4928fe7640 1 -- start start 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f4928fe7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f49241980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f4928fe7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f492410ab90 con 0x7f4924100990 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f492259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f49241980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f492259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f49241980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33668/0 (socket says 172.21.15.137:33668) 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f492259c640 1 -- 172.21.15.137:0/571173906 learned_addr learned my addr 172.21.15.137:0/571173906 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f492259c640 1 -- 172.21.15.137:0/571173906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49241985e0 con 0x7f4924100990 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f492259c640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f49241980a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4914009230 tx=0x7f4914003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.785+0000 7f491b7fe640 1 -- 172.21.15.137:0/571173906 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f491403f070 con 0x7f4924100990 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.786+0000 7f491b7fe640 1 -- 172.21.15.137:0/571173906 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f491403bc80 con 0x7f4924100990 2024-09-12T12:04:32.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.786+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49241988d0 con 0x7f4924100990 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.786+0000 7f491b7fe640 1 -- 172.21.15.137:0/571173906 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4914035030 con 0x7f4924100990 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.786+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4924198d90 con 0x7f4924100990 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.786+0000 7f491b7fe640 1 -- 172.21.15.137:0/571173906 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4914042070 con 0x7f4924100990 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.787+0000 7f491b7fe640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f49100390c0 0x7f491003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.786+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4924101c40 con 0x7f4924100990 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.787+0000 7f491b7fe640 1 -- 172.21.15.137:0/571173906 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f491406cc90 con 0x7f4924100990 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.787+0000 7f4921d9b640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f49100390c0 0x7f491003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.787+0000 7f4921d9b640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f49100390c0 0x7f491003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f490c0098c0 tx=0x7f490c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:32.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.791+0000 7f491b7fe640 1 -- 172.21.15.137:0/571173906 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f491404a260 con 0x7f4924100990 2024-09-12T12:04:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.963+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4924100df0 con 0x7f49100390c0 2024-09-12T12:04:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.965+0000 7f491b7fe640 1 -- 172.21.15.137:0/571173906 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f4924100df0 con 0x7f49100390c0 2024-09-12T12:04:32.969 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:32.969 INFO:teuthology.orchestra.run.smithi137.stdout:[{"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-09-12T12:04:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f49100390c0 msgr2=0x7f491003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f49100390c0 0x7f491003b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f490c0098c0 tx=0x7f490c006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 msgr2=0x7f49241980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f49241980a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4914009230 tx=0x7f4914003d40 comp rx=0 tx=0).stop 2024-09-12T12:04:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 shutdown_connections 2024-09-12T12:04:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f49100390c0 0x7f491003b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 --2- 172.21.15.137:0/571173906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4924100990 0x7f49241980a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 >> 172.21.15.137:0/571173906 conn(0x7f4924076250 msgr2=0x7f49241020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 shutdown_connections 2024-09-12T12:04:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:32.967+0000 7f4928fe7640 1 -- 172.21.15.137:0/571173906 wait complete. 2024-09-12T12:04:33.622 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-09-12T12:04:33.622 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:04:33.622 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-09-12T12:04:33.641 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:04:33.642 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:33.700 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-09-12T12:04:33.700 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch host add smithi189 2024-09-12T12:04:33.946 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:04:34.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:34.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:33 smithi137 ceph-mon[30950]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:04:34.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 -- 172.21.15.137:0/2261409578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 msgr2=0x7f7990107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:34.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 --2- 172.21.15.137:0/2261409578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990107a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7980009880 tx=0x7f798002a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:34.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 -- 172.21.15.137:0/2261409578 shutdown_connections 2024-09-12T12:04:34.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 --2- 172.21.15.137:0/2261409578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990107a70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:34.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 -- 172.21.15.137:0/2261409578 >> 172.21.15.137:0/2261409578 conn(0x7f7990076250 msgr2=0x7f7990076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 -- 172.21.15.137:0/2261409578 shutdown_connections 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 -- 172.21.15.137:0/2261409578 wait complete. 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 Processor -- start 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.374+0000 7f7998073640 1 -- start start 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7998073640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7998073640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f799010bce0 con 0x7f7990107690 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7995e0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7995e0f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33684/0 (socket says 172.21.15.137:33684) 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7995e0f640 1 -- 172.21.15.137:0/2718697537 learned_addr learned my addr 172.21.15.137:0/2718697537 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7995e0f640 1 -- 172.21.15.137:0/2718697537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7990197650 con 0x7f7990107690 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7995e0f640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7980009230 tx=0x7f7980003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:34.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7986ffd640 1 -- 172.21.15.137:0/2718697537 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f798003f070 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7986ffd640 1 -- 172.21.15.137:0/2718697537 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f798003bc80 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901978e0 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7986ffd640 1 -- 172.21.15.137:0/2718697537 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7980035030 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.375+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799019b9e0 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.376+0000 7f7986ffd640 1 -- 172.21.15.137:0/2718697537 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7980042070 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.376+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7990102d30 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.376+0000 7f7986ffd640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f797c039060 0x7f797c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.377+0000 7f7986ffd640 1 -- 172.21.15.137:0/2718697537 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f798006cda0 con 0x7f7990107690 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.377+0000 7f799560e640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f797c039060 0x7f797c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:34.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.377+0000 7f799560e640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f797c039060 0x7f797c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f798c0098c0 tx=0x7f798c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:34.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.382+0000 7f7986ffd640 1 -- 172.21.15.137:0/2718697537 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79800372d0 con 0x7f7990107690 2024-09-12T12:04:34.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:34.551+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) -- 0x7f7990198470 con 0x7f797c039060 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:04:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: Deploying cephadm binary to smithi189 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:37.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.662+0000 7f7986ffd640 1 -- 172.21.15.137:0/2718697537 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7990198470 con 0x7f797c039060 2024-09-12T12:04:37.662 INFO:teuthology.orchestra.run.smithi137.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-09-12T12:04:37.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f797c039060 msgr2=0x7f797c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:37.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f797c039060 0x7f797c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f798c0098c0 tx=0x7f798c006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:37.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 msgr2=0x7f7990197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:37.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7980009230 tx=0x7f7980003d40 comp rx=0 tx=0).stop 2024-09-12T12:04:37.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 shutdown_connections 2024-09-12T12:04:37.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f797c039060 0x7f797c03b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:37.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 --2- 172.21.15.137:0/2718697537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7990107690 0x7f7990197110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:37.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.664+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 >> 172.21.15.137:0/2718697537 conn(0x7f7990076250 msgr2=0x7f79900fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:37.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.665+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 shutdown_connections 2024-09-12T12:04:37.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:37.665+0000 7f7998073640 1 -- 172.21.15.137:0/2718697537 wait complete. 2024-09-12T12:04:38.247 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:38 smithi137 ceph-mon[30950]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:38.248 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:38 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:38.248 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:38 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:38.248 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:38 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:38.275 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T12:04:38.514 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.982+0000 7f723d5ec640 1 -- 172.21.15.137:0/3632017987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 msgr2=0x7f7238107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.982+0000 7f723d5ec640 1 --2- 172.21.15.137:0/3632017987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f7238107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7228009880 tx=0x7f722802a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.982+0000 7f723d5ec640 1 -- 172.21.15.137:0/3632017987 shutdown_connections 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.982+0000 7f723d5ec640 1 --2- 172.21.15.137:0/3632017987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f7238107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.982+0000 7f723d5ec640 1 -- 172.21.15.137:0/3632017987 >> 172.21.15.137:0/3632017987 conn(0x7f72380fd2b0 msgr2=0x7f72380ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.982+0000 7f723d5ec640 1 -- 172.21.15.137:0/3632017987 shutdown_connections 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.982+0000 7f723d5ec640 1 -- 172.21.15.137:0/3632017987 wait complete. 2024-09-12T12:04:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f723d5ec640 1 Processor -- start 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f723d5ec640 1 -- start start 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f723d5ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f72380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f723d5ec640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f723810bc70 con 0x7f7238107660 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f7236ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f72380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f7236ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f72380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33696/0 (socket says 172.21.15.137:33696) 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f7236ffd640 1 -- 172.21.15.137:0/4238343434 learned_addr learned my addr 172.21.15.137:0/4238343434 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.983+0000 7f7236ffd640 1 -- 172.21.15.137:0/4238343434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72380763e0 con 0x7f7238107660 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.984+0000 7f7236ffd640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f72380744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7228009230 tx=0x7f7228003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.984+0000 7f721ffff640 1 -- 172.21.15.137:0/4238343434 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f722803f070 con 0x7f7238107660 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.984+0000 7f721ffff640 1 -- 172.21.15.137:0/4238343434 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f722803bc80 con 0x7f7238107660 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.984+0000 7f721ffff640 1 -- 172.21.15.137:0/4238343434 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7228035030 con 0x7f7238107660 2024-09-12T12:04:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.984+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7238074a90 con 0x7f7238107660 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.984+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7238074f50 con 0x7f7238107660 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.985+0000 7f721ffff640 1 -- 172.21.15.137:0/4238343434 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7228042070 con 0x7f7238107660 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.985+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7238102d10 con 0x7f7238107660 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.985+0000 7f721ffff640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7220039060 0x7f722003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.985+0000 7f721ffff640 1 -- 172.21.15.137:0/4238343434 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f722806ce00 con 0x7f7238107660 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.985+0000 7f72367fc640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7220039060 0x7f722003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.985+0000 7f72367fc640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7220039060 0x7f722003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f722c0098c0 tx=0x7f722c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:38.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:38.990+0000 7f721ffff640 1 -- 172.21.15.137:0/4238343434 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f722804c350 con 0x7f7238107660 2024-09-12T12:04:39.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.161+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7238075a20 con 0x7f7220039060 2024-09-12T12:04:39.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.162+0000 7f721ffff640 1 -- 172.21.15.137:0/4238343434 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f7238075a20 con 0x7f7220039060 2024-09-12T12:04:39.161 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:04:39.161 INFO:teuthology.orchestra.run.smithi137.stdout:[{"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-09-12T12:04:39.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7220039060 msgr2=0x7f722003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:39.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7220039060 0x7f722003b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f722c0098c0 tx=0x7f722c006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:39.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 msgr2=0x7f72380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:39.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f72380744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7228009230 tx=0x7f7228003d40 comp rx=0 tx=0).stop 2024-09-12T12:04:39.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 shutdown_connections 2024-09-12T12:04:39.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7220039060 0x7f722003b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:39.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 --2- 172.21.15.137:0/4238343434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7238107660 0x7f72380744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:39.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 >> 172.21.15.137:0/4238343434 conn(0x7f72380fd2b0 msgr2=0x7f72380fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:39.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 shutdown_connections 2024-09-12T12:04:39.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:39.164+0000 7f723d5ec640 1 -- 172.21.15.137:0/4238343434 wait complete. 2024-09-12T12:04:39.462 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:39 smithi137 ceph-mon[30950]: Added host smithi189 2024-09-12T12:04:39.462 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:39 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:39.462 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:39 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:39.766 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T12:04:39.766 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T12:04:40.006 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:04:40.398 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:40 smithi137 ceph-mon[30950]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:40.398 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:40 smithi137 ceph-mon[30950]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:04:40.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.479+0000 7f1ca2202640 1 -- 172.21.15.137:0/1070974507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 msgr2=0x7f1c9c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.479+0000 7f1ca2202640 1 --2- 172.21.15.137:0/1070974507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c106970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009880 tx=0x7f1c8c02a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.479+0000 7f1ca2202640 1 -- 172.21.15.137:0/1070974507 shutdown_connections 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.479+0000 7f1ca2202640 1 --2- 172.21.15.137:0/1070974507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c106970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.479+0000 7f1ca2202640 1 -- 172.21.15.137:0/1070974507 >> 172.21.15.137:0/1070974507 conn(0x7f1c9c0fc1a0 msgr2=0x7f1c9c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.479+0000 7f1ca2202640 1 -- 172.21.15.137:0/1070974507 shutdown_connections 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.479+0000 7f1ca2202640 1 -- 172.21.15.137:0/1070974507 wait complete. 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1ca2202640 1 Processor -- start 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1ca2202640 1 -- start start 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1ca2202640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1ca2202640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9c10abc0 con 0x7f1c9c106590 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1c9b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55040/0 (socket says 172.21.15.137:55040) 2024-09-12T12:04:40.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1c9b7fe640 1 -- 172.21.15.137:0/242114454 learned_addr learned my addr 172.21.15.137:0/242114454 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1c9b7fe640 1 -- 172.21.15.137:0/242114454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9c06ace0 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1c9b7fe640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c195d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009230 tx=0x7f1c8c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.480+0000 7f1c997fa640 1 -- 172.21.15.137:0/242114454 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c8c03f070 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.481+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9c196290 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.481+0000 7f1c997fa640 1 -- 172.21.15.137:0/242114454 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c8c03bc80 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.481+0000 7f1c997fa640 1 -- 172.21.15.137:0/242114454 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c8c035030 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.481+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9c1966f0 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.481+0000 7f1c997fa640 1 -- 172.21.15.137:0/242114454 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1c8c042070 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.481+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9c101c00 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.482+0000 7f1c997fa640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1c88039060 0x7f1c8803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.482+0000 7f1c997fa640 1 -- 172.21.15.137:0/242114454 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c8c06cda0 con 0x7f1c9c106590 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.482+0000 7f1c93fff640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1c88039060 0x7f1c8803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:40.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.482+0000 7f1c93fff640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1c88039060 0x7f1c8803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c840098c0 tx=0x7f1c84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:40.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.486+0000 7f1c997fa640 1 -- 172.21.15.137:0/242114454 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c8c03ac60 con 0x7f1c9c106590 2024-09-12T12:04:40.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:40.656+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1c9c06a4d0 con 0x7f1c9c106590 2024-09-12T12:04:41.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.249+0000 7f1c997fa640 1 -- 172.21.15.137:0/242114454 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1c8c03b7a0 con 0x7f1c9c106590 2024-09-12T12:04:41.249 INFO:teuthology.orchestra.run.smithi137.stderr:adjusted tunables profile to default 2024-09-12T12:04:41.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1c88039060 msgr2=0x7f1c8803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:41.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1c88039060 0x7f1c8803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c840098c0 tx=0x7f1c84006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:41.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 msgr2=0x7f1c9c195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:41.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c195d50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009230 tx=0x7f1c8c003d40 comp rx=0 tx=0).stop 2024-09-12T12:04:41.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 shutdown_connections 2024-09-12T12:04:41.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1c88039060 0x7f1c8803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:41.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 --2- 172.21.15.137:0/242114454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c9c106590 0x7f1c9c195d50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:41.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 >> 172.21.15.137:0/242114454 conn(0x7f1c9c0fc1a0 msgr2=0x7f1c9c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:41.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 shutdown_connections 2024-09-12T12:04:41.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:04:41.252+0000 7f1ca2202640 1 -- 172.21.15.137:0/242114454 wait complete. 2024-09-12T12:04:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:41 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/242114454' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T12:04:41.902 INFO:tasks.cephadm:Adding mon.a on smithi137 2024-09-12T12:04:41.902 INFO:tasks.cephadm:Adding mon.b on smithi171 2024-09-12T12:04:41.902 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-09-12T12:04:41.902 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c' 2024-09-12T12:04:42.148 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/242114454' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:04:42.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:42.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:42 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:42.636 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.635+0000 7f4559cea640 1 -- 172.21.15.189:0/137672036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 msgr2=0x7f4554107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:42.636 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.635+0000 7f4559cea640 1 --2- 172.21.15.189:0/137672036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f4554107a40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f454c009c20 tx=0x7f454c02aa90 comp rx=0 tx=0).stop 2024-09-12T12:04:42.636 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.635+0000 7f4559cea640 1 -- 172.21.15.189:0/137672036 shutdown_connections 2024-09-12T12:04:42.636 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.635+0000 7f4559cea640 1 --2- 172.21.15.189:0/137672036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f4554107a40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:42.636 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.635+0000 7f4559cea640 1 -- 172.21.15.189:0/137672036 >> 172.21.15.189:0/137672036 conn(0x7f45540fd2b0 msgr2=0x7f45540ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:42.636 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.635+0000 7f4559cea640 1 -- 172.21.15.189:0/137672036 shutdown_connections 2024-09-12T12:04:42.637 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.636+0000 7f4559cea640 1 -- 172.21.15.189:0/137672036 wait complete. 2024-09-12T12:04:42.637 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.636+0000 7f4559cea640 1 Processor -- start 2024-09-12T12:04:42.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.636+0000 7f4559cea640 1 -- start start 2024-09-12T12:04:42.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.636+0000 7f4559cea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f45541970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:42.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.636+0000 7f4559cea640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f455410bc70 con 0x7f4554107660 2024-09-12T12:04:42.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f45537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f45541970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:42.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f45537fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f45541970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.189:39830/0 (socket says 172.21.15.189:39830) 2024-09-12T12:04:42.638 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f45537fe640 1 -- 172.21.15.189:0/1305274006 learned_addr learned my addr 172.21.15.189:0/1305274006 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:04:42.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f45537fe640 1 -- 172.21.15.189:0/1305274006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45541975e0 con 0x7f4554107660 2024-09-12T12:04:42.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f45537fe640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f45541970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f454c005e30 tx=0x7f454c02fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:42.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f4550ff9640 1 -- 172.21.15.189:0/1305274006 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f454c047070 con 0x7f4554107660 2024-09-12T12:04:42.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f4550ff9640 1 -- 172.21.15.189:0/1305274006 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c003df0 con 0x7f4554107660 2024-09-12T12:04:42.639 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.637+0000 7f4550ff9640 1 -- 172.21.15.189:0/1305274006 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f454c033050 con 0x7f4554107660 2024-09-12T12:04:42.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.638+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45541978d0 con 0x7f4554107660 2024-09-12T12:04:42.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.638+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455419b970 con 0x7f4554107660 2024-09-12T12:04:42.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.639+0000 7f4550ff9640 1 -- 172.21.15.189:0/1305274006 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f454c04a070 con 0x7f4554107660 2024-09-12T12:04:42.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.639+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4554102d10 con 0x7f4554107660 2024-09-12T12:04:42.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.640+0000 7f4550ff9640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45400390c0 0x7f454003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:42.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.640+0000 7f4550ff9640 1 -- 172.21.15.189:0/1305274006 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f454c03e030 con 0x7f4554107660 2024-09-12T12:04:42.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.640+0000 7f4552ffd640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45400390c0 0x7f454003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:42.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.640+0000 7f4552ffd640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45400390c0 0x7f454003b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f453c0098c0 tx=0x7f453c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:42.645 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.645+0000 7f4550ff9640 1 -- 172.21.15.189:0/1305274006 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f454c04a360 con 0x7f4554107660 2024-09-12T12:04:42.822 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.822+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) -- 0x7f4554198400 con 0x7f45400390c0 2024-09-12T12:04:42.827 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.827+0000 7f4550ff9640 1 -- 172.21.15.189:0/1305274006 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4554198400 con 0x7f45400390c0 2024-09-12T12:04:42.827 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-09-12T12:04:42.829 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45400390c0 msgr2=0x7f454003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:42.829 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45400390c0 0x7f454003b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f453c0098c0 tx=0x7f453c006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:42.829 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 msgr2=0x7f45541970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:42.829 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f45541970a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f454c005e30 tx=0x7f454c02fc10 comp rx=0 tx=0).stop 2024-09-12T12:04:42.829 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 shutdown_connections 2024-09-12T12:04:42.829 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45400390c0 0x7f454003b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:42.830 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 --2- 172.21.15.189:0/1305274006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4554107660 0x7f45541970a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:42.830 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 >> 172.21.15.189:0/1305274006 conn(0x7f45540fd2b0 msgr2=0x7f45540feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:42.830 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 shutdown_connections 2024-09-12T12:04:42.830 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:42.829+0000 7f4559cea640 1 -- 172.21.15.189:0/1305274006 wait complete. 2024-09-12T12:04:43.391 DEBUG:teuthology.orchestra.run.smithi171:mon.b> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T12:04:43.394 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T12:04:43.396 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T12:04:43.397 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T12:04:43.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:43.924 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:43.925 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:43.925 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:43.925 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:04:43.925 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:44.392 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:04:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:44 smithi137 ceph-mon[30950]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:04:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:44 smithi137 ceph-mon[30950]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:44 smithi137 ceph-mon[30950]: Saving service mon spec with placement smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c;count:3 2024-09-12T12:04:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:44 smithi137 ceph-mon[30950]: Deploying daemon mon.c on smithi189 2024-09-12T12:04:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:44 smithi137 ceph-mon[30950]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:45 smithi137 ceph-mon[30950]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.678+0000 7fc150cdd640 1 -- 172.21.15.189:0/645199644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 msgr2=0x7fc14c107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.678+0000 7fc150cdd640 1 --2- 172.21.15.189:0/645199644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c107a70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009880 tx=0x7fc13c02a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.678+0000 7fc150cdd640 1 -- 172.21.15.189:0/645199644 shutdown_connections 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.678+0000 7fc150cdd640 1 --2- 172.21.15.189:0/645199644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c107a70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.678+0000 7fc150cdd640 1 -- 172.21.15.189:0/645199644 >> 172.21.15.189:0/645199644 conn(0x7fc14c076250 msgr2=0x7fc14c076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.678+0000 7fc150cdd640 1 -- 172.21.15.189:0/645199644 shutdown_connections 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.678+0000 7fc150cdd640 1 -- 172.21.15.189:0/645199644 wait complete. 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.679+0000 7fc150cdd640 1 Processor -- start 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.679+0000 7fc150cdd640 1 -- start start 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.679+0000 7fc150cdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:45.689 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.679+0000 7fc150cdd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c10bce0 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.680+0000 7fc14a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.680+0000 7fc14a59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.189:39838/0 (socket says 172.21.15.189:39838) 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.680+0000 7fc14a59c640 1 -- 172.21.15.189:0/2280351791 learned_addr learned my addr 172.21.15.189:0/2280351791 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.680+0000 7fc14a59c640 1 -- 172.21.15.189:0/2280351791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c197650 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.680+0000 7fc14a59c640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009230 tx=0x7fc13c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.680+0000 7fc1437fe640 1 -- 172.21.15.189:0/2280351791 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc13c03f070 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.681+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14c1978e0 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.681+0000 7fc1437fe640 1 -- 172.21.15.189:0/2280351791 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc13c03bc80 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.681+0000 7fc1437fe640 1 -- 172.21.15.189:0/2280351791 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc13c035030 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.681+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14c19b9e0 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.681+0000 7fc1437fe640 1 -- 172.21.15.189:0/2280351791 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc13c042070 con 0x7fc14c107690 2024-09-12T12:04:45.690 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.681+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc14c102d30 con 0x7fc14c107690 2024-09-12T12:04:45.691 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.682+0000 7fc1437fe640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc138039060 0x7fc13803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:45.691 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.682+0000 7fc1437fe640 1 -- 172.21.15.189:0/2280351791 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc13c06cdc0 con 0x7fc14c107690 2024-09-12T12:04:45.691 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.682+0000 7fc143fff640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc138039060 0x7fc13803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:45.691 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.682+0000 7fc143fff640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc138039060 0x7fc13803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1340098c0 tx=0x7fc134006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:45.691 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.687+0000 7fc1437fe640 1 -- 172.21.15.189:0/2280351791 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc13c031e20 con 0x7fc14c107690 2024-09-12T12:04:45.934 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.934+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc14c198470 con 0x7fc14c107690 2024-09-12T12:04:45.935 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.935+0000 7fc1437fe640 1 -- 172.21.15.189:0/2280351791 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fc13c031570 con 0x7fc14c107690 2024-09-12T12:04:45.935 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:04:45.936 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","modified":"2024-09-12T12:03:19.533021Z","created":"2024-09-12T12:03:19.533021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T12:04:45.936 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-09-12T12:04:45.937 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc138039060 msgr2=0x7fc13803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:45.937 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc138039060 0x7fc13803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1340098c0 tx=0x7fc134006db0 comp rx=0 tx=0).stop 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 msgr2=0x7fc14c197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c197110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc13c009230 tx=0x7fc13c003d40 comp rx=0 tx=0).stop 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 shutdown_connections 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc138039060 0x7fc13803b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 --2- 172.21.15.189:0/2280351791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc14c107690 0x7fc14c197110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 >> 172.21.15.189:0/2280351791 conn(0x7fc14c076250 msgr2=0x7fc14c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 shutdown_connections 2024-09-12T12:04:45.938 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:45.937+0000 7fc150cdd640 1 -- 172.21.15.189:0/2280351791 wait complete. 2024-09-12T12:04:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:46 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/2280351791' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout: e1 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout: btime 2024-09-12T12:03:22:696421+0000 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout: legacy client fscid: -1 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout: No filesystems configured 2024-09-12T12:04:47.115 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T12:04:47.562 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T12:04:47.563 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T12:04:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T12:04:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T12:04:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T12:04:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T12:04:47.587 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/242114454' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/242114454' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:04:47.588 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.589 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: Saving service mon spec with placement smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c;count:3 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: Deploying daemon mon.c on smithi189 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/2280351791' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:04:47.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 ceph-mon[35475]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T12:04:47.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 bash[35456]: 8560af655c48a78dc1190e710553c6f2149fa156267ed4299d02772a73178a8d 2024-09-12T12:04:47.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:47 smithi189 systemd[1]: Started Ceph mon.c for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:04:47.812 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:04:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: mon.a calling monitor election 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: mon.c calling monitor election 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: monmap epoch 2 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: last_changed 2024-09-12T12:04:47.120905+0000 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: election_strategy: 1 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: fsmap 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: mgrmap e13: x(active, since 37s) 2024-09-12T12:04:52.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: overall HEALTH_OK 2024-09-12T12:04:52.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:52.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:52.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:52.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:04:52.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: mon.a calling monitor election 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: mon.c calling monitor election 2024-09-12T12:04:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: monmap epoch 2 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: last_changed 2024-09-12T12:04:47.120905+0000 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: min_mon_release 19 (squid) 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: election_strategy: 1 2024-09-12T12:04:52.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: fsmap 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: mgrmap e13: x(active, since 37s) 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: overall HEALTH_OK 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:04:52.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:52.643 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.642+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/1392829432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 msgr2=0x7f1f640059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.642+0000 7f1f7fb0c640 1 --2- 172.21.15.189:0/1392829432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 0x7f1f640059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f781016a0 tx=0x7f1f7402c490 comp rx=0 tx=0).stop 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.642+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/1392829432 shutdown_connections 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.642+0000 7f1f7fb0c640 1 --2- 172.21.15.189:0/1392829432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 0x7f1f640059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.642+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/1392829432 >> 172.21.15.189:0/1392829432 conn(0x7f1f780fc220 msgr2=0x7f1f780fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.642+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/1392829432 shutdown_connections 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.642+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/1392829432 wait complete. 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.643+0000 7f1f7fb0c640 1 Processor -- start 2024-09-12T12:04:52.644 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.643+0000 7f1f7fb0c640 1 -- start start 2024-09-12T12:04:52.645 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7fb0c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 0x7f1f78198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7fb0c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 0x7f1f78198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7fb0c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1f78108670 con 0x7f1f64003530 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7fb0c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f781084f0 con 0x7f1f78198640 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d0a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 0x7f1f78198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d8a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 0x7f1f78198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d0a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 0x7f1f78198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:44420/0 (socket says 172.21.15.189:44420) 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d0a7640 1 -- 172.21.15.189:0/4189866256 learned_addr learned my addr 172.21.15.189:0/4189866256 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d0a7640 1 -- 172.21.15.189:0/4189866256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 msgr2=0x7f1f78198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d0a7640 1 -- 172.21.15.189:0/4189866256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 msgr2=0x7f1f78198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d0a7640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 0x7f1f78198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d0a7640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 0x7f1f78198aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:04:52.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d8a8640 1 -- 172.21.15.189:0/4189866256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 msgr2=0x7f1f78198aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d8a8640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 0x7f1f78198aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.644+0000 7f1f7d8a8640 1 -- 172.21.15.189:0/4189866256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f781a1170 con 0x7f1f64003530 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.645+0000 7f1f7d8a8640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 0x7f1f78198100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005f30 tx=0x7f1f740337c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.645+0000 7f1f6effd640 1 -- 172.21.15.189:0/4189866256 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1f74038040 con 0x7f1f64003530 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.645+0000 7f1f6effd640 1 -- 172.21.15.189:0/4189866256 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f74032e00 con 0x7f1f64003530 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.645+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f781a1340 con 0x7f1f64003530 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.645+0000 7f1f6effd640 1 -- 172.21.15.189:0/4189866256 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1f7403cdd0 con 0x7f1f64003530 2024-09-12T12:04:52.647 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.645+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f781a1840 con 0x7f1f64003530 2024-09-12T12:04:52.648 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.646+0000 7f1f6effd640 1 -- 172.21.15.189:0/4189866256 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f1f74043020 con 0x7f1f64003530 2024-09-12T12:04:52.648 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.646+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f78101ca0 con 0x7f1f64003530 2024-09-12T12:04:52.648 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.646+0000 7f1f6effd640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1f5c038e00 0x7f1f5c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:52.648 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.646+0000 7f1f6effd640 1 -- 172.21.15.189:0/4189866256 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1f7406e8c0 con 0x7f1f64003530 2024-09-12T12:04:52.648 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.647+0000 7f1f7d0a7640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1f5c038e00 0x7f1f5c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:52.648 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.647+0000 7f1f7d0a7640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1f5c038e00 0x7f1f5c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009100 tx=0x7f1f68008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:52.652 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.652+0000 7f1f6effd640 1 -- 172.21.15.189:0/4189866256 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f7404a020 con 0x7f1f64003530 2024-09-12T12:04:52.885 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.885+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1f7802d7c0 con 0x7f1f64003530 2024-09-12T12:04:52.886 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.885+0000 7f1f6effd640 1 -- 172.21.15.189:0/4189866256 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f1f740422c0 con 0x7f1f64003530 2024-09-12T12:04:52.886 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:04:52.886 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","modified":"2024-09-12T12:04:47.120905Z","created":"2024-09-12T12:03:19.533021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T12:04:52.886 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1f5c038e00 msgr2=0x7f1f5c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1f5c038e00 0x7f1f5c03b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009100 tx=0x7f1f68008040 comp rx=0 tx=0).stop 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 msgr2=0x7f1f78198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 0x7f1f78198100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f74005f30 tx=0x7f1f740337c0 comp rx=0 tx=0).stop 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 shutdown_connections 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1f5c038e00 0x7f1f5c03b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f78198640 0x7f1f78198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:52.888 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 --2- 172.21.15.189:0/4189866256 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f64003530 0x7f1f78198100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:52.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 >> 172.21.15.189:0/4189866256 conn(0x7f1f780fc220 msgr2=0x7f1f780fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:52.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 shutdown_connections 2024-09-12T12:04:52.889 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:52.888+0000 7f1f7fb0c640 1 -- 172.21.15.189:0/4189866256 wait complete. 2024-09-12T12:04:53.470 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:53 smithi189 ceph-mon[35475]: Deploying daemon mon.b on smithi171 2024-09-12T12:04:53.470 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:53 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/4189866256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:04:53.470 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:53 smithi137 ceph-mon[30950]: Deploying daemon mon.b on smithi171 2024-09-12T12:04:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:53 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/4189866256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:04:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:54.498 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T12:04:54.498 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T12:04:54.522 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:04:54 smithi189 ceph-mon[35475]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:04:54 smithi137 ceph-mon[30950]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:54.750 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:04:55.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.186+0000 7f2f46aff640 1 -- 172.21.15.189:0/2096273730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40107660 msgr2=0x7f2f40107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:55.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.186+0000 7f2f46aff640 1 --2- 172.21.15.189:0/2096273730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40107660 0x7f2f40107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009880 tx=0x7f2f3002a5d0 comp rx=0 tx=0).stop 2024-09-12T12:04:55.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.186+0000 7f2f46aff640 1 -- 172.21.15.189:0/2096273730 shutdown_connections 2024-09-12T12:04:55.227 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.186+0000 7f2f46aff640 1 --2- 172.21.15.189:0/2096273730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40107660 0x7f2f40107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.186+0000 7f2f46aff640 1 -- 172.21.15.189:0/2096273730 >> 172.21.15.189:0/2096273730 conn(0x7f2f400fd320 msgr2=0x7f2f400ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.186+0000 7f2f46aff640 1 -- 172.21.15.189:0/2096273730 shutdown_connections 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.186+0000 7f2f46aff640 1 -- 172.21.15.189:0/2096273730 wait complete. 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.187+0000 7f2f46aff640 1 Processor -- start 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.187+0000 7f2f46aff640 1 -- start start 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.187+0000 7f2f46aff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 0x7f2f401973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.187+0000 7f2f46aff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40197900 0x7f2f4019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.187+0000 7f2f46aff640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4010bd20 con 0x7f2f40197900 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.187+0000 7f2f46aff640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4010bba0 con 0x7f2f40107660 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.187+0000 7f2f4489b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 0x7f2f401973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f4489b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 0x7f2f401973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:44436/0 (socket says 172.21.15.189:44436) 2024-09-12T12:04:55.228 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f4489b640 1 -- 172.21.15.189:0/3933232824 learned_addr learned my addr 172.21.15.189:0/3933232824 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3ffff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40197900 0x7f2f4019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f4489b640 1 -- 172.21.15.189:0/3933232824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 msgr2=0x7f2f401973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f4489b640 1 -- 172.21.15.189:0/3933232824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 msgr2=0x7f2f401973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f4489b640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 0x7f2f401973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f4489b640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 0x7f2f401973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3ffff640 1 -- 172.21.15.189:0/3933232824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 msgr2=0x7f2f401973c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3ffff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 0x7f2f401973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3ffff640 1 -- 172.21.15.189:0/3933232824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f40197f20 con 0x7f2f40197900 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3ffff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40197900 0x7f2f4019bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800b9b0 tx=0x7f2f2800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3dffb640 1 -- 172.21.15.189:0/3933232824 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2f2800c920 con 0x7f2f40197900 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3dffb640 1 -- 172.21.15.189:0/3933232824 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f28014440 con 0x7f2f40197900 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f3dffb640 1 -- 172.21.15.189:0/3933232824 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2f28012500 con 0x7f2f40197900 2024-09-12T12:04:55.229 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4019c1d0 con 0x7f2f40197900 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.188+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4019c710 con 0x7f2f40197900 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.189+0000 7f2f3dffb640 1 -- 172.21.15.189:0/3933232824 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f2f2801c070 con 0x7f2f40197900 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.189+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f40102da0 con 0x7f2f40197900 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.189+0000 7f2f3dffb640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2f2c038e50 0x7f2f2c03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.190+0000 7f2f3dffb640 1 -- 172.21.15.189:0/3933232824 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2f2804b9b0 con 0x7f2f40197900 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.190+0000 7f2f4489b640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2f2c038e50 0x7f2f2c03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.190+0000 7f2f4489b640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2f2c038e50 0x7f2f2c03b310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2f300096c0 tx=0x7f2f30005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:04:55.230 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.195+0000 7f2f3dffb640 1 -- 172.21.15.189:0/3933232824 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f28010ca0 con 0x7f2f40197900 2024-09-12T12:04:55.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.425+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2f4019ca20 con 0x7f2f40197900 2024-09-12T12:04:55.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.425+0000 7f2f3dffb640 1 -- 172.21.15.189:0/3933232824 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f2f28014960 con 0x7f2f40197900 2024-09-12T12:04:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:04:55.426 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","modified":"2024-09-12T12:04:47.120905Z","created":"2024-09-12T12:03:19.533021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T12:04:55.426 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-09-12T12:04:55.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.427+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2f2c038e50 msgr2=0x7f2f2c03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:55.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.427+0000 7f2f46aff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2f2c038e50 0x7f2f2c03b310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2f300096c0 tx=0x7f2f30005a40 comp rx=0 tx=0).stop 2024-09-12T12:04:55.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.427+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40197900 msgr2=0x7f2f4019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:04:55.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.427+0000 7f2f46aff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40197900 0x7f2f4019bc90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800b9b0 tx=0x7f2f2800be70 comp rx=0 tx=0).stop 2024-09-12T12:04:55.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.427+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 shutdown_connections 2024-09-12T12:04:55.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.427+0000 7f2f46aff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2f2c038e50 0x7f2f2c03b310 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:55.428 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.428+0000 7f2f46aff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f40197900 0x7f2f4019bc90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:55.429 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.428+0000 7f2f46aff640 1 --2- 172.21.15.189:0/3933232824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f40107660 0x7f2f401973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:04:55.429 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.428+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 >> 172.21.15.189:0/3933232824 conn(0x7f2f400fd320 msgr2=0x7f2f400fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:04:55.429 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.428+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 shutdown_connections 2024-09-12T12:04:55.429 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:04:55.428+0000 7f2f46aff640 1 -- 172.21.15.189:0/3933232824 wait complete. 2024-09-12T12:04:55.573 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:55 smithi171 systemd[1]: Starting Ceph mon.b for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:04:55.885 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:55 smithi171 podman[35079]: 2024-09-12 12:04:55.659129309 +0000 UTC m=+0.015920044 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:04:55.885 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:55 smithi171 podman[35079]: 2024-09-12 12:04:55.777384985 +0000 UTC m=+0.134175700 container create 03ba81652799a972d2b01af54dc071512d5243d3e5c0a82130b1972b13170f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:04:56.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:55 smithi171 podman[35079]: 2024-09-12 12:04:55.988039551 +0000 UTC m=+0.344830266 container init 03ba81652799a972d2b01af54dc071512d5243d3e5c0a82130b1972b13170f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T12:04:56.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:55 smithi171 podman[35079]: 2024-09-12 12:04:55.991260806 +0000 UTC m=+0.348051524 container start 03ba81652799a972d2b01af54dc071512d5243d3e5c0a82130b1972b13170f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pidfile_write: ignore empty --pid-file 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: load: jerasure load: lrc 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: RocksDB version: 7.9.2 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Git sha 0 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: DB SUMMARY 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: DB Session ID: H1XTC50CIEB5NUZXX916 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: CURRENT file: CURRENT 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T12:04:56.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.error_if_exists: 0 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.create_if_missing: 0 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.paranoid_checks: 1 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.env: 0x5572a6c78940 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.info_log: 0x5572a87f76e0 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.statistics: (nil) 2024-09-12T12:04:56.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.use_fsync: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_log_file_size: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.allow_fallocate: 1 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.use_direct_reads: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.db_log_dir: 2024-09-12T12:04:56.203 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.wal_dir: 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.write_buffer_manager: 0x5572a8806dc0 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T12:04:56.204 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.unordered_write: 0 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.row_cache: None 2024-09-12T12:04:56.205 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.wal_filter: None 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.two_write_queues: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.wal_compression: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.atomic_flush: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.log_readahead_size: 0 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T12:04:56.207 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_background_jobs: 2 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_background_compactions: -1 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_subcompactions: 1 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T12:04:56.208 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_open_files: -1 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_background_flushes: -1 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Compression algorithms supported: 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kZSTD supported: 0 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kXpressCompression supported: 0 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kBZip2Compression supported: 0 2024-09-12T12:04:56.209 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kLZ4Compression supported: 1 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kZlibCompression supported: 1 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: kSnappyCompression supported: 1 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.merge_operator: 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_filter: None 2024-09-12T12:04:56.210 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5572a87f7320) 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: index_type: 0 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: data_block_index_type: 0 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: index_shortening: 1 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: checksum: 4 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: no_block_cache: 0 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache: 0x5572a87eb1f0 2024-09-12T12:04:56.211 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache_options: 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: capacity : 536870912 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: num_shard_bits : 4 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: strict_capacity_limit : 0 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_cache_compressed: (nil) 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: persistent_cache: (nil) 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_size: 4096 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_size_deviation: 10 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_restart_interval: 16 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: index_block_restart_interval: 1 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: metadata_block_size: 4096 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: partition_filters: 0 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: use_delta_encoding: 1 2024-09-12T12:04:56.212 INFO:journalctl@ceph.mon.b.smithi171.stdout: filter_policy: bloomfilter 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: whole_key_filtering: 1 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: verify_compression: 0 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: format_version: 5 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: enable_index_compression: 1 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: block_align: 0 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: max_auto_readahead_size: 262144 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: prepopulate_block_cache: 0 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: initial_auto_readahead_size: 8192 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression: NoCompression 2024-09-12T12:04:56.213 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.num_levels: 7 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T12:04:56.214 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T12:04:56.215 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T12:04:56.216 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T12:04:56.217 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.inplace_update_support: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.bloom_locality: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.max_successive_merges: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T12:04:56.218 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.ttl: 2592000 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.enable_blob_files: false 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.min_blob_size: 0 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T12:04:56.219 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bfe09e89-1d95-4c15-a42f-8ea5362203ae 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726142696019965, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726142696020381, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726142696, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bfe09e89-1d95-4c15-a42f-8ea5362203ae", "db_session_id": "H1XTC50CIEB5NUZXX916", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726142696020475, "job": 1, "event": "recovery_finished"} 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T12:04:56.220 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5572a8818e00 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: DB pointer 0x5572a8914000 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** DB Stats ** 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-09-12T12:04:56.221 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** Compaction Stats [default] ** 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** Compaction Stats [default] ** 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T12:04:56.222 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: Block cache BinnedLRUCache@0x5572a87eb1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T12:04:56.223 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: starting mon.b rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(???) e0 preinit fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout: e1 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout: btime 2024-09-12T12:03:22:696421+0000 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout: legacy client fscid: -1 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout: 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout: No filesystems configured 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T12:04:56.224 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/242114454' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/242114454' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T12:04:56.225 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:04:56.226 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:04:56.227 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: Saving service mon spec with placement smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c;count:3 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: Deploying daemon mon.c on smithi189 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/2280351791' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.a calling monitor election 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.c calling monitor election 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.228 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: monmap epoch 2 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: last_changed 2024-09-12T12:04:47.120905+0000 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: min_mon_release 19 (squid) 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: election_strategy: 1 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: fsmap 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mgrmap e13: x(active, since 37s) 2024-09-12T12:04:56.229 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: overall HEALTH_OK 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: Deploying daemon mon.b on smithi171 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/4189866256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 ceph-mon[35098]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 bash[35079]: 03ba81652799a972d2b01af54dc071512d5243d3e5c0a82130b1972b13170f5b 2024-09-12T12:04:56.230 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:04:56 smithi171 systemd[1]: Started Ceph mon.b for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:04:56.944 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T12:04:56.945 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T12:04:57.199 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:05:01.311 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.310+0000 7f0208699640 1 -- 172.21.15.189:0/2587713384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ec003530 msgr2=0x7f01ec0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:01.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.311+0000 7f0208699640 1 --2- 172.21.15.189:0/2587713384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ec003530 0x7f01ec0059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0200102af0 tx=0x7f01fc02c3f0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.311+0000 7f0208699640 1 -- 172.21.15.189:0/2587713384 shutdown_connections 2024-09-12T12:05:01.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.311+0000 7f0208699640 1 --2- 172.21.15.189:0/2587713384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01ec003530 0x7f01ec0059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.311+0000 7f0208699640 1 -- 172.21.15.189:0/2587713384 >> 172.21.15.189:0/2587713384 conn(0x7f02000fd320 msgr2=0x7f02000ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:01.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.311+0000 7f0208699640 1 -- 172.21.15.189:0/2587713384 shutdown_connections 2024-09-12T12:05:01.312 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.311+0000 7f0208699640 1 -- 172.21.15.189:0/2587713384 wait complete. 2024-09-12T12:05:01.315 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 Processor -- start 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 -- start start 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 0x7f02001972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 0x7f020019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f020019c100 0x7f020019c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0200109720 con 0x7f020019c100 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f02001095a0 con 0x7f0200197830 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.312+0000 7f0208699640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02001098a0 con 0x7f01ec003530 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206435640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 0x7f02001972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206435640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 0x7f02001972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:58296/0 (socket says 172.21.15.189:58296) 2024-09-12T12:05:01.316 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206435640 1 -- 172.21.15.189:0/2573664401 learned_addr learned my addr 172.21.15.189:0/2573664401 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0205c34640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 0x7f020019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206435640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 msgr2=0x7f02001972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206435640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 msgr2=0x7f02001972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206435640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 0x7f02001972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206c36640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f020019c100 0x7f020019c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206435640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 0x7f02001972f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0205c34640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 msgr2=0x7f020019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0205c34640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 msgr2=0x7f020019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0205c34640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 0x7f020019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:05:01.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0205c34640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 0x7f020019bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206c36640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 msgr2=0x7f02001972f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206c36640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 0x7f02001972f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206c36640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 msgr2=0x7f020019bbc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206c36640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 0x7f020019bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206c36640 1 -- 172.21.15.189:0/2573664401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02001a3e70 con 0x7f020019c100 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f0206c36640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f020019c100 0x7f020019c5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f01f800bd50 tx=0x7f01f800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.313+0000 7f01f77fe640 1 -- 172.21.15.189:0/2573664401 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8019070 con 0x7f020019c100 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.314+0000 7f01f77fe640 1 -- 172.21.15.189:0/2573664401 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f8014920 con 0x7f020019c100 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.314+0000 7f01f77fe640 1 -- 172.21.15.189:0/2573664401 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8012990 con 0x7f020019c100 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.314+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02001a40a0 con 0x7f020019c100 2024-09-12T12:05:01.318 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.314+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02001a4560 con 0x7f020019c100 2024-09-12T12:05:01.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.314+0000 7f01f77fe640 1 -- 172.21.15.189:0/2573664401 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f01f801c070 con 0x7f020019c100 2024-09-12T12:05:01.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.315+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020010a850 con 0x7f020019c100 2024-09-12T12:05:01.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.315+0000 7f01f77fe640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f01e40394f0 0x7f01e403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:01.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.315+0000 7f01f77fe640 1 -- 172.21.15.189:0/2573664401 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f01f804be30 con 0x7f020019c100 2024-09-12T12:05:01.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.315+0000 7f0206435640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f01e40394f0 0x7f01e403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:01.319 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.315+0000 7f0206435640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f01e40394f0 0x7f01e403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01fc014a90 tx=0x7f01fc005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:01.320 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.320+0000 7f01f77fe640 1 -- 172.21.15.189:0/2573664401 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01f8013ba0 con 0x7f020019c100 2024-09-12T12:05:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: mon.c calling monitor election 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: mon.a calling monitor election 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: monmap epoch 3 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: last_changed 2024-09-12T12:04:56.044702+0000 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: election_strategy: 1 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: fsmap 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: mgrmap e13: x(active, since 46s) 2024-09-12T12:05:01.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: overall HEALTH_OK 2024-09-12T12:05:01.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.565 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.564+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f02001a4990 con 0x7f020019c100 2024-09-12T12:05:01.565 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.565+0000 7f01f77fe640 1 -- 172.21.15.189:0/2573664401 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f01f800e090 con 0x7f020019c100 2024-09-12T12:05:01.565 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:05:01.566 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","modified":"2024-09-12T12:04:56.044702Z","created":"2024-09-12T12:03:19.533021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: mon.c calling monitor election 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: mon.a calling monitor election 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:01.566 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: monmap epoch 3 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: last_changed 2024-09-12T12:04:56.044702+0000 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: min_mon_release 19 (squid) 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: election_strategy: 1 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:05:01.567 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: fsmap 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: mgrmap e13: x(active, since 46s) 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: overall HEALTH_OK 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f01e40394f0 msgr2=0x7f01e403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f01e40394f0 0x7f01e403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01fc014a90 tx=0x7f01fc005f60 comp rx=0 tx=0).stop 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f020019c100 msgr2=0x7f020019c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f020019c100 0x7f020019c5b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f01f800bd50 tx=0x7f01f800be40 comp rx=0 tx=0).stop 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 shutdown_connections 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f01e40394f0 0x7f01e403b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f020019c100 0x7f020019c5b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0200197830 0x7f020019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 --2- 172.21.15.189:0/2573664401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01ec003530 0x7f02001972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:01.569 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.567+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 >> 172.21.15.189:0/2573664401 conn(0x7f02000fd320 msgr2=0x7f02000ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:01.570 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.568+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 shutdown_connections 2024-09-12T12:05:01.570 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:01.568+0000 7f0208699640 1 -- 172.21.15.189:0/2573664401 wait complete. 2024-09-12T12:05:02.060 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T12:05:02.060 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T12:05:02.316 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:05:02.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:02 smithi189 ceph-mon[35475]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:02.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:02.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:02 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/2573664401' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:05:02.337 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:02.437 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:02 smithi137 ceph-mon[30950]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:02.437 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:02.437 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:02 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/2573664401' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:05:02.437 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 -- 172.21.15.137:0/1647734963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c107690 msgr2=0x7f377c107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 --2- 172.21.15.137:0/1647734963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c107690 0x7f377c107a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f376c009880 tx=0x7f376c02a5d0 comp rx=0 tx=0).stop 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 -- 172.21.15.137:0/1647734963 shutdown_connections 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 --2- 172.21.15.137:0/1647734963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c107690 0x7f377c107a70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 -- 172.21.15.137:0/1647734963 >> 172.21.15.137:0/1647734963 conn(0x7f377c076250 msgr2=0x7f377c076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 -- 172.21.15.137:0/1647734963 shutdown_connections 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 -- 172.21.15.137:0/1647734963 wait complete. 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.914+0000 7f37822dd640 1 Processor -- start 2024-09-12T12:05:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f37822dd640 1 -- start start 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f37822dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f377c107690 0x7f377c1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f37822dd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 0x7f377c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f37822dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c19c1c0 0x7f377c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f37822dd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f377c10bc90 con 0x7f377c19c1c0 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f37822dd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f377c10bb10 con 0x7f377c1978f0 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f37822dd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f377c10be10 con 0x7f377c107690 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f378087a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c19c1c0 0x7f377c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f378087a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c19c1c0 0x7f377c19c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34196/0 (socket says 172.21.15.137:34196) 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f378087a640 1 -- 172.21.15.137:0/1064254681 learned_addr learned my addr 172.21.15.137:0/1064254681 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f377b7fe640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 0x7f377c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f377bfff640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f377c107690 0x7f377c1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f377b7fe640 1 -- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 msgr2=0x7f377c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f377b7fe640 1 -- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 msgr2=0x7f377c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f377b7fe640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 0x7f377c19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.915+0000 7f377b7fe640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 0x7f377c19bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f378087a640 1 -- 172.21.15.137:0/1064254681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f377c107690 msgr2=0x7f377c1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f378087a640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f377c107690 0x7f377c1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f378087a640 1 -- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 msgr2=0x7f377c19bc80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f378087a640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 0x7f377c19bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f378087a640 1 -- 172.21.15.137:0/1064254681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c1a3f30 con 0x7f377c19c1c0 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f377bfff640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f377c107690 0x7f377c1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f378087a640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c19c1c0 0x7f377c19c670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f377400ba50 tx=0x7f377400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f37797fa640 1 -- 172.21.15.137:0/1064254681 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3774019070 con 0x7f377c19c1c0 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f37797fa640 1 -- 172.21.15.137:0/1064254681 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377400cbe0 con 0x7f377c19c1c0 2024-09-12T12:05:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f37797fa640 1 -- 172.21.15.137:0/1064254681 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37740076a0 con 0x7f377c19c1c0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c1a4160 con 0x7f377c19c1c0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.916+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1a46f0 con 0x7f377c19c1c0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.917+0000 7f37797fa640 1 -- 172.21.15.137:0/1064254681 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f377401c070 con 0x7f377c19c1c0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.917+0000 7f37797fa640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37640394f0 0x7f376403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.917+0000 7f37797fa640 1 -- 172.21.15.137:0/1064254681 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f377404ca80 con 0x7f377c19c1c0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.917+0000 7f377bfff640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37640394f0 0x7f376403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.918+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c107af0 con 0x7f377c19c1c0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.918+0000 7f377bfff640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37640394f0 0x7f376403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f376c009230 tx=0x7f376c005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:02.923+0000 7f37797fa640 1 -- 172.21.15.137:0/1064254681 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f377400e0a0 con 0x7f377c19c1c0 2024-09-12T12:05:03.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.213+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f377c1a49d0 con 0x7f377c19c1c0 2024-09-12T12:05:03.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.214+0000 7f37797fa640 1 -- 172.21.15.137:0/1064254681 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f37740112a0 con 0x7f377c19c1c0 2024-09-12T12:05:03.215 INFO:teuthology.orchestra.run.smithi137.stdout:# minimal ceph.conf for dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:03.215 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-09-12T12:05:03.215 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:03.215 INFO:teuthology.orchestra.run.smithi137.stdout: mon_host = [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37640394f0 msgr2=0x7f376403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37640394f0 0x7f376403b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f376c009230 tx=0x7f376c005a40 comp rx=0 tx=0).stop 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c19c1c0 msgr2=0x7f377c19c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c19c1c0 0x7f377c19c670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f377400ba50 tx=0x7f377400bf10 comp rx=0 tx=0).stop 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 shutdown_connections 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37640394f0 0x7f376403b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f377c19c1c0 0x7f377c19c670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f377c1978f0 0x7f377c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 --2- 172.21.15.137:0/1064254681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f377c107690 0x7f377c1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 >> 172.21.15.137:0/1064254681 conn(0x7f377c076250 msgr2=0x7f377c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 shutdown_connections 2024-09-12T12:05:03.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:03.216+0000 7f37822dd640 1 -- 172.21.15.137:0/1064254681 wait complete. 2024-09-12T12:05:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:05:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.c calling monitor election 2024-09-12T12:05:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.a calling monitor election 2024-09-12T12:05:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:05:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: monmap epoch 3 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: last_changed 2024-09-12T12:04:56.044702+0000 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: min_mon_release 19 (squid) 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: election_strategy: 1 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:05:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: fsmap 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mgrmap e13: x(active, since 46s) 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: overall HEALTH_OK 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/2573664401' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T12:05:03.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:03.835 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: mon.b calling monitor election 2024-09-12T12:05:03.835 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-12T12:05:03.835 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-12T12:05:03.835 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:05:03.835 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: mon.b calling monitor election 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: mon.c calling monitor election 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: mon.a calling monitor election 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: monmap epoch 3 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: last_changed 2024-09-12T12:04:56.044702+0000 2024-09-12T12:05:03.836 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: min_mon_release 19 (squid) 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: election_strategy: 1 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: fsmap 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: mgrmap e13: x(active, since 48s) 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: overall HEALTH_OK 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.837 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1064254681' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:03.838 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:03.863 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T12:05:03.863 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:05:03.863 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T12:05:03.890 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:05:03.890 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:05:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.b calling monitor election 2024-09-12T12:05:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-12T12:05:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-12T12:05:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:05:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.b calling monitor election 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.c calling monitor election 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.a calling monitor election 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: monmap epoch 3 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: last_changed 2024-09-12T12:04:56.044702+0000 2024-09-12T12:05:03.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: min_mon_release 19 (squid) 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: election_strategy: 1 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: fsmap 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: mgrmap e13: x(active, since 48s) 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: overall HEALTH_OK 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1064254681' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:03.953 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:03.957 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:05:03.957 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T12:05:03.991 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:05:03.991 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:05:04.061 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:05:04.061 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T12:05:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: mon.b calling monitor election 2024-09-12T12:05:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-12T12:05:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-12T12:05:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:05:04.088 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:05:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: mon.b calling monitor election 2024-09-12T12:05:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: mon.c calling monitor election 2024-09-12T12:05:04.089 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: mon.a calling monitor election 2024-09-12T12:05:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T12:05:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: monmap epoch 3 2024-09-12T12:05:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:05:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: last_changed 2024-09-12T12:04:56.044702+0000 2024-09-12T12:05:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: created 2024-09-12T12:03:19.533021+0000 2024-09-12T12:05:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: min_mon_release 19 (squid) 2024-09-12T12:05:04.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: election_strategy: 1 2024-09-12T12:05:04.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: 0: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.a 2024-09-12T12:05:04.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: 1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.c 2024-09-12T12:05:04.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: 2: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-12T12:05:04.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: fsmap 2024-09-12T12:05:04.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T12:05:04.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: mgrmap e13: x(active, since 48s) 2024-09-12T12:05:04.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: overall HEALTH_OK 2024-09-12T12:05:04.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1064254681' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:04.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:04.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:04.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:04.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:04.097 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:05:04.097 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:05:04.165 INFO:tasks.cephadm:Adding mgr.x on smithi137 2024-09-12T12:05:04.165 INFO:tasks.cephadm:Adding mgr.y on smithi171 2024-09-12T12:05:04.165 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-09-12T12:05:04.165 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi137=x;smithi171=y;smithi189=z' 2024-09-12T12:05:04.444 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:05:04.748 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:04 smithi189 ceph-mon[35475]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T12:05:04.748 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:04 smithi189 ceph-mon[35475]: Reconfiguring daemon mon.a on smithi137 2024-09-12T12:05:04.748 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:04.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:04 smithi137 ceph-mon[30950]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T12:05:04.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:04 smithi137 ceph-mon[30950]: Reconfiguring daemon mon.a on smithi137 2024-09-12T12:05:04.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.830+0000 7f67efe56640 1 -- 172.21.15.189:0/3818990548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81005e0 msgr2=0x7f67e81009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.830+0000 7f67efe56640 1 --2- 172.21.15.189:0/3818990548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81005e0 0x7f67e81009e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009c20 tx=0x7f67d802aa90 comp rx=0 tx=0).stop 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.830+0000 7f67efe56640 1 -- 172.21.15.189:0/3818990548 shutdown_connections 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.830+0000 7f67efe56640 1 --2- 172.21.15.189:0/3818990548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81005e0 0x7f67e81009e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.830+0000 7f67efe56640 1 -- 172.21.15.189:0/3818990548 >> 172.21.15.189:0/3818990548 conn(0x7f67e80fbdb0 msgr2=0x7f67e80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.830+0000 7f67efe56640 1 -- 172.21.15.189:0/3818990548 shutdown_connections 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.830+0000 7f67efe56640 1 -- 172.21.15.189:0/3818990548 wait complete. 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 Processor -- start 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 -- start start 2024-09-12T12:05:04.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67e81005e0 0x7f67e8197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8197540 0x7f67e819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81979c0 0x7f67e819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f67e81066d0 con 0x7f67e81979c0 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8106550 con 0x7f67e81005e0 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67efe56640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f67e8106850 con 0x7f67e8197540 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8197540 0x7f67e819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8197540 0x7f67e819c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:58298/0 (socket says 172.21.15.189:58298) 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 -- 172.21.15.189:0/4230791614 learned_addr learned my addr 172.21.15.189:0/4230791614 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ee3f3640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81979c0 0x7f67e819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:04.835 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67edbf2640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67e81005e0 0x7f67e8197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 -- 172.21.15.189:0/4230791614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67e81005e0 msgr2=0x7f67e8197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67e81005e0 0x7f67e8197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 -- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81979c0 msgr2=0x7f67e819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81979c0 0x7f67e819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ed3f1640 1 -- 172.21.15.189:0/4230791614 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e819f400 con 0x7f67e8197540 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67ee3f3640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81979c0 0x7f67e819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.831+0000 7f67edbf2640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67e81005e0 0x7f67e8197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.832+0000 7f67ed3f1640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8197540 0x7f67e819c5b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b520 tx=0x7f67dc00b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.832+0000 7f67e6ffd640 1 -- 172.21.15.189:0/4230791614 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc004230 con 0x7f67e8197540 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.832+0000 7f67e6ffd640 1 -- 172.21.15.189:0/4230791614 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc0043d0 con 0x7f67e8197540 2024-09-12T12:05:04.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.832+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e819f6f0 con 0x7f67e8197540 2024-09-12T12:05:04.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.832+0000 7f67e6ffd640 1 -- 172.21.15.189:0/4230791614 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc003d60 con 0x7f67e8197540 2024-09-12T12:05:04.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.832+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e819fcb0 con 0x7f67e8197540 2024-09-12T12:05:04.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.833+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e8190820 con 0x7f67e8197540 2024-09-12T12:05:04.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.833+0000 7f67e6ffd640 1 -- 172.21.15.189:0/4230791614 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f67dc00e070 con 0x7f67e8197540 2024-09-12T12:05:04.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.833+0000 7f67e6ffd640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67d00394f0 0x7f67d003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:04.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.833+0000 7f67e6ffd640 1 -- 172.21.15.189:0/4230791614 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f67dc04ba60 con 0x7f67e8197540 2024-09-12T12:05:04.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.834+0000 7f67edbf2640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67d00394f0 0x7f67d003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:04.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.834+0000 7f67edbf2640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67d00394f0 0x7f67d003b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006010 tx=0x7f67d8005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:04.838 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:04.838+0000 7f67e6ffd640 1 -- 172.21.15.189:0/4230791614 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67dc014020 con 0x7f67e8197540 2024-09-12T12:05:04.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:04 smithi171 ceph-mon[35098]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T12:05:04.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:04 smithi171 ceph-mon[35098]: Reconfiguring daemon mon.a on smithi137 2024-09-12T12:05:04.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:05:05.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.015+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi171=y;smithi189=z", "target": ["mon-mgr", ""]}) -- 0x7f67e80626c0 con 0x7f67d00394f0 2024-09-12T12:05:05.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.021+0000 7f67e6ffd640 1 -- 172.21.15.189:0/4230791614 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f67e80626c0 con 0x7f67d00394f0 2024-09-12T12:05:05.022 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-09-12T12:05:05.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67d00394f0 msgr2=0x7f67d003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:05.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67d00394f0 0x7f67d003b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006010 tx=0x7f67d8005c50 comp rx=0 tx=0).stop 2024-09-12T12:05:05.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8197540 msgr2=0x7f67e819c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:05.024 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8197540 0x7f67e819c5b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00b520 tx=0x7f67dc00b9e0 comp rx=0 tx=0).stop 2024-09-12T12:05:05.025 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 shutdown_connections 2024-09-12T12:05:05.025 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67d00394f0 0x7f67d003b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:05.025 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67e81979c0 0x7f67e819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:05.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8197540 0x7f67e819c5b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:05.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 --2- 172.21.15.189:0/4230791614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67e81005e0 0x7f67e8197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:05.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.023+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 >> 172.21.15.189:0/4230791614 conn(0x7f67e80fbdb0 msgr2=0x7f67e806a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:05.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.024+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 shutdown_connections 2024-09-12T12:05:05.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:05:05.024+0000 7f67efe56640 1 -- 172.21.15.189:0/4230791614 wait complete. 2024-09-12T12:05:05.580 DEBUG:teuthology.orchestra.run.smithi171:mgr.y> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T12:05:05.582 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T12:05:05.584 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T12:05:05.585 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:05:05.585 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T12:05:05.602 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T12:05:05.602 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-12 12:04:23.986444321 +0000 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-12 11:59:38.843606104 +0000 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-12 11:59:38.843606104 +0000 2024-09-12T12:05:05.658 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-12 11:59:38.843606104 +0000 2024-09-12T12:05:05.659 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T12:05:05.724 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-12T12:05:05.724 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-12T12:05:05.724 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000217076 s, 2.4 MB/s 2024-09-12T12:05:05.725 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T12:05:05.781 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-09-12T12:05:05.836 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T12:05:05.836 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:05.836 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T12:05:05.836 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:05.837 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:05.837 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-12 12:04:24.014443810 +0000 2024-09-12T12:05:05.837 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-12 11:59:39.191599839 +0000 2024-09-12T12:05:05.837 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-12 11:59:39.191599839 +0000 2024-09-12T12:05:05.837 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-12 11:59:39.191599839 +0000 2024-09-12T12:05:05.837 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T12:05:05.900 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-12T12:05:05.900 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-12T12:05:05.901 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000235893 s, 2.2 MB/s 2024-09-12T12:05:05.902 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T12:05:05.959 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-09-12T12:05:06.014 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-12 12:04:24.040443334 +0000 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-12 11:59:39.515594007 +0000 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-12 11:59:39.515594007 +0000 2024-09-12T12:05:06.015 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-12 11:59:39.515594007 +0000 2024-09-12T12:05:06.015 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T12:05:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-12T12:05:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-12T12:05:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000273125 s, 1.9 MB/s 2024-09-12T12:05:06.081 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T12:05:06.137 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-09-12T12:05:06.192 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-12 12:04:24.071442768 +0000 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-12 11:59:39.847588030 +0000 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-12 11:59:39.847588030 +0000 2024-09-12T12:05:06.193 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-12 11:59:39.847588030 +0000 2024-09-12T12:05:06.193 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T12:05:06.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi171=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: Saving service mgr spec with placement smithi137=x;smithi171=y;smithi189=z;count:3 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:06.255 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:06 smithi137 ceph-mon[30950]: Reconfiguring daemon mgr.x on smithi137 2024-09-12T12:05:06.256 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-09-12T12:05:06.256 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-09-12T12:05:06.257 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000217218 s, 2.4 MB/s 2024-09-12T12:05:06.258 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T12:05:06.315 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-12T12:05:06.315 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T12:05:06.333 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T12:05:06.333 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi171=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: Saving service mgr spec with placement smithi137=x;smithi171=y;smithi189=z;count:3 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:06 smithi189 ceph-mon[35475]: Reconfiguring daemon mgr.x on smithi137 2024-09-12T12:05:06.389 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-12 12:04:55.460174998 +0000 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-12 12:00:00.481978460 +0000 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-12 12:00:00.481978460 +0000 2024-09-12T12:05:06.390 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-12 12:00:00.481978460 +0000 2024-09-12T12:05:06.390 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T12:05:06.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi171=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: Saving service mgr spec with placement smithi137=x;smithi171=y;smithi189=z;count:3 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:06 smithi171 ceph-mon[35098]: Reconfiguring daemon mgr.x on smithi137 2024-09-12T12:05:06.455 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-12T12:05:06.455 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-12T12:05:06.455 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.00024797 s, 2.1 MB/s 2024-09-12T12:05:06.456 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T12:05:06.513 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-12 12:04:55.460174998 +0000 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-12 12:00:00.803971619 +0000 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-12 12:00:00.803971619 +0000 2024-09-12T12:05:06.570 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-12 12:00:00.803971619 +0000 2024-09-12T12:05:06.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T12:05:06.633 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-12T12:05:06.634 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-12T12:05:06.634 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000221307 s, 2.3 MB/s 2024-09-12T12:05:06.635 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T12:05:06.691 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-12 12:04:55.460174998 +0000 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-12 12:00:01.142964416 +0000 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-12 12:00:01.142964416 +0000 2024-09-12T12:05:06.748 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-12 12:00:01.142964416 +0000 2024-09-12T12:05:06.749 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T12:05:06.813 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-12T12:05:06.813 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-12T12:05:06.813 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000198037 s, 2.6 MB/s 2024-09-12T12:05:06.815 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T12:05:06.871 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-09-12T12:05:06.927 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T12:05:06.927 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:06.927 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T12:05:06.927 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:06.928 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:06.928 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-12 12:04:55.461174977 +0000 2024-09-12T12:05:06.928 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-12 12:00:01.466957532 +0000 2024-09-12T12:05:06.928 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-12 12:00:01.466957532 +0000 2024-09-12T12:05:06.928 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-12 12:00:01.466957532 +0000 2024-09-12T12:05:06.928 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T12:05:06.992 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-12T12:05:06.992 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-12T12:05:06.992 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000191917 s, 2.7 MB/s 2024-09-12T12:05:06.993 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T12:05:07.051 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-12T12:05:07.051 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T12:05:07.071 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T12:05:07.072 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-12T12:05:07.131 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T12:05:07.131 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:07.131 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T12:05:07.131 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:07.132 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:07.132 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-12 12:04:46.577548195 +0000 2024-09-12T12:05:07.132 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-12 11:59:55.078530552 +0000 2024-09-12T12:05:07.132 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-12 11:59:55.078530552 +0000 2024-09-12T12:05:07.132 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-12 11:59:55.078530552 +0000 2024-09-12T12:05:07.132 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T12:05:07.201 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-12T12:05:07.201 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-12T12:05:07.201 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000292279 s, 1.8 MB/s 2024-09-12T12:05:07.203 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T12:05:07.263 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-12 12:04:46.577548195 +0000 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-12 11:59:55.407523668 +0000 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-12 11:59:55.407523668 +0000 2024-09-12T12:05:07.321 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-12 11:59:55.407523668 +0000 2024-09-12T12:05:07.322 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T12:05:07.387 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-12T12:05:07.388 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-12T12:05:07.388 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000245549 s, 2.1 MB/s 2024-09-12T12:05:07.389 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T12:05:07.447 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-12T12:05:07.504 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-12 12:04:46.578548174 +0000 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-12 11:59:55.761516261 +0000 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-12 11:59:55.761516261 +0000 2024-09-12T12:05:07.505 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-12 11:59:55.761516261 +0000 2024-09-12T12:05:07.505 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T12:05:07.570 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-12T12:05:07.570 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-12T12:05:07.571 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000303899 s, 1.7 MB/s 2024-09-12T12:05:07.572 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T12:05:07.630 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-12 12:04:46.578548174 +0000 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-12 11:59:56.095509272 +0000 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-12 11:59:56.095509272 +0000 2024-09-12T12:05:07.687 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-12 11:59:56.095509272 +0000 2024-09-12T12:05:07.687 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T12:05:07.752 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-12T12:05:07.752 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-12T12:05:07.752 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000203073 s, 2.5 MB/s 2024-09-12T12:05:07.753 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: Reconfiguring mon.b (monmap changed)... 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: Reconfiguring daemon mon.b on smithi171 2024-09-12T12:05:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:07 smithi137 ceph-mon[30950]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:07.810 INFO:tasks.cephadm:Deploying osd.0 on smithi137 with /dev/vg_nvme/lv_4... 2024-09-12T12:05:07.810 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T12:05:07.821 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:07.822 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:07.822 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: Reconfiguring mon.b (monmap changed)... 2024-09-12T12:05:07.822 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:07.822 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:07.822 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:07.822 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: Reconfiguring daemon mon.b on smithi171 2024-09-12T12:05:07.822 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:07 smithi171 ceph-mon[35098]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:07.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:07.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:07.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: Reconfiguring mon.b (monmap changed)... 2024-09-12T12:05:07.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:07.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:07.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:07.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: Reconfiguring daemon mon.b on smithi171 2024-09-12T12:05:07.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:07 smithi189 ceph-mon[35475]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:08.065 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:05:09.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:08 smithi171 ceph-mon[35098]: Reconfiguring mon.c (monmap changed)... 2024-09-12T12:05:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:08 smithi171 ceph-mon[35098]: Reconfiguring daemon mon.c on smithi189 2024-09-12T12:05:09.221 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:09.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:09.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:08 smithi137 ceph-mon[30950]: Reconfiguring mon.c (monmap changed)... 2024-09-12T12:05:09.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:09.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:09.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:09.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:08 smithi137 ceph-mon[30950]: Reconfiguring daemon mon.c on smithi189 2024-09-12T12:05:09.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:09.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:09.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:08 smithi189 ceph-mon[35475]: Reconfiguring mon.c (monmap changed)... 2024-09-12T12:05:09.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T12:05:09.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T12:05:09.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:09.255 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:08 smithi189 ceph-mon[35475]: Reconfiguring daemon mon.c on smithi189 2024-09-12T12:05:10.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:10.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:10 smithi137 ceph-mon[30950]: Deploying daemon mgr.z on smithi189 2024-09-12T12:05:10.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:10.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:10.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:10.283 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.284 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:10.284 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T12:05:10.284 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:10.284 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:10.284 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:10.284 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:10 smithi189 ceph-mon[35475]: Deploying daemon mgr.z on smithi189 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T12:05:10.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:10.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:10.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:10.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:10 smithi171 ceph-mon[35098]: Deploying daemon mgr.z on smithi189 2024-09-12T12:05:10.573 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:05:10.600 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-09-12T12:05:10.840 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:05:10.841 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:10 smithi189 systemd[1]: Starting Ceph mgr.z for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:05:11.198 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:10 smithi189 podman[37269]: 2024-09-12 12:05:10.848636049 +0000 UTC m=+0.015751497 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:05:11.198 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:10 smithi189 podman[37269]: 2024-09-12 12:05:10.950364095 +0000 UTC m=+0.117479567 container create 39d267a0e552249fa20646fedc55473d07c6184e16797404e50afa091d276e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:05:11.198 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 podman[37269]: 2024-09-12 12:05:11.169743298 +0000 UTC m=+0.336858736 container init 39d267a0e552249fa20646fedc55473d07c6184e16797404e50afa091d276e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T12:05:11.198 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 podman[37269]: 2024-09-12 12:05:11.172903509 +0000 UTC m=+0.340018935 container start 39d267a0e552249fa20646fedc55473d07c6184e16797404e50afa091d276e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T12:05:11.198 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 ceph-mgr[37288]: -- 172.21.15.189:0/3888798771 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56268b2f70e0 con 0x56268b2d5000 2024-09-12T12:05:11.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 -- 172.21.15.137:0/2513907371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d81037c0 msgr2=0x7f61d810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 --2- 172.21.15.137:0/2513907371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d81037c0 0x7f61d810a050 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009ca0 tx=0x7f61d002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 -- 172.21.15.137:0/2513907371 shutdown_connections 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 --2- 172.21.15.137:0/2513907371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d81037c0 0x7f61d810a050 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 --2- 172.21.15.137:0/2513907371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d8102e00 0x7f61d8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 --2- 172.21.15.137:0/2513907371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d8101c00 0x7f61d8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 -- 172.21.15.137:0/2513907371 >> 172.21.15.137:0/2513907371 conn(0x7f61d80fd390 msgr2=0x7f61d80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 -- 172.21.15.137:0/2513907371 shutdown_connections 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.315+0000 7f61df752640 1 -- 172.21.15.137:0/2513907371 wait complete. 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 Processor -- start 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 -- start start 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d8101c00 0x7f61d8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d8102e00 0x7f61d80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d81037c0 0x7f61d8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61d810d860 con 0x7f61d81037c0 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f61d810d6e0 con 0x7f61d8102e00 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61df752640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61d810d9e0 con 0x7f61d8101c00 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d81037c0 0x7f61d8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d81037c0 0x7f61d8074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43762/0 (socket says 172.21.15.137:43762) 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 -- 172.21.15.137:0/4269372195 learned_addr learned my addr 172.21.15.137:0/4269372195 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61dd4ee640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d8101c00 0x7f61d8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61dcced640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d8102e00 0x7f61d80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 -- 172.21.15.137:0/4269372195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d8101c00 msgr2=0x7f61d8075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d8101c00 0x7f61d8075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:11.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 -- 172.21.15.137:0/4269372195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d8102e00 msgr2=0x7f61d80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d8102e00 0x7f61d80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61ddcef640 1 -- 172.21.15.137:0/4269372195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d80753e0 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.316+0000 7f61dd4ee640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d8101c00 0x7f61d8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.317+0000 7f61ddcef640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d81037c0 0x7f61d8074b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61d0031ae0 tx=0x7f61d0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.317+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0003c30 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.317+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d81a1f80 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.317+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d0003dd0 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.317+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d002fcb0 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.317+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d81a2480 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.318+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f61d00305c0 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.318+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d8108fb0 con 0x7f61d81037c0 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.318+0000 7f61ca7fc640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c4039290 0x7f61c403b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:11.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.318+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f61d006d5e0 con 0x7f61d81037c0 2024-09-12T12:05:11.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.318+0000 7f61dd4ee640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c4039290 0x7f61c403b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:11.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.319+0000 7f61dd4ee640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c4039290 0x7f61c403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005fd0 tx=0x7f61d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:11.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.323+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61d0030a00 con 0x7f61d81037c0 2024-09-12T12:05:11.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:11.502+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f61d8062630 con 0x7f61c4039290 2024-09-12T12:05:11.503 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 bash[37269]: 39d267a0e552249fa20646fedc55473d07c6184e16797404e50afa091d276e50 2024-09-12T12:05:11.503 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 systemd[1]: Started Ceph mgr.z for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:05:11.503 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:11.294+0000 7f3bc91b4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:05:11.503 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:11.355+0000 7f3bc91b4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:05:11.504 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:11.503+0000 7f3bc91b4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:05:11.840 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:11.569+0000 7f3bc91b4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:05:11.840 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:11.717+0000 7f3bc91b4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:11.857+0000 7f3bc91b4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:12.173+0000 7f3bc91b4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:12.174 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T12:05:12.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:12.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:12.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:05:12.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:05:12.175 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:05:12.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:05:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:05:12.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:12.590 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:12.235+0000 7f3bc91b4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:05:12.591 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:12.420+0000 7f3bc91b4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:05:13.148 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:13 smithi137 ceph-mon[30950]: Deploying daemon mgr.y on smithi171 2024-09-12T12:05:13.149 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:13 smithi137 ceph-mon[30950]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:13 smithi171 ceph-mon[35098]: Deploying daemon mgr.y on smithi171 2024-09-12T12:05:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:13 smithi171 ceph-mon[35098]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:13.200 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 systemd[1]: Starting Ceph mgr.y for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:05:13.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:13 smithi189 ceph-mon[35475]: Deploying daemon mgr.y on smithi171 2024-09-12T12:05:13.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:13 smithi189 ceph-mon[35475]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:13.341 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:13.299+0000 7f3bc91b4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:05:13.559 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 podman[36447]: 2024-09-12 12:05:13.295895781 +0000 UTC m=+0.099400406 container create ac545c0fa2692ea33d0492c5614f047bbab56e3ebca99661f21a655f859d58c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T12:05:13.559 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 podman[36447]: 2024-09-12 12:05:13.21239775 +0000 UTC m=+0.015902376 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:05:13.559 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 podman[36447]: 2024-09-12 12:05:13.531462 +0000 UTC m=+0.334966626 container init ac545c0fa2692ea33d0492c5614f047bbab56e3ebca99661f21a655f859d58c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T12:05:13.559 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 podman[36447]: 2024-09-12 12:05:13.534665181 +0000 UTC m=+0.338169815 container start ac545c0fa2692ea33d0492c5614f047bbab56e3ebca99661f21a655f859d58c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:05:13.559 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 ceph-mgr[36466]: -- 172.21.15.171:0/1460999732 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b0f37790e0 con 0x55b0f3757400 2024-09-12T12:05:13.840 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:13.366+0000 7f3bc91b4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:05:13.873 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 bash[36447]: ac545c0fa2692ea33d0492c5614f047bbab56e3ebca99661f21a655f859d58c1 2024-09-12T12:05:13.873 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 systemd[1]: Started Ceph mgr.y for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:05:13.873 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:13.656+0000 7f891d94c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:05:13.873 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:13.715+0000 7f891d94c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:05:13.873 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:13.873+0000 7f891d94c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:05:14.200 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:13.943+0000 7f891d94c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:05:14.200 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:14.104+0000 7f891d94c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:05:14.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:14.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:14.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3477359149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d33c8c-abf3-4322-aeb0-28f4141abb52"}]: dispatch 2024-09-12T12:05:14.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3477359149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d33c8c-abf3-4322-aeb0-28f4141abb52"}]': finished 2024-09-12T12:05:14.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T12:05:14.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:14.340 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.130+0000 7f3bc91b4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.255+0000 7f3bc91b4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3477359149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d33c8c-abf3-4322-aeb0-28f4141abb52"}]: dispatch 2024-09-12T12:05:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3477359149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d33c8c-abf3-4322-aeb0-28f4141abb52"}]': finished 2024-09-12T12:05:14.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T12:05:14.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3477359149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45d33c8c-abf3-4322-aeb0-28f4141abb52"}]: dispatch 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3477359149' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d33c8c-abf3-4322-aeb0-28f4141abb52"}]': finished 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T12:05:14.412 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:14.593 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:14.260+0000 7f891d94c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:05:14.690 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.399+0000 7f3bc91b4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:05:14.690 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.463+0000 7f3bc91b4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:05:14.690 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.619+0000 7f3bc91b4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:05:14.690 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.688+0000 7f3bc91b4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:05:14.851 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:14.594+0000 7f891d94c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:05:14.852 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:14.658+0000 7f891d94c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:05:14.852 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:14.851+0000 7f891d94c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:05:14.988 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.748+0000 7f3bc91b4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:05:14.988 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.916+0000 7f3bc91b4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:05:14.988 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:14.986+0000 7f3bc91b4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:05:15.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2807600230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2807600230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:05:15.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:15.262 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:15.054+0000 7f3bc91b4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:15.141+0000 7f3bc91b4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:15.201+0000 7f3bc91b4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:05:15.263 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:15.260+0000 7f3bc91b4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:05:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2807600230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:05:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:15.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:15.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:15.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:15.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:15.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:16.087 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:15.753+0000 7f891d94c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:05:16.087 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:15 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:15.821+0000 7f891d94c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:05:16.047+0000 7f3bc91b4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-mon[35475]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-mon[35475]: Standby manager daemon z started 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T12:05:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:16 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:05:16.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-mon[35098]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-mon[35098]: Standby manager daemon z started 2024-09-12T12:05:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T12:05:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:05:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T12:05:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:05:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:16 smithi137 ceph-mon[30950]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:16 smithi137 ceph-mon[30950]: Standby manager daemon z started 2024-09-12T12:05:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:16 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T12:05:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:16 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:05:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:16 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T12:05:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:16 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/2723571183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:05:16.810 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:16.554+0000 7f891d94c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:05:16.810 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:16.668+0000 7f891d94c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:05:16.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:16.991+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f61d004d440 con 0x7f61d81037c0 2024-09-12T12:05:17.089 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:16.809+0000 7f891d94c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:05:17.089 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:16.871+0000 7f891d94c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:05:17.089 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.021+0000 7f891d94c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:05:17.089 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.089+0000 7f891d94c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:05:17.371 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.149+0000 7f891d94c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:05:17.371 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.306+0000 7f891d94c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:05:17.371 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.371+0000 7f891d94c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:05:17.637 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.437+0000 7f891d94c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:05:17.637 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.519+0000 7f891d94c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:05:17.637 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.578+0000 7f891d94c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:05:17.637 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:17.636+0000 7f891d94c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:05:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:17 smithi137 ceph-mon[30950]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T12:05:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T12:05:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:17 smithi137 ceph-mon[30950]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:18.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:17 smithi189 ceph-mon[35475]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T12:05:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T12:05:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:17 smithi189 ceph-mon[35475]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:18.426 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-mon[35098]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T12:05:18.426 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T12:05:18.426 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:17 smithi171 ceph-mon[35098]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:18.699 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:05:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:05:18.425+0000 7f891d94c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:05:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:19.004+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f61d0043cf0 con 0x7f61d81037c0 2024-09-12T12:05:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:18 smithi137 ceph-mon[30950]: Standby manager daemon y started 2024-09-12T12:05:19.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:18 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T12:05:19.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:18 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:05:19.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:18 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T12:05:19.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:18 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:05:19.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:18 smithi189 ceph-mon[35475]: Standby manager daemon y started 2024-09-12T12:05:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:18 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T12:05:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:18 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:05:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:18 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T12:05:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:18 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:05:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:18 smithi171 ceph-mon[35098]: Standby manager daemon y started 2024-09-12T12:05:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:18 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T12:05:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:18 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:05:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:18 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T12:05:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:18 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/1610370572' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:05:20.124 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:20 smithi137 ceph-mon[30950]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T12:05:20.124 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T12:05:20.124 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:20 smithi137 ceph-mon[30950]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:20.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:20 smithi189 ceph-mon[35475]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T12:05:20.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T12:05:20.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:20 smithi189 ceph-mon[35475]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:20 smithi171 ceph-mon[35098]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T12:05:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T12:05:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:20 smithi171 ceph-mon[35098]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:22.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:22 smithi137 ceph-mon[30950]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:22.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T12:05:22.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:22 smithi171 ceph-mon[35098]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T12:05:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:22.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:22 smithi189 ceph-mon[35475]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T12:05:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:23.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:23 smithi137 ceph-mon[30950]: Deploying daemon osd.0 on smithi137 2024-09-12T12:05:23.370 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:23 smithi171 ceph-mon[35098]: Deploying daemon osd.0 on smithi137 2024-09-12T12:05:23.492 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:23 smithi189 ceph-mon[35475]: Deploying daemon osd.0 on smithi137 2024-09-12T12:05:24.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:24 smithi137 ceph-mon[30950]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:24.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:24 smithi171 ceph-mon[35098]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:24.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:24 smithi189 ceph-mon[35475]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:26.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:26 smithi171 ceph-mon[35098]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:26.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:26 smithi137 ceph-mon[30950]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:26.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:26 smithi189 ceph-mon[35475]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:28.383 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:28 smithi137 ceph-mon[30950]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:28 smithi171 ceph-mon[35098]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:28.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:28 smithi189 ceph-mon[35475]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:29.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:29.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:29.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:29.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:29.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:29.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:30.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:30 smithi137 ceph-mon[30950]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:30.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:30 smithi171 ceph-mon[35098]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:30.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:30 smithi189 ceph-mon[35475]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:32.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:31 smithi171 ceph-mon[35098]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:32.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:32.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:32.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:31 smithi137 ceph-mon[30950]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:32.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:32.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:32.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:31 smithi189 ceph-mon[35475]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:32.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:32.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:32.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:32.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.629+0000 7f61ca7fc640 1 -- 172.21.15.137:0/4269372195 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f61d8062630 con 0x7f61c4039290 2024-09-12T12:05:32.629 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 0 on host 'smithi137' 2024-09-12T12:05:32.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c4039290 msgr2=0x7f61c403b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c4039290 0x7f61c403b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005fd0 tx=0x7f61d400b410 comp rx=0 tx=0).stop 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d81037c0 msgr2=0x7f61d8074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d81037c0 0x7f61d8074b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61d0031ae0 tx=0x7f61d0031ee0 comp rx=0 tx=0).stop 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 shutdown_connections 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c4039290 0x7f61c403b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d81037c0 0x7f61d8074b20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d8102e00 0x7f61d80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 --2- 172.21.15.137:0/4269372195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d8101c00 0x7f61d8075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 >> 172.21.15.137:0/4269372195 conn(0x7f61d80fd390 msgr2=0x7f61d8106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 shutdown_connections 2024-09-12T12:05:32.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:05:32.632+0000 7f61df752640 1 -- 172.21.15.137:0/4269372195 wait complete. 2024-09-12T12:05:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:33.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:33.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:33.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:33.357 DEBUG:teuthology.orchestra.run.smithi137:osd.0> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T12:05:33.359 INFO:tasks.cephadm:Deploying osd.1 on smithi171 with /dev/vg_nvme/lv_4... 2024-09-12T12:05:33.359 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T12:05:33.599 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:05:34.044 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:33 smithi171 ceph-mon[35098]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:34.219 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:33 smithi189 ceph-mon[35475]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:34.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:33 smithi137 ceph-mon[30950]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:34.300 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:05:34 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0[42068]: 2024-09-12T12:05:34.011+0000 7f75c1114640 -1 osd.0 0 log_to_monitors true 2024-09-12T12:05:34.961 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:34 smithi171 ceph-mon[35098]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T12:05:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:34 smithi189 ceph-mon[35475]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T12:05:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:34 smithi137 ceph-mon[30950]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T12:05:36.002 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:35 smithi171 ceph-mon[35098]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T12:05:36.002 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:35 smithi171 ceph-mon[35098]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T12:05:36.002 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:35 smithi171 ceph-mon[35098]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:05:36.003 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:36.003 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:35 smithi171 ceph-mon[35098]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:36.005 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:05:36.032 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-09-12T12:05:36.282 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:05:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:35 smithi137 ceph-mon[30950]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T12:05:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:35 smithi137 ceph-mon[30950]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T12:05:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:35 smithi137 ceph-mon[30950]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:05:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:35 smithi137 ceph-mon[30950]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:36.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:35 smithi189 ceph-mon[35475]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T12:05:36.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:35 smithi189 ceph-mon[35475]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T12:05:36.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:35 smithi189 ceph-mon[35475]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:05:36.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:36.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:35 smithi189 ceph-mon[35475]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:36.666 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0[42068]: 2024-09-12T12:05:36.379+0000 7f75bd8ab640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T12:05:36.666 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0[42068]: 2024-09-12T12:05:36.382+0000 7f75b4eb7640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:05:36.742 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 -- 172.21.15.171:0/552663673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47641026a0 msgr2=0x7f4764104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 --2- 172.21.15.171:0/552663673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47641026a0 0x7f4764104a90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4754009900 tx=0x7f475402a410 comp rx=0 tx=0).stop 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 -- 172.21.15.171:0/552663673 shutdown_connections 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 --2- 172.21.15.171:0/552663673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4764107900 0x7f4764109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 --2- 172.21.15.171:0/552663673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4764104fd0 0x7f47641073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 --2- 172.21.15.171:0/552663673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47641026a0 0x7f4764104a90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 -- 172.21.15.171:0/552663673 >> 172.21.15.171:0/552663673 conn(0x7f47640fc2a0 msgr2=0x7f47640fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 -- 172.21.15.171:0/552663673 shutdown_connections 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 -- 172.21.15.171:0/552663673 wait complete. 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 Processor -- start 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.728+0000 7f476a91a640 1 -- start start 2024-09-12T12:05:36.743 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f476a91a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47641026a0 0x7f4764197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f476a91a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4764104fd0 0x7f4764198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f476a91a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4764107900 0x7f476419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f476a91a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f476410d920 con 0x7f47641026a0 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f476a91a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f476410d7a0 con 0x7f4764104fd0 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f476a91a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f476410daa0 con 0x7f4764107900 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4764104fd0 0x7f4764198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4764104fd0 0x7f4764198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:39294/0 (socket says 172.21.15.171:39294) 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 -- 172.21.15.171:0/1985625403 learned_addr learned my addr 172.21.15.171:0/1985625403 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f4763fff640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47641026a0 0x7f4764197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f4768eb7640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4764107900 0x7f476419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 -- 172.21.15.171:0/1985625403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4764107900 msgr2=0x7f476419f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:36.744 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4764107900 0x7f476419f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 -- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47641026a0 msgr2=0x7f4764197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47641026a0 0x7f4764197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 -- 172.21.15.171:0/1985625403 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476419fc00 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47637fe640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4764104fd0 0x7f4764198510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f475800d770 tx=0x7f475800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47617fa640 1 -- 172.21.15.171:0/1985625403 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758017070 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.729+0000 7f47617fa640 1 -- 172.21.15.171:0/1985625403 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4758009c30 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.730+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476419fef0 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.730+0000 7f47617fa640 1 -- 172.21.15.171:0/1985625403 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475800f620 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.730+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47641a0470 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.731+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47640fdd90 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.732+0000 7f47617fa640 1 -- 172.21.15.171:0/1985625403 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f475801a070 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.732+0000 7f47617fa640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f474c0a2ba0 0x7f474c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.732+0000 7f47617fa640 1 -- 172.21.15.171:0/1985625403 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f47580ccef0 con 0x7f4764104fd0 2024-09-12T12:05:36.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.732+0000 7f4763fff640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f474c0a2ba0 0x7f474c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:05:36.746 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.733+0000 7f4763fff640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f474c0a2ba0 0x7f474c0a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47540314e0 tx=0x7f4754000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:05:36.746 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.737+0000 7f47617fa640 1 -- 172.21.15.171:0/1985625403 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f475809b9e0 con 0x7f4764104fd0 2024-09-12T12:05:36.911 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:36.910+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4764062630 con 0x7f474c0a2ba0 2024-09-12T12:05:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:05:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T12:05:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi137 2024-09-12T12:05:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:05:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:37.313 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:05:37.313 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T12:05:37.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:37.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:37.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi137 2024-09-12T12:05:37.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.314 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:05:37.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-12T12:05:37.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:37.315 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:37.316 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.316 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' 2024-09-12T12:05:37.316 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:05:37.317 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:05:37.317 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:37.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:05:37.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T12:05:37.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:37.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:37.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi137 2024-09-12T12:05:37.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:05:37.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi137 to 17088M 2024-09-12T12:05:37.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:37.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:37.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:37.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089]' entity='osd.0' 2024-09-12T12:05:37.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:05:37.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:05:37.344 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] boot 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T12:05:38.257 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] boot 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T12:05:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:38.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:05:38.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:05:38.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:05:38.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:38.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T12:05:38.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: osd.0 [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] boot 2024-09-12T12:05:38.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T12:05:38.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:05:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:39 smithi171 ceph-mon[35098]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T12:05:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:39 smithi171 ceph-mon[35098]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:39 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/100507542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]: dispatch 2024-09-12T12:05:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:39 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]: dispatch 2024-09-12T12:05:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:39 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]': finished 2024-09-12T12:05:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:39 smithi171 ceph-mon[35098]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T12:05:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:39 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:05:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:39 smithi137 ceph-mon[30950]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T12:05:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:39 smithi137 ceph-mon[30950]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:39 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/100507542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]: dispatch 2024-09-12T12:05:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:39 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]: dispatch 2024-09-12T12:05:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:39 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]': finished 2024-09-12T12:05:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:39 smithi137 ceph-mon[30950]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T12:05:39.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:39 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:05:39.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:39 smithi189 ceph-mon[35475]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T12:05:39.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:39 smithi189 ceph-mon[35475]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:39.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:39 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/100507542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]: dispatch 2024-09-12T12:05:39.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:39 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]: dispatch 2024-09-12T12:05:39.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:39 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1a7f639-1cce-4a38-b5b2-9e6035281396"}]': finished 2024-09-12T12:05:39.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:39 smithi189 ceph-mon[35475]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T12:05:39.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:39 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:05:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:40 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/2509894493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:05:40.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:40 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/2509894493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:05:40.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:40 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/2509894493' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:05:41.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:41 smithi171 ceph-mon[35098]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:41 smithi137 ceph-mon[30950]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:41.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:41 smithi189 ceph-mon[35475]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:44.414 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:44 smithi171 ceph-mon[35098]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:44 smithi137 ceph-mon[30950]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:44.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:44 smithi189 ceph-mon[35475]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:46.322 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:46 smithi171 ceph-mon[35098]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:46 smithi137 ceph-mon[30950]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:46.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:46 smithi189 ceph-mon[35475]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:48 smithi171 ceph-mon[35098]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T12:05:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:48 smithi137 ceph-mon[30950]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T12:05:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:48 smithi189 ceph-mon[35475]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T12:05:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:49.186 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:49 smithi171 ceph-mon[35098]: Deploying daemon osd.1 on smithi171 2024-09-12T12:05:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:49 smithi137 ceph-mon[30950]: Deploying daemon osd.1 on smithi171 2024-09-12T12:05:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:49 smithi189 ceph-mon[35475]: Deploying daemon osd.1 on smithi171 2024-09-12T12:05:50.414 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:50 smithi171 ceph-mon[35098]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:50.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:50 smithi137 ceph-mon[30950]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:50.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:50 smithi189 ceph-mon[35475]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:52.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:52 smithi171 ceph-mon[35098]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:52 smithi137 ceph-mon[30950]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:52.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:52 smithi189 ceph-mon[35475]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:54.393 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:54 smithi189 ceph-mon[35475]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:54.393 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:54.393 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:54.393 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:54.437 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:54 smithi171 ceph-mon[35098]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:54.437 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:54.437 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:54.437 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:54 smithi137 ceph-mon[30950]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:56 smithi171 ceph-mon[35098]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:56.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:56.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:56.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:56 smithi137 ceph-mon[30950]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:56 smithi189 ceph-mon[35475]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:56.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:56.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:05:56.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:05:56.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:57.866 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.865+0000 7f47617fa640 1 -- 172.21.15.171:0/1985625403 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4764062630 con 0x7f474c0a2ba0 2024-09-12T12:05:57.866 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 1 on host 'smithi171' 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f474c0a2ba0 msgr2=0x7f474c0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f474c0a2ba0 0x7f474c0a5060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f47540314e0 tx=0x7f4754000bc0 comp rx=0 tx=0).stop 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4764104fd0 msgr2=0x7f4764198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4764104fd0 0x7f4764198510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f475800d770 tx=0x7f475800dc30 comp rx=0 tx=0).stop 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 shutdown_connections 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f474c0a2ba0 0x7f474c0a5060 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4764107900 0x7f476419f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4764104fd0 0x7f4764198510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.867+0000 7f476a91a640 1 --2- 172.21.15.171:0/1985625403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47641026a0 0x7f4764197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:05:57.869 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.868+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 >> 172.21.15.171:0/1985625403 conn(0x7f47640fc2a0 msgr2=0x7f4764102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:05:57.870 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.868+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 shutdown_connections 2024-09-12T12:05:57.870 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:05:57.868+0000 7f476a91a640 1 -- 172.21.15.171:0/1985625403 wait complete. 2024-09-12T12:05:58.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:58 smithi171 ceph-mon[35098]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:58.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:58 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:58.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:58 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:58.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:05:58 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:58.487 DEBUG:teuthology.orchestra.run.smithi171:osd.1> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T12:05:58.489 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-09-12T12:05:58.489 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T12:05:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:58 smithi189 ceph-mon[35475]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:58 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:58 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:58.518 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:05:58 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:58 smithi137 ceph-mon[30950]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:05:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:58 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:05:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:58 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:05:58 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:05:58.741 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:05:59.844 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:05:59 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1[39551]: 2024-09-12T12:05:59.488+0000 7feac5215640 -1 osd.1 0 log_to_monitors true 2024-09-12T12:06:00.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:00 smithi171 ceph-mon[35098]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:00 smithi171 ceph-mon[35098]: from='osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T12:06:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:00 smithi171 ceph-mon[35098]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T12:06:00.523 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:00 smithi189 ceph-mon[35475]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:00.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:00 smithi189 ceph-mon[35475]: from='osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T12:06:00.524 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:00 smithi189 ceph-mon[35475]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T12:06:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:00 smithi137 ceph-mon[30950]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:00 smithi137 ceph-mon[30950]: from='osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T12:06:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:00 smithi137 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T12:06:01.245 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:06:01.273 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-09-12T12:06:01.325 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:01 smithi171 ceph-mon[35098]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T12:06:01.325 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:01 smithi171 ceph-mon[35098]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T12:06:01.325 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:01.325 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:01 smithi171 ceph-mon[35098]: from='osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:06:01.325 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:01 smithi171 ceph-mon[35098]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:06:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:01 smithi189 ceph-mon[35475]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T12:06:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:01 smithi189 ceph-mon[35475]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T12:06:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:01 smithi189 ceph-mon[35475]: from='osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:06:01.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:01 smithi189 ceph-mon[35475]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:06:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:01 smithi137 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T12:06:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:01 smithi137 ceph-mon[30950]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T12:06:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:01 smithi137 ceph-mon[30950]: from='osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:06:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:01 smithi137 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:06:01.562 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:06:01.577 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1[39551]: 2024-09-12T12:06:01.571+0000 7feac1198640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T12:06:01.577 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1[39551]: 2024-09-12T12:06:01.577+0000 7feab8fb8640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:06:02.012 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.009+0000 7fa4f609e640 1 -- 172.21.15.189:0/3061920532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f01037c0 msgr2=0x7fa4f010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:02.012 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.009+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3061920532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f01037c0 0x7fa4f010a050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009ca0 tx=0x7fa4e802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:06:02.012 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 -- 172.21.15.189:0/3061920532 shutdown_connections 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3061920532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f01037c0 0x7fa4f010a050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3061920532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4f0102e00 0x7fa4f0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3061920532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4f0101c00 0x7fa4f0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 -- 172.21.15.189:0/3061920532 >> 172.21.15.189:0/3061920532 conn(0x7fa4f00fd390 msgr2=0x7fa4f00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 -- 172.21.15.189:0/3061920532 shutdown_connections 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 -- 172.21.15.189:0/3061920532 wait complete. 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.010+0000 7fa4f609e640 1 Processor -- start 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4f609e640 1 -- start start 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4f609e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f0101c00 0x7fa4f0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4f609e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4f0102e00 0x7fa4f00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:02.013 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4f609e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4f01037c0 0x7fa4f0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:02.014 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4f609e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f010d860 con 0x7fa4f01037c0 2024-09-12T12:06:02.014 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4f609e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f010d6e0 con 0x7fa4f0102e00 2024-09-12T12:06:02.014 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4f609e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f010d9e0 con 0x7fa4f0101c00 2024-09-12T12:06:02.014 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f0101c00 0x7fa4f0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:02.014 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f0101c00 0x7fa4f0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:55294/0 (socket says 172.21.15.189:55294) 2024-09-12T12:06:02.014 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 -- 172.21.15.189:0/3926051715 learned_addr learned my addr 172.21.15.189:0/3926051715 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:06:02.014 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4eeffd640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4f0102e00 0x7fa4f00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 -- 172.21.15.189:0/3926051715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4f0102e00 msgr2=0x7fa4f00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4f0102e00 0x7fa4f00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4effff640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4f01037c0 0x7fa4f0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 -- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4f01037c0 msgr2=0x7fa4f0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4f01037c0 0x7fa4f0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.011+0000 7fa4ef7fe640 1 -- 172.21.15.189:0/3926051715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f00753e0 con 0x7fa4f0101c00 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.012+0000 7fa4ef7fe640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f0101c00 0x7fa4f0075f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c970 tx=0x7fa4e000cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:06:02.015 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.012+0000 7fa4ecff9640 1 -- 172.21.15.189:0/3926051715 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0007b50 con 0x7fa4f0101c00 2024-09-12T12:06:02.016 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.012+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f01a1ec0 con 0x7fa4f0101c00 2024-09-12T12:06:02.016 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.012+0000 7fa4ecff9640 1 -- 172.21.15.189:0/3926051715 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0007cf0 con 0x7fa4f0101c00 2024-09-12T12:06:02.016 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.012+0000 7fa4ecff9640 1 -- 172.21.15.189:0/3926051715 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000f870 con 0x7fa4f0101c00 2024-09-12T12:06:02.016 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.012+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f01a22b0 con 0x7fa4f0101c00 2024-09-12T12:06:02.016 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.013+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f0108ed0 con 0x7fa4f0101c00 2024-09-12T12:06:02.017 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.015+0000 7fa4ecff9640 1 -- 172.21.15.189:0/3926051715 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fa4e0018480 con 0x7fa4f0101c00 2024-09-12T12:06:02.018 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.015+0000 7fa4ecff9640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4d80a2af0 0x7fa4d80a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:02.018 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.016+0000 7fa4ecff9640 1 -- 172.21.15.189:0/3926051715 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fa4e00cdd70 con 0x7fa4f0101c00 2024-09-12T12:06:02.018 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.016+0000 7fa4eeffd640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4d80a2af0 0x7fa4d80a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:02.018 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.016+0000 7fa4eeffd640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4d80a2af0 0x7fa4d80a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0075d10 tx=0x7fa4e400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:06:02.020 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.019+0000 7fa4ecff9640 1 -- 172.21.15.189:0/3926051715 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4e0026080 con 0x7fa4f0101c00 2024-09-12T12:06:02.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:02.195+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa4f0062630 con 0x7fa4d80a2af0 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:02.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:02.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:02 smithi189 ceph-mon[35475]: from='osd.1 ' entity='osd.1' 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:02.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:02.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:02.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:02 smithi171 ceph-mon[35098]: from='osd.1 ' entity='osd.1' 2024-09-12T12:06:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:06:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T12:06:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:02.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:02 smithi137 ceph-mon[30950]: from='osd.1 ' entity='osd.1' 2024-09-12T12:06:03.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:06:03.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:06:03.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi171 2024-09-12T12:06:03.259 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi171 to 17088M 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] boot 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T12:06:03.260 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi171 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi171 to 17088M 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] boot 2024-09-12T12:06:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T12:06:03.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:06:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:06:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi171 2024-09-12T12:06:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi171 to 17088M 2024-09-12T12:06:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:03.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:03.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:03.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: osd.1 [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] boot 2024-09-12T12:06:03.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T12:06:03.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:06:04.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:04 smithi171 ceph-mon[35098]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:04.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:04 smithi137 ceph-mon[30950]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:04.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:04 smithi189 ceph-mon[35475]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T12:06:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:05 smithi189 ceph-mon[35475]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T12:06:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:05 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/2564519529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]: dispatch 2024-09-12T12:06:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:05 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]: dispatch 2024-09-12T12:06:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:05 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]': finished 2024-09-12T12:06:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:05 smithi189 ceph-mon[35475]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T12:06:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:05 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:05 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/624084246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:05.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:05 smithi171 ceph-mon[35098]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T12:06:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:05 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/2564519529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]: dispatch 2024-09-12T12:06:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:05 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]: dispatch 2024-09-12T12:06:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:05 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]': finished 2024-09-12T12:06:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:05 smithi171 ceph-mon[35098]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T12:06:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:05 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:05 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/624084246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:05 smithi137 ceph-mon[30950]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T12:06:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:05 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/2564519529' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]: dispatch 2024-09-12T12:06:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:05 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]: dispatch 2024-09-12T12:06:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:05 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "845d927b-63b2-4eba-961c-222c975d8779"}]': finished 2024-09-12T12:06:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:05 smithi137 ceph-mon[30950]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T12:06:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:05 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:05 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/624084246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:06 smithi171 ceph-mon[35098]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:06 smithi137 ceph-mon[30950]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:06 smithi189 ceph-mon[35475]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:08.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:08 smithi171 ceph-mon[35098]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:08 smithi137 ceph-mon[30950]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:08 smithi189 ceph-mon[35475]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:10.247 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:10 smithi189 ceph-mon[35475]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:10.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:10 smithi171 ceph-mon[35098]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:10 smithi137 ceph-mon[30950]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:12 smithi171 ceph-mon[35098]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:12.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:12 smithi137 ceph-mon[30950]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:12.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:12 smithi189 ceph-mon[35475]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:13.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:13 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T12:06:13.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:13 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:13.468 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:13 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T12:06:13.469 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:13 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:13 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T12:06:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:13 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:14.222 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:14 smithi189 ceph-mon[35475]: Deploying daemon osd.2 on smithi189 2024-09-12T12:06:14.222 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:14 smithi189 ceph-mon[35475]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:14 smithi171 ceph-mon[35098]: Deploying daemon osd.2 on smithi189 2024-09-12T12:06:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:14 smithi171 ceph-mon[35098]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:14 smithi137 ceph-mon[30950]: Deploying daemon osd.2 on smithi189 2024-09-12T12:06:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:14 smithi137 ceph-mon[30950]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:16 smithi171 ceph-mon[35098]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:16.499 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:16 smithi189 ceph-mon[35475]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:16 smithi137 ceph-mon[30950]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:18.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:18 smithi171 ceph-mon[35098]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:18.544 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:18 smithi189 ceph-mon[35475]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:18 smithi137 ceph-mon[30950]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:20 smithi171 ceph-mon[35098]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:20 smithi137 ceph-mon[30950]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:20 smithi189 ceph-mon[35475]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:22 smithi189 ceph-mon[35475]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:22.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:22.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:22.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:22 smithi171 ceph-mon[35098]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:22 smithi137 ceph-mon[30950]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:23.737 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.737+0000 7fa4ecff9640 1 -- 172.21.15.189:0/3926051715 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4f0062630 con 0x7fa4d80a2af0 2024-09-12T12:06:23.737 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-09-12T12:06:23.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4d80a2af0 msgr2=0x7fa4d80a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4d80a2af0 0x7fa4d80a4fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0075d10 tx=0x7fa4e400a2d0 comp rx=0 tx=0).stop 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f0101c00 msgr2=0x7fa4f0075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f0101c00 0x7fa4f0075f90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c970 tx=0x7fa4e000cd70 comp rx=0 tx=0).stop 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 shutdown_connections 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4d80a2af0 0x7fa4d80a4fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4f01037c0 0x7fa4f0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4f0102e00 0x7fa4f00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 --2- 172.21.15.189:0/3926051715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4f0101c00 0x7fa4f0075f90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 >> 172.21.15.189:0/3926051715 conn(0x7fa4f00fd390 msgr2=0x7fa4f01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 shutdown_connections 2024-09-12T12:06:23.740 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:06:23.739+0000 7fa4f609e640 1 -- 172.21.15.189:0/3926051715 wait complete. 2024-09-12T12:06:24.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:24 smithi189 ceph-mon[35475]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:24 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:24 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:24.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:24 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:24.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:24 smithi171 ceph-mon[35098]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:24 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:24 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:24 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:24.453 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T12:06:24.496 INFO:tasks.cephadm:Deploying osd.3 on smithi137 with /dev/vg_nvme/lv_3... 2024-09-12T12:06:24.496 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T12:06:24.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:24 smithi137 ceph-mon[30950]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:24.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:24 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:24.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:24 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:24.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:24 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:24.755 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:06:25.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:25 smithi171 ceph-mon[35098]: from='osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T12:06:25.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:25 smithi171 ceph-mon[35098]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T12:06:25.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:25 smithi137 ceph-mon[30950]: from='osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T12:06:25.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:25 smithi137 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T12:06:25.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:25 smithi189 ceph-mon[35475]: from='osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T12:06:25.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:25 smithi189 ceph-mon[35475]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T12:06:25.591 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:25 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2[40435]: 2024-09-12T12:06:25.145+0000 7f73b4900640 -1 osd.2 0 log_to_monitors true 2024-09-12T12:06:26.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:26 smithi137 ceph-mon[30950]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:26.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:26 smithi137 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T12:06:26.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:26 smithi137 ceph-mon[30950]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T12:06:26.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:26 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:26.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:26 smithi137 ceph-mon[30950]: from='osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:06:26.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:26 smithi137 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:06:26.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-mon[35475]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:26.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-mon[35475]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T12:06:26.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-mon[35475]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T12:06:26.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:26.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-mon[35475]: from='osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:06:26.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-mon[35475]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:06:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:26 smithi171 ceph-mon[35098]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:26 smithi171 ceph-mon[35098]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T12:06:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:26 smithi171 ceph-mon[35098]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T12:06:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:26 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:26 smithi171 ceph-mon[35098]: from='osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:06:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:26 smithi171 ceph-mon[35098]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:06:27.090 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2[40435]: 2024-09-12T12:06:26.724+0000 7f73b0883640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T12:06:27.091 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2[40435]: 2024-09-12T12:06:26.728+0000 7f73a86a3640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:06:27.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:27 smithi137 ceph-mon[30950]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:06:27.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:27 smithi137 ceph-mon[30950]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T12:06:27.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:27.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:27.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:27 smithi137 ceph-mon[30950]: from='osd.2 ' entity='osd.2' 2024-09-12T12:06:27.359 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:06:27.386 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-09-12T12:06:27.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:27 smithi189 ceph-mon[35475]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:06:27.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:27 smithi189 ceph-mon[35475]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T12:06:27.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:27 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:27.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:27 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:27.448 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:27 smithi189 ceph-mon[35475]: from='osd.2 ' entity='osd.2' 2024-09-12T12:06:27.698 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:06:27.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:27 smithi171 ceph-mon[35098]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:06:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:27 smithi171 ceph-mon[35098]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T12:06:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:27 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:27 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:27 smithi171 ceph-mon[35098]: from='osd.2 ' entity='osd.2' 2024-09-12T12:06:28.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 -- 172.21.15.137:0/3605793081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 msgr2=0x7f3ba8103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:28.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3605793081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8103430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009930 tx=0x7f3b9002a420 comp rx=0 tx=0).stop 2024-09-12T12:06:28.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 -- 172.21.15.137:0/3605793081 shutdown_connections 2024-09-12T12:06:28.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3605793081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ba8103970 0x7f3ba810a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:28.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3605793081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8103430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3605793081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ba80fff30 0x7f3ba8100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 -- 172.21.15.137:0/3605793081 >> 172.21.15.137:0/3605793081 conn(0x7f3ba80fba80 msgr2=0x7f3ba80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 -- 172.21.15.137:0/3605793081 shutdown_connections 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 -- 172.21.15.137:0/3605793081 wait complete. 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 Processor -- start 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 -- start start 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ba80fff30 0x7f3ba8102910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ba8103970 0x7f3ba81014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.220+0000 7f3badb6c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba810d970 con 0x7f3ba8102fb0 2024-09-12T12:06:28.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3badb6c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba810d7f0 con 0x7f3ba8103970 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3badb6c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba810daf0 con 0x7f3ba80fff30 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35374/0 (socket says 172.21.15.137:35374) 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 -- 172.21.15.137:0/3062242966 learned_addr learned my addr 172.21.15.137:0/3062242966 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3ba77fe640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ba80fff30 0x7f3ba8102910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 -- 172.21.15.137:0/3062242966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ba80fff30 msgr2=0x7f3ba8102910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ba80fff30 0x7f3ba8102910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3ba7fff640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ba8103970 0x7f3ba81014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 -- 172.21.15.137:0/3062242966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ba8103970 msgr2=0x7f3ba81014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ba8103970 0x7f3ba81014a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 -- 172.21.15.137:0/3062242966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba8101d60 con 0x7f3ba8102fb0 2024-09-12T12:06:28.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3b9ffff640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8100f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b90031610 tx=0x7f3b900043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9003f030 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba81a21d0 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b90003ce0 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90035070 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.221+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba81a2700 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.223+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba8109340 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.224+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f3b90042030 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.224+0000 7f3ba57fa640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b940a2af0 0x7f3b940a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.225+0000 7f3ba77fe640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b940a2af0 0x7f3b940a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.225+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f3b900edd10 con 0x7f3ba8102fb0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.225+0000 7f3ba77fe640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b940a2af0 0x7f3b940a4fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8069cb0 tx=0x7f3b98009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:06:28.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.228+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b900bc2a0 con 0x7f3ba8102fb0 2024-09-12T12:06:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:06:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:06:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] boot 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T12:06:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:28.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:28.410+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3ba8062630 con 0x7f3b940a2af0 2024-09-12T12:06:28.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] boot 2024-09-12T12:06:28.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T12:06:28.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:06:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:06:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T12:06:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:28.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:28.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:28.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:28.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: osd.2 [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] boot 2024-09-12T12:06:28.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T12:06:28.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:06:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:29 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi189 2024-09-12T12:06:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:29 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-12T12:06:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:29.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:29 smithi189 ceph-mon[35475]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T12:06:29.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:29 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi189 2024-09-12T12:06:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:29 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-12T12:06:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:29 smithi171 ceph-mon[35098]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T12:06:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:29 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi189 2024-09-12T12:06:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:29 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi189 to 17088M 2024-09-12T12:06:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:29.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:29.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:29.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:29 smithi137 ceph-mon[30950]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T12:06:30.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:30 smithi137 ceph-mon[30950]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:30.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:30 smithi137 ceph-mon[30950]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:30.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:30 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:06:30.354 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:30 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T12:06:30.354 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:30 smithi137 ceph-mon[30950]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T12:06:30.354 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:30 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:06:30.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:30 smithi189 ceph-mon[35475]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:30.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:30 smithi189 ceph-mon[35475]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:30.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:30 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:06:30.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:30 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T12:06:30.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:30 smithi189 ceph-mon[35475]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T12:06:30.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:30 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:06:30.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:30 smithi171 ceph-mon[35098]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:30.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:30 smithi171 ceph-mon[35098]: pgmap v76: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:30.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:30 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:06:30.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:30 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T12:06:30.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:30 smithi171 ceph-mon[35098]: osdmap e20: 3 total, 3 up, 3 in 2024-09-12T12:06:30.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:30 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:06:31.361 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T12:06:31.362 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:31.362 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46478]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:31.362 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46478]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:31.747 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46505]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T12:06:31.747 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:31.747 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46505]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:31.748 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:06:31 smithi137 sudo[46505]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:31.860+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3b900b8210 con 0x7f3ba8102fb0 2024-09-12T12:06:31.890 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43542]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T12:06:31.890 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43542]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:31.890 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43542]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:31.890 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43542]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:31.890 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T12:06:31.890 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T12:06:31.890 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2888462851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825bb9bf-6e25-4933-b0fc-4be4950fac0a"}]: dispatch 2024-09-12T12:06:31.891 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2888462851' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "825bb9bf-6e25-4933-b0fc-4be4950fac0a"}]': finished 2024-09-12T12:06:31.891 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T12:06:31.891 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:31.891 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:31.891 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:31.891 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:31 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3660872926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2888462851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825bb9bf-6e25-4933-b0fc-4be4950fac0a"}]: dispatch 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2888462851' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "825bb9bf-6e25-4933-b0fc-4be4950fac0a"}]': finished 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:32.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:31 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3660872926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:32.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: osdmap e21: 3 total, 3 up, 3 in 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2888462851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "825bb9bf-6e25-4933-b0fc-4be4950fac0a"}]: dispatch 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2888462851' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "825bb9bf-6e25-4933-b0fc-4be4950fac0a"}]': finished 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:31 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3660872926' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:32.200 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43545]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T12:06:32.200 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43545]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:32.200 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43545]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:32.200 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:06:31 smithi171 sudo[43545]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:32.421 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44519]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T12:06:32.422 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44519]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:32.422 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44519]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:32.422 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44519]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:32.841 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44522]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T12:06:32.841 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44522]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:32.841 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44522]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:32.841 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:06:32 smithi189 sudo[44522]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:32.951 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46742]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T12:06:32.951 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46742]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:32.952 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46742]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:32.952 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46742]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:33.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:32 smithi189 ceph-mon[35475]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T12:06:33.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:33.212 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:33.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:32 smithi189 ceph-mon[35475]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:33.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44525]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T12:06:33.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44525]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:33.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44525]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:33.213 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44525]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46928]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T12:06:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46928]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46928]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 sudo[46928]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 ceph-mon[30950]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T12:06:33.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:33.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:33.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:32 smithi137 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:33.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:32 smithi171 ceph-mon[35098]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T12:06:33.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:33.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:33.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:32 smithi171 ceph-mon[35098]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:33.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44528]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T12:06:33.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44528]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:33.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44528]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:33.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 sudo[44528]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:33.945 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43548]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T12:06:33.946 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43548]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:33.946 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43548]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:33.946 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43548]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43551]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T12:06:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T12:06:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43551]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T12:06:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 sudo[43551]: pam_unix(sudo:session): session closed for user root 2024-09-12T12:06:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.202 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:33 smithi171 ceph-mon[35098]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:33 smithi137 ceph-mon[30950]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:06:34.342 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:33 smithi189 ceph-mon[35475]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T12:06:36.383 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:36 smithi137 ceph-mon[30950]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:36.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:36 smithi171 ceph-mon[35098]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:36.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:36 smithi189 ceph-mon[35475]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:38.411 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:38 smithi137 ceph-mon[30950]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:38.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:38 smithi171 ceph-mon[35098]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:38.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:38 smithi189 ceph-mon[35475]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:40 smithi137 ceph-mon[30950]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-12T12:06:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T12:06:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:40 smithi137 ceph-mon[30950]: Deploying daemon osd.3 on smithi137 2024-09-12T12:06:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:40 smithi171 ceph-mon[35098]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-12T12:06:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T12:06:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:40 smithi171 ceph-mon[35098]: Deploying daemon osd.3 on smithi137 2024-09-12T12:06:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:40 smithi189 ceph-mon[35475]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-12T12:06:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T12:06:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:40 smithi189 ceph-mon[35475]: Deploying daemon osd.3 on smithi137 2024-09-12T12:06:42.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:42 smithi137 ceph-mon[30950]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T12:06:42.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:42 smithi171 ceph-mon[35098]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T12:06:42.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:42 smithi189 ceph-mon[35475]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-12T12:06:44.418 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:44 smithi137 ceph-mon[30950]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T12:06:44.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:44 smithi171 ceph-mon[35098]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T12:06:44.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:44 smithi189 ceph-mon[35475]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-12T12:06:46.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:46 smithi137 ceph-mon[30950]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T12:06:46.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:46.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:46.354 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:46 smithi171 ceph-mon[35098]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T12:06:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:46 smithi189 ceph-mon[35475]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T12:06:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:48 smithi171 ceph-mon[35098]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T12:06:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:48 smithi137 ceph-mon[30950]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T12:06:48.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:48 smithi189 ceph-mon[35475]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T12:06:49.585 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.585 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.585 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:49.585 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:49.585 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.585 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:49 smithi137 ceph-mon[30950]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T12:06:49.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.624+0000 7f3ba57fa640 1 -- 172.21.15.137:0/3062242966 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ba8062630 con 0x7f3b940a2af0 2024-09-12T12:06:49.625 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 3 on host 'smithi137' 2024-09-12T12:06:49.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b940a2af0 msgr2=0x7f3b940a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:49.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b940a2af0 0x7f3b940a4fb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8069cb0 tx=0x7f3b98009110 comp rx=0 tx=0).stop 2024-09-12T12:06:49.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 msgr2=0x7f3ba8100f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8100f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b90031610 tx=0x7f3b900043b0 comp rx=0 tx=0).stop 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 shutdown_connections 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b940a2af0 0x7f3b940a4fb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ba8103970 0x7f3ba81014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ba8102fb0 0x7f3ba8100f60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 --2- 172.21.15.137:0/3062242966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ba80fff30 0x7f3ba8102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 >> 172.21.15.137:0/3062242966 conn(0x7f3ba80fba80 msgr2=0x7f3ba8106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 shutdown_connections 2024-09-12T12:06:49.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:06:49.627+0000 7f3badb6c640 1 -- 172.21.15.137:0/3062242966 wait complete. 2024-09-12T12:06:49.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:49.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:49.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:49 smithi171 ceph-mon[35098]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T12:06:49.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:49.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:49.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:49.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:49 smithi189 ceph-mon[35475]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-12T12:06:50.355 DEBUG:teuthology.orchestra.run.smithi137:osd.3> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T12:06:50.357 INFO:tasks.cephadm:Deploying osd.4 on smithi171 with /dev/vg_nvme/lv_3... 2024-09-12T12:06:50.357 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T12:06:50.609 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:06:50.649 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:50 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:50.650 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:50 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:50.650 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:50 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:50.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:50 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:50.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:50 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:50.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:50 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:50.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:50 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:06:50.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:50 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:50.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:50 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:51.678 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:51 smithi171 ceph-mon[35098]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:51.718 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:51 smithi137 ceph-mon[30950]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:51.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:51 smithi189 ceph-mon[35475]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:51.975 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:06:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3[48530]: 2024-09-12T12:06:51.750+0000 7f93ee194640 -1 osd.3 0 log_to_monitors true 2024-09-12T12:06:52.614 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:52 smithi171 ceph-mon[35098]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T12:06:52.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:52 smithi137 ceph-mon[30950]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T12:06:52.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:52 smithi189 ceph-mon[35475]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T12:06:53.039 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:06:53.066 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-09-12T12:06:53.317 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:06:53.549 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3[48530]: 2024-09-12T12:06:53.424+0000 7f93ea117640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T12:06:53.549 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3[48530]: 2024-09-12T12:06:53.428+0000 7f93e5f3f640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:06:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T12:06:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:53.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:53.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:53.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.805 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 -- 172.21.15.171:0/1812389690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 msgr2=0x7fe0841086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:53.805 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 --2- 172.21.15.171:0/1812389690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe0841086e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009ca0 tx=0x7fe07c02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:06:53.805 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 -- 172.21.15.171:0/1812389690 shutdown_connections 2024-09-12T12:06:53.805 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 --2- 172.21.15.171:0/1812389690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe0841086e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 --2- 172.21.15.171:0/1812389690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe084101490 0x7fe084101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 --2- 172.21.15.171:0/1812389690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe084100290 0x7fe084100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 -- 172.21.15.171:0/1812389690 >> 172.21.15.171:0/1812389690 conn(0x7fe0840fba40 msgr2=0x7fe0840fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 -- 172.21.15.171:0/1812389690 shutdown_connections 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.800+0000 7fe089581640 1 -- 172.21.15.171:0/1812389690 wait complete. 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 Processor -- start 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 -- start start 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe084100290 0x7fe084197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe084101490 0x7fe084198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe08419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:53.806 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe08410d8f0 con 0x7fe084101490 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe08410d770 con 0x7fe084101e50 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe089581640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe08410da70 con 0x7fe084100290 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0837fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe08419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0837fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe08419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:49082/0 (socket says 172.21.15.171:49082) 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0837fe640 1 -- 172.21.15.171:0/738192880 learned_addr learned my addr 172.21.15.171:0/738192880 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe082ffd640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe084100290 0x7fe084197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0827fc640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe084101490 0x7fe084198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0837fe640 1 -- 172.21.15.171:0/738192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe084100290 msgr2=0x7fe084197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0837fe640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe084100290 0x7fe084197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0837fe640 1 -- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe084101490 msgr2=0x7fe084198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.801+0000 7fe0837fe640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe084101490 0x7fe084198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe0837fe640 1 -- 172.21.15.171:0/738192880 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08419fb30 con 0x7fe084101e50 2024-09-12T12:06:53.807 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe082ffd640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe084100290 0x7fe084197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe0837fe640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe08419f4c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe07c005e30 tx=0x7fe07c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe06bfff640 1 -- 172.21.15.171:0/738192880 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c003b70 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe06bfff640 1 -- 172.21.15.171:0/738192880 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe07c003d10 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe06bfff640 1 -- 172.21.15.171:0/738192880 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c02fbd0 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08419fdc0 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.802+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0841a0220 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.803+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe084107690 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.804+0000 7fe06bfff640 1 -- 172.21.15.171:0/738192880 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe07c02fd70 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.805+0000 7fe06bfff640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe06c0a2bb0 0x7fe06c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.805+0000 7fe06bfff640 1 -- 172.21.15.171:0/738192880 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7fe07c0eef70 con 0x7fe084101e50 2024-09-12T12:06:53.808 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.805+0000 7fe082ffd640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe06c0a2bb0 0x7fe06c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:06:53.809 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.805+0000 7fe082ffd640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe06c0a2bb0 0x7fe06c0a5070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe074005d00 tx=0x7fe074005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:06:53.809 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.808+0000 7fe06bfff640 1 -- 172.21.15.171:0/738192880 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe07c0bd210 con 0x7fe084101e50 2024-09-12T12:06:53.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:06:53.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:06:53.982 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:06:53.982+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe084062630 con 0x7fe06c0a2bb0 2024-09-12T12:06:54.666 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi137 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:54.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:54.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi137 2024-09-12T12:06:54.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-12T12:06:54.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:06:54.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T12:06:54.742 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:54.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:54.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:54.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:54.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:54.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi137 2024-09-12T12:06:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi137 to 8544M 2024-09-12T12:06:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: from='osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:06:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T12:06:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:06:54.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:06:54.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:06:54.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:06:55.687 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:55 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:06:55.687 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:55 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:06:55.687 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:55 smithi171 ceph-mon[35098]: osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] boot 2024-09-12T12:06:55.687 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:55 smithi171 ceph-mon[35098]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T12:06:55.687 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:55 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:55.688 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:55 smithi171 ceph-mon[35098]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:06:55.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:55 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:06:55.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:55 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:06:55.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:55 smithi137 ceph-mon[30950]: osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] boot 2024-09-12T12:06:55.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:55 smithi137 ceph-mon[30950]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T12:06:55.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:55 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:55.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:55 smithi137 ceph-mon[30950]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:06:55.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:55 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:06:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:55 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:06:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:55 smithi189 ceph-mon[35475]: osd.3 [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] boot 2024-09-12T12:06:55.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:55 smithi189 ceph-mon[35475]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T12:06:55.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:55 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:06:55.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:55 smithi189 ceph-mon[35475]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:06:56.435 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:56 smithi171 ceph-mon[35098]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T12:06:56.435 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:56 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/1976678333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]: dispatch 2024-09-12T12:06:56.685 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:56 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]: dispatch 2024-09-12T12:06:56.685 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:56 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]': finished 2024-09-12T12:06:56.685 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:56 smithi171 ceph-mon[35098]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T12:06:56.685 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:56.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:56 smithi137 ceph-mon[30950]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T12:06:56.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:56 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/1976678333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]: dispatch 2024-09-12T12:06:56.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:56 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]: dispatch 2024-09-12T12:06:56.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:56 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]': finished 2024-09-12T12:06:56.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:56 smithi137 ceph-mon[30950]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T12:06:56.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:56.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:56 smithi189 ceph-mon[35475]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T12:06:56.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:56 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/1976678333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]: dispatch 2024-09-12T12:06:56.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:56 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]: dispatch 2024-09-12T12:06:56.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:56 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d266b7f-3149-4edd-b3d3-15c3ba62ce7f"}]': finished 2024-09-12T12:06:56.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:56 smithi189 ceph-mon[35475]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T12:06:56.885 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:57.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:57 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/2601282359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:57 smithi171 ceph-mon[35098]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:06:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:57 smithi171 ceph-mon[35098]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T12:06:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:57 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/2601282359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:57 smithi137 ceph-mon[30950]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:06:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:57 smithi137 ceph-mon[30950]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T12:06:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:57.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:57 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/2601282359' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:06:57.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:57 smithi189 ceph-mon[35475]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:06:57.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:57 smithi189 ceph-mon[35475]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T12:06:57.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:59.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:59 smithi171 ceph-mon[35098]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T12:06:59.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:59 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:59.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:06:59 smithi171 ceph-mon[35098]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T12:06:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:59 smithi137 ceph-mon[30950]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T12:06:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:59 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:06:59 smithi137 ceph-mon[30950]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T12:06:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:59 smithi189 ceph-mon[35475]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T12:06:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:59 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:06:59.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:06:59 smithi189 ceph-mon[35475]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T12:07:02.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:02 smithi171 ceph-mon[35098]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T12:07:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:02 smithi137 ceph-mon[30950]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T12:07:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:02 smithi189 ceph-mon[35475]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T12:07:04.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:04 smithi171 ceph-mon[35098]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T12:07:04.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:04 smithi137 ceph-mon[30950]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T12:07:04.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:04 smithi189 ceph-mon[35475]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-12T12:07:05.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:05 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T12:07:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:05 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:05.485 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:05 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T12:07:05.485 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:05 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:05 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T12:07:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:05 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:06 smithi189 ceph-mon[35475]: Deploying daemon osd.4 on smithi171 2024-09-12T12:07:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:06 smithi189 ceph-mon[35475]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T12:07:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:06 smithi171 ceph-mon[35098]: Deploying daemon osd.4 on smithi171 2024-09-12T12:07:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:06 smithi171 ceph-mon[35098]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T12:07:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:06 smithi137 ceph-mon[30950]: Deploying daemon osd.4 on smithi171 2024-09-12T12:07:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:06 smithi137 ceph-mon[30950]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T12:07:08.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:08 smithi171 ceph-mon[35098]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:07:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:08 smithi137 ceph-mon[30950]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:07:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:08 smithi189 ceph-mon[35475]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:07:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:10 smithi171 ceph-mon[35098]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:10 smithi137 ceph-mon[30950]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:10.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:10 smithi189 ceph-mon[35475]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:11.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:11 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:11.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:11 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:11.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:11 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:11.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:11 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:11.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:11 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:11.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:11 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:11.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:11 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:11 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:11 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:12 smithi171 ceph-mon[35098]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:12.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:12 smithi137 ceph-mon[30950]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:12.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:12 smithi189 ceph-mon[35475]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:14 smithi171 ceph-mon[35098]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:14 smithi137 ceph-mon[30950]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:14 smithi189 ceph-mon[35475]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:15.201 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.201+0000 7fe06bfff640 1 -- 172.21.15.171:0/738192880 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe084062630 con 0x7fe06c0a2bb0 2024-09-12T12:07:15.201 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 4 on host 'smithi171' 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe06c0a2bb0 msgr2=0x7fe06c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe06c0a2bb0 0x7fe06c0a5070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe074005d00 tx=0x7fe074005c90 comp rx=0 tx=0).stop 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 msgr2=0x7fe08419f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe08419f4c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe07c005e30 tx=0x7fe07c004ab0 comp rx=0 tx=0).stop 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 shutdown_connections 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe06c0a2bb0 0x7fe06c0a5070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe084101e50 0x7fe08419f4c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe084101490 0x7fe084198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 --2- 172.21.15.171:0/738192880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe084100290 0x7fe084197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.203+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 >> 172.21.15.171:0/738192880 conn(0x7fe0840fba40 msgr2=0x7fe084104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:07:15.204 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.204+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 shutdown_connections 2024-09-12T12:07:15.205 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:07:15.204+0000 7fe089581640 1 -- 172.21.15.171:0/738192880 wait complete. 2024-09-12T12:07:15.834 DEBUG:teuthology.orchestra.run.smithi171:osd.4> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T12:07:15.837 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-09-12T12:07:15.837 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T12:07:16.091 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:07:16.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:16 smithi171 ceph-mon[35098]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:16.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:16 smithi189 ceph-mon[35475]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:16.456 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:16 smithi137 ceph-mon[30950]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:16.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:17.162 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:07:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:07:16.886+0000 7f36c0ae6640 -1 osd.4 0 log_to_monitors true 2024-09-12T12:07:17.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:17 smithi189 ceph-mon[35475]: from='osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T12:07:17.334 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:17 smithi189 ceph-mon[35475]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T12:07:17.434 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:17 smithi171 ceph-mon[35098]: from='osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T12:07:17.434 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:17 smithi171 ceph-mon[35098]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T12:07:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:17 smithi137 ceph-mon[30950]: from='osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T12:07:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:17 smithi137 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T12:07:18.321 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:07:18.171+0000 7f36bca69640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T12:07:18.321 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:07:18.176+0000 7f36b8891640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:07:18.321 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-mon[35098]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:18.321 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-mon[35098]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T12:07:18.321 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-mon[35098]: from='osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:07:18.321 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-mon[35098]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T12:07:18.322 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:18.322 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:18 smithi171 ceph-mon[35098]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:07:18.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:18 smithi189 ceph-mon[35475]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:18 smithi189 ceph-mon[35475]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T12:07:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:18 smithi189 ceph-mon[35475]: from='osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:07:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:18 smithi189 ceph-mon[35475]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T12:07:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:18 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:18 smithi189 ceph-mon[35475]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:07:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:18 smithi137 ceph-mon[30950]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:18 smithi137 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T12:07:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:18 smithi137 ceph-mon[30950]: from='osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:07:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:18 smithi137 ceph-mon[30950]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T12:07:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:18 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:18 smithi137 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:07:18.737 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:07:18.764 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-09-12T12:07:19.019 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:07:19.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:07:19.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T12:07:19.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:19.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:19.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:19.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:19.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:19.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:19.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:07:19.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:07:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T12:07:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:19.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:19.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:19.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:19.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 -- 172.21.15.189:0/935399497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a81037c0 msgr2=0x7f11a810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 --2- 172.21.15.189:0/935399497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a81037c0 0x7f11a810a050 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009ca0 tx=0x7f11a002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 -- 172.21.15.189:0/935399497 shutdown_connections 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 --2- 172.21.15.189:0/935399497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a81037c0 0x7f11a810a050 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 --2- 172.21.15.189:0/935399497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11a8102e00 0x7f11a8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 --2- 172.21.15.189:0/935399497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a8101c00 0x7f11a8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 -- 172.21.15.189:0/935399497 >> 172.21.15.189:0/935399497 conn(0x7f11a80fd390 msgr2=0x7f11a80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 -- 172.21.15.189:0/935399497 shutdown_connections 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.529+0000 7f11ac8df640 1 -- 172.21.15.189:0/935399497 wait complete. 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 Processor -- start 2024-09-12T12:07:19.555 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 -- start start 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a8101c00 0x7f11a8197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a8102e00 0x7f11a8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11a81037c0 0x7f11a819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810d860 con 0x7f11a81037c0 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810d6e0 con 0x7f11a8102e00 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11ac8df640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810d9e0 con 0x7f11a8101c00 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11a659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a8101c00 0x7f11a8197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11a659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a8101c00 0x7f11a8197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:48224/0 (socket says 172.21.15.189:48224) 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11a659c640 1 -- 172.21.15.189:0/4247029418 learned_addr learned my addr 172.21.15.189:0/4247029418 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11a6d9d640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11a81037c0 0x7f11a819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11a659c640 1 -- 172.21.15.189:0/4247029418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a8102e00 msgr2=0x7f11a8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:19.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.530+0000 7f11a659c640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a8102e00 0x7f11a8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f11a659c640 1 -- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11a81037c0 msgr2=0x7f11a819f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f11a659c640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11a81037c0 0x7f11a819f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f11a659c640 1 -- 172.21.15.189:0/4247029418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a819fa30 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f11a659c640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a8101c00 0x7f11a8197e00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f119800d950 tx=0x7f119800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f119f7fe640 1 -- 172.21.15.189:0/4247029418 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198017020 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a819fd20 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f119f7fe640 1 -- 172.21.15.189:0/4247029418 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1198009c30 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f119f7fe640 1 -- 172.21.15.189:0/4247029418 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119800f740 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.531+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a0200 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.532+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a8108f30 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.533+0000 7f119f7fe640 1 -- 172.21.15.189:0/4247029418 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f119801a020 con 0x7f11a8101c00 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.534+0000 7f119f7fe640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f11940a2bb0 0x7f11940a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:19.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.534+0000 7f119f7fe640 1 -- 172.21.15.189:0/4247029418 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f11980cddc0 con 0x7f11a8101c00 2024-09-12T12:07:19.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.534+0000 7f119ffff640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f11940a2bb0 0x7f11940a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:07:19.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.534+0000 7f119ffff640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f11940a2bb0 0x7f11940a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f11a8199320 tx=0x7f1190005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:07:19.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.537+0000 7f119f7fe640 1 -- 172.21.15.189:0/4247029418 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1198021070 con 0x7f11a8101c00 2024-09-12T12:07:19.715 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:19.714+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f11a8062630 con 0x7f11940a2bb0 2024-09-12T12:07:20.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:07:20.440 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi171 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi171 to 8544M 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] boot 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:07:20.441 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi171 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi171 to 8544M 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] boot 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:07:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:07:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:07:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi171 2024-09-12T12:07:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi171 to 8544M 2024-09-12T12:07:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-09-12T12:07:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: osd.4 [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] boot 2024-09-12T12:07:20.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T12:07:20.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:07:20.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:07:20.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:07:20.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:21.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:21 smithi189 ceph-mon[35475]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:07:21.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:21 smithi189 ceph-mon[35475]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T12:07:21.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:21 smithi171 ceph-mon[35098]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:07:21.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:21 smithi171 ceph-mon[35098]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T12:07:21.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:21 smithi137 ceph-mon[30950]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:07:21.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:21 smithi137 ceph-mon[30950]: osdmap e33: 5 total, 5 up, 5 in 2024-09-12T12:07:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:22 smithi171 ceph-mon[35098]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:22 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/2710022177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]: dispatch 2024-09-12T12:07:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:22 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]: dispatch 2024-09-12T12:07:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:22 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]': finished 2024-09-12T12:07:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:22 smithi171 ceph-mon[35098]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T12:07:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:22 smithi137 ceph-mon[30950]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:22 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/2710022177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]: dispatch 2024-09-12T12:07:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:22 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]: dispatch 2024-09-12T12:07:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:22 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]': finished 2024-09-12T12:07:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:22 smithi137 ceph-mon[30950]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T12:07:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:22 smithi189 ceph-mon[35475]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:22 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/2710022177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]: dispatch 2024-09-12T12:07:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:22 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]: dispatch 2024-09-12T12:07:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:22 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80eab3c2-6d42-4a43-a151-b8e92c20dda1"}]': finished 2024-09-12T12:07:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:22 smithi189 ceph-mon[35475]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T12:07:22.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:23.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:23 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/795726968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:07:23.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:23 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/795726968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:07:23.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:23 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/795726968' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:07:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:24 smithi171 ceph-mon[35098]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:24.491 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:24 smithi189 ceph-mon[35475]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:24.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:24 smithi137 ceph-mon[30950]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:26.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:26 smithi171 ceph-mon[35098]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:26.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:26 smithi137 ceph-mon[30950]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:26.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:26 smithi189 ceph-mon[35475]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:28 smithi171 ceph-mon[35098]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:28.513 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:28 smithi189 ceph-mon[35475]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:28 smithi137 ceph-mon[30950]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:30.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:30 smithi171 ceph-mon[35098]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:30.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:30 smithi137 ceph-mon[30950]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:30.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:30 smithi189 ceph-mon[35475]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:31.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T12:07:31.318 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T12:07:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:31.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T12:07:31.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:32.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:32 smithi189 ceph-mon[35475]: Deploying daemon osd.5 on smithi189 2024-09-12T12:07:32.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:32 smithi189 ceph-mon[35475]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:32.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:32 smithi137 ceph-mon[30950]: Deploying daemon osd.5 on smithi189 2024-09-12T12:07:32.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:32 smithi137 ceph-mon[30950]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:32.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:32 smithi171 ceph-mon[35098]: Deploying daemon osd.5 on smithi189 2024-09-12T12:07:32.720 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:32 smithi171 ceph-mon[35098]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:34.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:34 smithi189 ceph-mon[35475]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:34.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:34 smithi137 ceph-mon[30950]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:34.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:34 smithi171 ceph-mon[35098]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:36.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:36 smithi189 ceph-mon[35475]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:36.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:36 smithi137 ceph-mon[30950]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:36.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:36 smithi171 ceph-mon[35098]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:37.474 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:37.475 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:37.475 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:37.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:37.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:37.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:38.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:38 smithi189 ceph-mon[35475]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:38.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:38 smithi137 ceph-mon[30950]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:38.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:38 smithi171 ceph-mon[35098]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:40 smithi137 ceph-mon[30950]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:40 smithi189 ceph-mon[35475]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:40 smithi171 ceph-mon[35098]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:41.081 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.080+0000 7f119f7fe640 1 -- 172.21.15.189:0/4247029418 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11a8062630 con 0x7f11940a2bb0 2024-09-12T12:07:41.081 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-09-12T12:07:41.083 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f11940a2bb0 msgr2=0x7f11940a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f11940a2bb0 0x7f11940a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f11a8199320 tx=0x7f1190005c00 comp rx=0 tx=0).stop 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a8101c00 msgr2=0x7f11a8197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a8101c00 0x7f11a8197e00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f119800d950 tx=0x7f119800de10 comp rx=0 tx=0).stop 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 shutdown_connections 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f11940a2bb0 0x7f11940a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11a81037c0 0x7f11a819f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a8102e00 0x7f11a8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 --2- 172.21.15.189:0/4247029418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11a8101c00 0x7f11a8197e00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 >> 172.21.15.189:0/4247029418 conn(0x7f11a80fd390 msgr2=0x7f11a8106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:07:41.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 shutdown_connections 2024-09-12T12:07:41.085 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:07:41.082+0000 7f11ac8df640 1 -- 172.21.15.189:0/4247029418 wait complete. 2024-09-12T12:07:41.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:41 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:41.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:41 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:41.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:41 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:41 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:41 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:41 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:41 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:07:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:41 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:41 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:41.791 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T12:07:41.795 INFO:tasks.cephadm:Deploying osd.6 on smithi137 with /dev/vg_nvme/lv_2... 2024-09-12T12:07:41.795 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T12:07:42.054 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:07:42.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:42 smithi189 ceph-mon[35475]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:42.436 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:42 smithi137 ceph-mon[30950]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:42.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:42 smithi171 ceph-mon[35098]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:42.840 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:07:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:07:42.561+0000 7f2c760d4640 -1 osd.5 0 log_to_monitors true 2024-09-12T12:07:43.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:43 smithi137 ceph-mon[30950]: from='osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T12:07:43.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:43 smithi137 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T12:07:43.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:43 smithi189 ceph-mon[35475]: from='osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T12:07:43.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:43 smithi189 ceph-mon[35475]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T12:07:43.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:43 smithi171 ceph-mon[35098]: from='osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T12:07:43.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:43 smithi171 ceph-mon[35098]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T12:07:44.316 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:44 smithi137 ceph-mon[30950]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:44.316 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:44 smithi137 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T12:07:44.316 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:44 smithi137 ceph-mon[30950]: from='osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:07:44.316 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:44 smithi137 ceph-mon[30950]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T12:07:44.317 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:44 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:44.317 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:44 smithi137 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:07:44.340 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:07:44.238+0000 7f2c72057640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T12:07:44.341 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:07:44.245+0000 7f2c6de7f640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:07:44.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-mon[35475]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:44.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-mon[35475]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T12:07:44.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-mon[35475]: from='osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:07:44.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-mon[35475]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T12:07:44.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:44.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:44 smithi189 ceph-mon[35475]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:07:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:44 smithi171 ceph-mon[35098]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:44 smithi171 ceph-mon[35098]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T12:07:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:44 smithi171 ceph-mon[35098]: from='osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:07:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:44 smithi171 ceph-mon[35098]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T12:07:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:44 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:44 smithi171 ceph-mon[35098]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:07:44.753 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:07:44.783 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-09-12T12:07:45.034 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:07:45.470 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:07:45.470 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T12:07:45.470 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:45.471 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 -- 172.21.15.137:0/3402363373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a41078c0 msgr2=0x7f17a4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:45.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 --2- 172.21.15.137:0/3402363373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a41078c0 0x7f17a4109d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f179c009ca0 tx=0x7f179c02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:07:45.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 -- 172.21.15.137:0/3402363373 shutdown_connections 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 --2- 172.21.15.137:0/3402363373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a41078c0 0x7f17a4109d10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 --2- 172.21.15.137:0/3402363373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a4104f90 0x7f17a4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 --2- 172.21.15.137:0/3402363373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a4102660 0x7f17a4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 -- 172.21.15.137:0/3402363373 >> 172.21.15.137:0/3402363373 conn(0x7f17a40fc280 msgr2=0x7f17a40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 -- 172.21.15.137:0/3402363373 shutdown_connections 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 -- 172.21.15.137:0/3402363373 wait complete. 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 Processor -- start 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.554+0000 7f17abcf1640 1 -- start start 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a4102660 0x7f17a4197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a4104f90 0x7f17a41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:45.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a41078c0 0x7f17a419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f17a410d860 con 0x7f17a4102660 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17a410d6e0 con 0x7f17a4104f90 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17a410d9e0 con 0x7f17a41078c0 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a4102660 0x7f17a4197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a4102660 0x7f17a4197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59714/0 (socket says 172.21.15.137:59714) 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 -- 172.21.15.137:0/887832122 learned_addr learned my addr 172.21.15.137:0/887832122 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a928c640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a4104f90 0x7f17a41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17aa28e640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a41078c0 0x7f17a419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 -- 172.21.15.137:0/887832122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a41078c0 msgr2=0x7f17a419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a41078c0 0x7f17a419f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 -- 172.21.15.137:0/887832122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a4104f90 msgr2=0x7f17a41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:07:45.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a4104f90 0x7f17a41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 -- 172.21.15.137:0/887832122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a419fae0 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17a9a8d640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a4102660 0x7f17a4197eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c890 tx=0x7f17a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f179affd640 1 -- 172.21.15.137:0/887832122 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004220 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a419fdd0 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f179affd640 1 -- 172.21.15.137:0/887832122 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a00043c0 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f179affd640 1 -- 172.21.15.137:0/887832122 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004a00 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.555+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a41a0220 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.556+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a40fdd70 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.557+0000 7f179affd640 1 -- 172.21.15.137:0/887832122 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f17a00079b0 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.558+0000 7f179affd640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f17900a2bb0 0x7f17900a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.558+0000 7f17a928c640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f17900a2bb0 0x7f17900a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.558+0000 7f179affd640 1 -- 172.21.15.137:0/887832122 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f17a00ce2b0 con 0x7f17a4102660 2024-09-12T12:07:45.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.558+0000 7f17a928c640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f17900a2bb0 0x7f17900a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1794005fd0 tx=0x7f1794005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:07:45.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.563+0000 7f179affd640 1 -- 172.21.15.137:0/887832122 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17a0014030 con 0x7f17a4102660 2024-09-12T12:07:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: osdmap e36: 6 total, 5 up, 6 in 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:07:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:45.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:07:45.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:07:45.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:07:45.738+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f17a4062630 con 0x7f17900a2bb0 2024-09-12T12:07:46.394 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi189 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] boot 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:07:46.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi189 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] boot 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T12:07:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:46.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:07:46.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:07:46.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi189 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi189 to 8544M 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: osd.5 [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] boot 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:07:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:07:46.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:47.496 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:47 smithi137 ceph-mon[30950]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:07:47.497 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:47 smithi137 ceph-mon[30950]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T12:07:47.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:47 smithi189 ceph-mon[35475]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:07:47.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:47 smithi189 ceph-mon[35475]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T12:07:47.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:47 smithi171 ceph-mon[35098]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:07:47.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:47 smithi171 ceph-mon[35098]: osdmap e38: 6 total, 6 up, 6 in 2024-09-12T12:07:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:48 smithi137 ceph-mon[30950]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:07:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:48 smithi137 ceph-mon[30950]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T12:07:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:48 smithi137 ceph-mon[30950]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T12:07:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:48 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3780223544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4971246b-c619-43a1-a798-e0007f23cab0"}]: dispatch 2024-09-12T12:07:48.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:48 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3780223544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4971246b-c619-43a1-a798-e0007f23cab0"}]': finished 2024-09-12T12:07:48.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:48 smithi137 ceph-mon[30950]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T12:07:48.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:07:48.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:48 smithi189 ceph-mon[35475]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:07:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:48 smithi189 ceph-mon[35475]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T12:07:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:48 smithi189 ceph-mon[35475]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T12:07:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:48 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3780223544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4971246b-c619-43a1-a798-e0007f23cab0"}]: dispatch 2024-09-12T12:07:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:48 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3780223544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4971246b-c619-43a1-a798-e0007f23cab0"}]': finished 2024-09-12T12:07:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:48 smithi189 ceph-mon[35475]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T12:07:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:07:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:48 smithi171 ceph-mon[35098]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 172 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:07:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:48 smithi171 ceph-mon[35098]: osdmap e39: 6 total, 6 up, 6 in 2024-09-12T12:07:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:48 smithi171 ceph-mon[35098]: osdmap e40: 6 total, 6 up, 6 in 2024-09-12T12:07:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:48 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3780223544' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4971246b-c619-43a1-a798-e0007f23cab0"}]: dispatch 2024-09-12T12:07:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:48 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3780223544' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4971246b-c619-43a1-a798-e0007f23cab0"}]': finished 2024-09-12T12:07:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:48 smithi171 ceph-mon[35098]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T12:07:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:07:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:49 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/936603623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:07:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:49 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/936603623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:07:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:49 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/936603623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:07:50.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:50 smithi137 ceph-mon[30950]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T12:07:50.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:50 smithi189 ceph-mon[35475]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T12:07:50.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:50 smithi171 ceph-mon[35098]: pgmap v138: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T12:07:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:52 smithi137 ceph-mon[30950]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-12T12:07:52.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:52 smithi189 ceph-mon[35475]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-12T12:07:52.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:52 smithi171 ceph-mon[35098]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-09-12T12:07:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:54 smithi137 ceph-mon[30950]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T12:07:54.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:54 smithi189 ceph-mon[35475]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T12:07:54.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:54 smithi171 ceph-mon[35098]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T12:07:56.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:56 smithi137 ceph-mon[30950]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T12:07:56.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T12:07:56.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:56.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:56 smithi189 ceph-mon[35475]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T12:07:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T12:07:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:56 smithi171 ceph-mon[35098]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T12:07:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T12:07:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:07:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:57 smithi137 ceph-mon[30950]: Deploying daemon osd.6 on smithi137 2024-09-12T12:07:57.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:57 smithi189 ceph-mon[35475]: Deploying daemon osd.6 on smithi137 2024-09-12T12:07:57.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:57 smithi171 ceph-mon[35098]: Deploying daemon osd.6 on smithi137 2024-09-12T12:07:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:58 smithi137 ceph-mon[30950]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:07:58.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:58 smithi189 ceph-mon[35475]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:07:58.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:58 smithi171 ceph-mon[35098]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:07:59.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:07:59 smithi137 ceph-mon[30950]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:07:59.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:07:59 smithi189 ceph-mon[35475]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:07:59.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:07:59 smithi171 ceph-mon[35098]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:02 smithi171 ceph-mon[35098]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:02.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:02 smithi137 ceph-mon[30950]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:02 smithi189 ceph-mon[35475]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:04.396 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:04 smithi137 ceph-mon[30950]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:04.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:04 smithi171 ceph-mon[35098]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:04.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:04 smithi189 ceph-mon[35475]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:06.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.264+0000 7f179affd640 1 -- 172.21.15.137:0/887832122 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f17a4062630 con 0x7f17900a2bb0 2024-09-12T12:08:06.265 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 6 on host 'smithi137' 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f17900a2bb0 msgr2=0x7f17900a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f17900a2bb0 0x7f17900a5070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1794005fd0 tx=0x7f1794005850 comp rx=0 tx=0).stop 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a4102660 msgr2=0x7f17a4197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a4102660 0x7f17a4197eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c890 tx=0x7f17a000cd50 comp rx=0 tx=0).stop 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 shutdown_connections 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f17900a2bb0 0x7f17900a5070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a41078c0 0x7f17a419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a4104f90 0x7f17a41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 --2- 172.21.15.137:0/887832122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17a4102660 0x7f17a4197eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:06.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 >> 172.21.15.137:0/887832122 conn(0x7f17a40fc280 msgr2=0x7f17a4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:08:06.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 shutdown_connections 2024-09-12T12:08:06.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:08:06.266+0000 7f17abcf1640 1 -- 172.21.15.137:0/887832122 wait complete. 2024-09-12T12:08:06.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:06 smithi171 ceph-mon[35098]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:06 smithi137 ceph-mon[30950]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:06 smithi189 ceph-mon[35475]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:06.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:06.914 DEBUG:teuthology.orchestra.run.smithi137:osd.6> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T12:08:06.917 INFO:tasks.cephadm:Deploying osd.7 on smithi171 with /dev/vg_nvme/lv_2... 2024-09-12T12:08:06.917 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T12:08:07.172 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:08:07.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:07.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:07.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:07 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:07.237 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:07.237 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:07.237 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:07 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:07.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:07.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:07.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:07 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:08.415 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:08 smithi137 ceph-mon[30950]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:08.415 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:08 smithi137 ceph-mon[30950]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T12:08:08.415 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:08:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:08:08.192+0000 7f6e0d153640 -1 osd.6 0 log_to_monitors true 2024-09-12T12:08:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:08 smithi189 ceph-mon[35475]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:08 smithi189 ceph-mon[35475]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T12:08:08.612 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:08 smithi171 ceph-mon[35098]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:08.613 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:08 smithi171 ceph-mon[35098]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T12:08:09.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:08:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:08:09.282+0000 7f6e090d6640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T12:08:09.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:08:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:08:09.291+0000 7f6e04efe640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:08:09.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:09 smithi137 ceph-mon[30950]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T12:08:09.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:09 smithi137 ceph-mon[30950]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T12:08:09.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:09 smithi137 ceph-mon[30950]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:08:09.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:09 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:09.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:09 smithi189 ceph-mon[35475]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T12:08:09.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:09 smithi189 ceph-mon[35475]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T12:08:09.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:09 smithi189 ceph-mon[35475]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:08:09.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:09 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:09.608 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:09 smithi171 ceph-mon[35098]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T12:08:09.608 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:09 smithi171 ceph-mon[35098]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T12:08:09.608 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:09 smithi171 ceph-mon[35098]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:08:09.608 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:09 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:09.937 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:08:09.963 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-09-12T12:08:10.210 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:08:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:08:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T12:08:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:10.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:10.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:10 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:10.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:10.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:10 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.629 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-12T12:08:10.629 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:08:10.629 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: osdmap e43: 7 total, 6 up, 7 in 2024-09-12T12:08:10.629 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:10.629 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:10.630 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:10 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:10.737 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- 172.21.15.171:0/1554159928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d8102660 msgr2=0x7fc3d8104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:10.737 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/1554159928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d8102660 0x7fc3d8104a50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009900 tx=0x7fc3c802a410 comp rx=0 tx=0).stop 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- 172.21.15.171:0/1554159928 shutdown_connections 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/1554159928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3d81078c0 0x7fc3d8109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/1554159928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3d8104f90 0x7fc3d8107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/1554159928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d8102660 0x7fc3d8104a50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- 172.21.15.171:0/1554159928 >> 172.21.15.171:0/1554159928 conn(0x7fc3d80fc280 msgr2=0x7fc3d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- 172.21.15.171:0/1554159928 shutdown_connections 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- 172.21.15.171:0/1554159928 wait complete. 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 Processor -- start 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- start start 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3d8102660 0x7fc3d8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3d8104f90 0x7fc3d81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d81078c0 0x7fc3d819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:10.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d810d8e0 con 0x7fc3d8104f90 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d810d760 con 0x7fc3d81078c0 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.713+0000 7fc3ded7b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d810da60 con 0x7fc3d8102660 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d81078c0 0x7fc3d819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d81078c0 0x7fc3d819f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:51952/0 (socket says 172.21.15.171:51952) 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 -- 172.21.15.171:0/3924292503 learned_addr learned my addr 172.21.15.171:0/3924292503 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3d8104f90 0x7fc3d81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dcb17640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3d8102660 0x7fc3d8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 -- 172.21.15.171:0/3924292503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3d8102660 msgr2=0x7fc3d8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3d8102660 0x7fc3d8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 -- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3d8104f90 msgr2=0x7fc3d81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3d8104f90 0x7fc3d81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 -- 172.21.15.171:0/3924292503 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d819fbc0 con 0x7fc3d81078c0 2024-09-12T12:08:10.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dcb17640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3d8102660 0x7fc3d8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3dd318640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d81078c0 0x7fc3d819f550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b970 tx=0x7fc3d000be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3d5ffb640 1 -- 172.21.15.171:0/3924292503 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0019070 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3d5ffb640 1 -- 172.21.15.171:0/3924292503 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d000cb40 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d819feb0 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3d5ffb640 1 -- 172.21.15.171:0/3924292503 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0007650 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.714+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d81a03e0 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.715+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d80fdd70 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.716+0000 7fc3d5ffb640 1 -- 172.21.15.171:0/3924292503 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc3d001c070 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.717+0000 7fc3d5ffb640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc3c00a2bb0 0x7fc3c00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.717+0000 7fc3d5ffb640 1 -- 172.21.15.171:0/3924292503 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7fc3d00cd850 con 0x7fc3d81078c0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.718+0000 7fc3dcb17640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc3c00a2bb0 0x7fc3c00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:10.740 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.718+0000 7fc3dcb17640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc3c00a2bb0 0x7fc3c00a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8000c00 tx=0x7fc3c8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:08:10.741 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.722+0000 7fc3d5ffb640 1 -- 172.21.15.171:0/3924292503 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3d009ae70 con 0x7fc3d81078c0 2024-09-12T12:08:10.896 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:10.895+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc3d8062630 con 0x7fc3c00a2bb0 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi137 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] boot 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:08:11.549 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:11 smithi171 ceph-mon[35098]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi137 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] boot 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:11.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:08:11.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:08:11.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:08:11.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:11.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:11 smithi137 ceph-mon[30950]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:11.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:08:11.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:08:11.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi137 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi137 to 5696M 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: osd.6 [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] boot 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:11 smithi189 ceph-mon[35475]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:12.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:12 smithi137 ceph-mon[30950]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T12:08:12.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:12 smithi189 ceph-mon[35475]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T12:08:12.650 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:12 smithi171 ceph-mon[35098]: osdmap e45: 7 total, 7 up, 7 in 2024-09-12T12:08:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:13 smithi137 ceph-mon[30950]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:13.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:13 smithi189 ceph-mon[35475]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:13.625 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:13 smithi171 ceph-mon[35098]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:14 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/3858014151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]: dispatch 2024-09-12T12:08:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:14 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]: dispatch 2024-09-12T12:08:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:14 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]': finished 2024-09-12T12:08:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:14 smithi137 ceph-mon[30950]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T12:08:14.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:14.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:14 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/606168343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:08:14.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:14 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/3858014151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]: dispatch 2024-09-12T12:08:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:14 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]: dispatch 2024-09-12T12:08:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:14 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]': finished 2024-09-12T12:08:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:14 smithi189 ceph-mon[35475]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T12:08:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:14 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/606168343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:08:14.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:14 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/3858014151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]: dispatch 2024-09-12T12:08:14.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:14 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]: dispatch 2024-09-12T12:08:14.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:14 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e16ef0a1-dcb9-45df-bde2-22fe86d53f55"}]': finished 2024-09-12T12:08:14.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:14 smithi171 ceph-mon[35098]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T12:08:14.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:14.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:14 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/606168343' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:08:15.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:15 smithi189 ceph-mon[35475]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:15.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:15 smithi171 ceph-mon[35098]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:15.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:15 smithi137 ceph-mon[30950]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:18.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:18 smithi171 ceph-mon[35098]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:18.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:18 smithi137 ceph-mon[30950]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:18.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:18 smithi189 ceph-mon[35475]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:20.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:20 smithi171 ceph-mon[35098]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:20 smithi137 ceph-mon[30950]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:20 smithi189 ceph-mon[35475]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:22.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:22 smithi171 ceph-mon[35098]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:22.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T12:08:22.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:22 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:22 smithi137 ceph-mon[30950]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T12:08:22.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:22 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:22.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:22 smithi189 ceph-mon[35475]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:22.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T12:08:22.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:22 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:23.412 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:23 smithi171 ceph-mon[35098]: Deploying daemon osd.7 on smithi171 2024-09-12T12:08:23.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:23 smithi137 ceph-mon[30950]: Deploying daemon osd.7 on smithi171 2024-09-12T12:08:23.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:23 smithi189 ceph-mon[35475]: Deploying daemon osd.7 on smithi171 2024-09-12T12:08:24.389 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:24 smithi171 ceph-mon[35098]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:24.492 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:24 smithi189 ceph-mon[35475]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:24.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:24 smithi137 ceph-mon[30950]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:26.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:26 smithi171 ceph-mon[35098]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:26.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:26 smithi137 ceph-mon[30950]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:26.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:26 smithi189 ceph-mon[35475]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:28.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:28 smithi171 ceph-mon[35098]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:28 smithi137 ceph-mon[30950]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:28.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:28 smithi189 ceph-mon[35475]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:28.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:30.277 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:30 smithi171 ceph-mon[35098]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:30.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:30 smithi137 ceph-mon[30950]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:30.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:30 smithi189 ceph-mon[35475]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:31.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:31.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:31.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:31.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:31.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:31.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:31 smithi171 ceph-mon[35098]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:32.054 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.054+0000 7fc3d5ffb640 1 -- 172.21.15.171:0/3924292503 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc3d8062630 con 0x7fc3c00a2bb0 2024-09-12T12:08:32.055 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 7 on host 'smithi171' 2024-09-12T12:08:32.057 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc3c00a2bb0 msgr2=0x7fc3c00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:32.057 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc3c00a2bb0 0x7fc3c00a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8000c00 tx=0x7fc3c8031400 comp rx=0 tx=0).stop 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d81078c0 msgr2=0x7fc3d819f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d81078c0 0x7fc3d819f550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b970 tx=0x7fc3d000be30 comp rx=0 tx=0).stop 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 shutdown_connections 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc3c00a2bb0 0x7fc3c00a5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc3d81078c0 0x7fc3d819f550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc3d8104f90 0x7fc3d81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 --2- 172.21.15.171:0/3924292503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3d8102660 0x7fc3d8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 >> 172.21.15.171:0/3924292503 conn(0x7fc3d80fc280 msgr2=0x7fc3d8102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 shutdown_connections 2024-09-12T12:08:32.058 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:08:32.057+0000 7fc3ded7b640 1 -- 172.21.15.171:0/3924292503 wait complete. 2024-09-12T12:08:32.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:31 smithi189 ceph-mon[35475]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:32.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:32.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:32.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:31 smithi137 ceph-mon[30950]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:32.718 DEBUG:teuthology.orchestra.run.smithi171:osd.7> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T12:08:32.724 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-09-12T12:08:32.724 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T12:08:32.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:32.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:32 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:32.981 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:08:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:32 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:32 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:34.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:33 smithi189 ceph-mon[35475]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:34.200 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:33 smithi137 ceph-mon[30950]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:33 smithi171 ceph-mon[35098]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:34.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:08:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:08:33.937+0000 7fe5677be640 -1 osd.7 0 log_to_monitors true 2024-09-12T12:08:34.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:34 smithi189 ceph-mon[35475]: from='osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T12:08:34.958 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:34 smithi189 ceph-mon[35475]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T12:08:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:34 smithi171 ceph-mon[35098]: from='osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T12:08:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:34 smithi171 ceph-mon[35098]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T12:08:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:34 smithi137 ceph-mon[30950]: from='osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T12:08:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:34 smithi137 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T12:08:35.735 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:08:35.761 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-09-12T12:08:35.891 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:08:35.883+0000 7fe563741640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T12:08:35.891 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:08:35.891+0000 7fe55f569640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:35.892 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:35.893 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:35.893 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:35.893 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:35 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.017 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:08:36.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:36.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:36.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:35 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T12:08:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:08:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T12:08:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:36.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:35 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:36.536 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 -- 172.21.15.189:0/1756820426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5188103870 msgr2=0x7f518810a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:36.536 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 --2- 172.21.15.189:0/1756820426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5188103870 0x7f518810a100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5180009cd0 tx=0x7f518002a8e0 comp rx=0 tx=0).stop 2024-09-12T12:08:36.536 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 -- 172.21.15.189:0/1756820426 shutdown_connections 2024-09-12T12:08:36.536 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 --2- 172.21.15.189:0/1756820426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5188103870 0x7f518810a100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:36.536 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 --2- 172.21.15.189:0/1756820426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5188102eb0 0x7f5188103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:36.536 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 --2- 172.21.15.189:0/1756820426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51880ffed0 0x7f51881002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 -- 172.21.15.189:0/1756820426 >> 172.21.15.189:0/1756820426 conn(0x7f51880fba40 msgr2=0x7f51880fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 -- 172.21.15.189:0/1756820426 shutdown_connections 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.519+0000 7f51903f7640 1 -- 172.21.15.189:0/1756820426 wait complete. 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 Processor -- start 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 -- start start 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51880ffed0 0x7f518819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5188102eb0 0x7f518819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5188103870 0x7f51881a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d910 con 0x7f5188102eb0 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d790 con 0x7f5188103870 2024-09-12T12:08:36.537 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f51903f7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f518810da90 con 0x7f51880ffed0 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51880ffed0 0x7f518819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51880ffed0 0x7f518819a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:43036/0 (socket says 172.21.15.189:43036) 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 -- 172.21.15.189:0/3254156889 learned_addr learned my addr 172.21.15.189:0/3254156889 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518d992640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5188102eb0 0x7f518819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e994640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5188103870 0x7f51881a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 -- 172.21.15.189:0/3254156889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5188103870 msgr2=0x7f51881a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5188103870 0x7f51881a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 -- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5188102eb0 msgr2=0x7f518819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5188102eb0 0x7f518819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 -- 172.21.15.189:0/3254156889 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51881a1dc0 con 0x7f51880ffed0 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518d992640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5188102eb0 0x7f518819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e994640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5188103870 0x7f51881a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:08:36.538 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.520+0000 7f518e193640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51880ffed0 0x7f518819a190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f518400d770 tx=0x7f518400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.521+0000 7f517f7fe640 1 -- 172.21.15.189:0/3254156889 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184017070 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.521+0000 7f517f7fe640 1 -- 172.21.15.189:0/3254156889 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184009c30 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.521+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51881a20b0 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.521+0000 7f517f7fe640 1 -- 172.21.15.189:0/3254156889 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184003dd0 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.521+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881a2670 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.522+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51881a28d0 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.523+0000 7f517f7fe640 1 -- 172.21.15.189:0/3254156889 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5184003f70 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.523+0000 7f517f7fe640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f51740a2bb0 0x7f51740a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.524+0000 7f517f7fe640 1 -- 172.21.15.189:0/3254156889 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7f51840ce200 con 0x7f51880ffed0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.524+0000 7f518d992640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f51740a2bb0 0x7f51740a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.524+0000 7f518d992640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f51740a2bb0 0x7f51740a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5178006fd0 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:08:36.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.527+0000 7f517f7fe640 1 -- 172.21.15.189:0/3254156889 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5184009de0 con 0x7f51880ffed0 2024-09-12T12:08:36.704 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:36.704+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5188109da0 con 0x7f51740a2bb0 2024-09-12T12:08:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi171 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi171 to 5696M 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:36 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi171 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi171 to 5696M 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:08:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:08:37.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:37.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:36 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi171 2024-09-12T12:08:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi171 to 5696M 2024-09-12T12:08:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:08:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: osdmap e48: 8 total, 7 up, 8 in 2024-09-12T12:08:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:08:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:08:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:37.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:36 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:38.159 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:08:38.159 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:08:38.159 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:08:38.160 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] boot 2024-09-12T12:08:38.160 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T12:08:38.160 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:38.160 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:38.160 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:37 smithi189 ceph-mon[35475]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T12:08:38.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:08:38.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:08:38.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:08:38.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] boot 2024-09-12T12:08:38.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T12:08:38.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:38.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:38.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:37 smithi171 ceph-mon[35098]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T12:08:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:08:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:08:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:08:38.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: osd.7 [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] boot 2024-09-12T12:08:38.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T12:08:38.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:08:38.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-12T12:08:38.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:37 smithi137 ceph-mon[30950]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T12:08:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:39 smithi171 ceph-mon[35098]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T12:08:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:39 smithi171 ceph-mon[35098]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:39 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/2359617549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]: dispatch 2024-09-12T12:08:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:39 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]: dispatch 2024-09-12T12:08:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:39 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]': finished 2024-09-12T12:08:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:39 smithi171 ceph-mon[35098]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T12:08:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:39 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:08:39.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:39 smithi189 ceph-mon[35475]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T12:08:39.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:39 smithi189 ceph-mon[35475]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:39.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:39 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/2359617549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]: dispatch 2024-09-12T12:08:39.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:39 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]: dispatch 2024-09-12T12:08:39.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:39 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]': finished 2024-09-12T12:08:39.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:39 smithi189 ceph-mon[35475]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T12:08:39.743 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:39 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:08:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:39 smithi137 ceph-mon[30950]: osdmap e51: 8 total, 8 up, 8 in 2024-09-12T12:08:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:39 smithi137 ceph-mon[30950]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:39 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/2359617549' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]: dispatch 2024-09-12T12:08:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:39 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]: dispatch 2024-09-12T12:08:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:39 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d4c0f4-ec90-4c43-a8da-53e18fcd1984"}]': finished 2024-09-12T12:08:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:39 smithi137 ceph-mon[30950]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T12:08:39.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:39 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:08:40.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:40 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/3099025832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:08:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:40 smithi171 ceph-mon[35098]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T12:08:40.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:08:40.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:40 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/3099025832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:08:40.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:40 smithi137 ceph-mon[30950]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T12:08:40.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:08:40.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:40 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/3099025832' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:08:40.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:40 smithi189 ceph-mon[35475]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T12:08:40.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:08:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:41 smithi171 ceph-mon[35098]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:41 smithi137 ceph-mon[30950]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:41.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:41 smithi189 ceph-mon[35475]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:44.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:44 smithi171 ceph-mon[35098]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:44 smithi137 ceph-mon[30950]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:44.577 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:44 smithi189 ceph-mon[35475]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:46.338 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:46 smithi189 ceph-mon[35475]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:46.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:46 smithi171 ceph-mon[35098]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:46 smithi137 ceph-mon[30950]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:48 smithi171 ceph-mon[35098]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T12:08:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:48 smithi137 ceph-mon[30950]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T12:08:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:48 smithi189 ceph-mon[35475]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T12:08:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:49.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:49 smithi189 ceph-mon[35475]: Deploying daemon osd.8 on smithi189 2024-09-12T12:08:49.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:49 smithi171 ceph-mon[35098]: Deploying daemon osd.8 on smithi189 2024-09-12T12:08:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:49 smithi137 ceph-mon[30950]: Deploying daemon osd.8 on smithi189 2024-09-12T12:08:50.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:50 smithi171 ceph-mon[35098]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:50.525 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:50 smithi189 ceph-mon[35475]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:50.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:50 smithi137 ceph-mon[30950]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:52.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:52 smithi171 ceph-mon[35098]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:52.462 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:52 smithi189 ceph-mon[35475]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:52 smithi137 ceph-mon[30950]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:54.371 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:54 smithi189 ceph-mon[35475]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:54.371 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:54.371 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:54.371 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:54.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:54 smithi171 ceph-mon[35098]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:54 smithi137 ceph-mon[30950]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:56.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:56 smithi171 ceph-mon[35098]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:56.460 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:56 smithi189 ceph-mon[35475]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:56 smithi137 ceph-mon[30950]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:57.652 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.651+0000 7f517f7fe640 1 -- 172.21.15.189:0/3254156889 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5188109da0 con 0x7f51740a2bb0 2024-09-12T12:08:57.652 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-09-12T12:08:57.654 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f51740a2bb0 msgr2=0x7f51740a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:57.654 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f51740a2bb0 0x7f51740a5070 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5178006fd0 tx=0x7f5178008040 comp rx=0 tx=0).stop 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51880ffed0 msgr2=0x7f518819a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51880ffed0 0x7f518819a190 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f518400d770 tx=0x7f518400dc30 comp rx=0 tx=0).stop 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 shutdown_connections 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f51740a2bb0 0x7f51740a5070 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5188103870 0x7f51881a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5188102eb0 0x7f518819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 --2- 172.21.15.189:0/3254156889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51880ffed0 0x7f518819a190 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 >> 172.21.15.189:0/3254156889 conn(0x7f51880fba40 msgr2=0x7f5188108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 shutdown_connections 2024-09-12T12:08:57.655 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:08:57.654+0000 7f51903f7640 1 -- 172.21.15.189:0/3254156889 wait complete. 2024-09-12T12:08:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:57 smithi171 ceph-mon[35098]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:57.976 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:57.976 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:57.976 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:57.976 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:57.977 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:57.977 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:57 smithi189 ceph-mon[35475]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:58.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:08:58.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:08:58.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:57 smithi137 ceph-mon[30950]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:58.337 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T12:08:58.339 INFO:tasks.cephadm:Deploying osd.9 on smithi137 with /dev/vg_nvme/lv_1... 2024-09-12T12:08:58.339 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T12:08:58.598 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:08:58.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:58 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:58.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:58 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:58 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.932 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:58 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:58.933 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:58 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.933 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:58 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:58 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:08:58.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:58 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:58.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:58 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:08:59.814 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:59 smithi137 ceph-mon[30950]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:59.814 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:59 smithi137 ceph-mon[30950]: from='osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T12:08:59.814 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:08:59 smithi137 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T12:08:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:59 smithi189 ceph-mon[35475]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:59 smithi189 ceph-mon[35475]: from='osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T12:08:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:08:59 smithi189 ceph-mon[35475]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T12:08:59.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:08:59 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:08:59.578+0000 7efe94edc640 -1 osd.8 0 log_to_monitors true 2024-09-12T12:08:59.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:59 smithi171 ceph-mon[35098]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:08:59.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:59 smithi171 ceph-mon[35098]: from='osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T12:08:59.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:08:59 smithi171 ceph-mon[35098]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T12:09:00.688 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:00 smithi137 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T12:09:00.689 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:00 smithi137 ceph-mon[30950]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T12:09:00.689 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:00 smithi137 ceph-mon[30950]: from='osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:09:00.689 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:00.689 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:00 smithi137 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:09:00.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:00 smithi171 ceph-mon[35098]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T12:09:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:00 smithi171 ceph-mon[35098]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T12:09:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:00 smithi171 ceph-mon[35098]: from='osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:09:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:00 smithi171 ceph-mon[35098]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:09:01.042 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:09:00 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:09:00.647+0000 7efe90e5f640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T12:09:01.042 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:09:00 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:09:00.655+0000 7efe8cc87640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:09:01.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:00 smithi189 ceph-mon[35475]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T12:09:01.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:00 smithi189 ceph-mon[35475]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T12:09:01.042 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:00 smithi189 ceph-mon[35475]: from='osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:09:01.043 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:01.043 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:00 smithi189 ceph-mon[35475]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:09:01.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:09:01.523 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-09-12T12:09:01.647 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.648 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:01.771 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:09:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:09:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T12:09:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:09:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:01.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:01.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:01 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:01.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:01.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:01.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:01.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:01 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:02.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:02.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:01 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:02.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 -- 172.21.15.137:0/826719488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678102e00 msgr2=0x7f6678103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:02.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 --2- 172.21.15.137:0/826719488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678102e00 0x7f6678103280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f666c009900 tx=0x7f666c02a410 comp rx=0 tx=0).stop 2024-09-12T12:09:02.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 -- 172.21.15.137:0/826719488 shutdown_connections 2024-09-12T12:09:02.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 --2- 172.21.15.137:0/826719488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66781037c0 0x7f667810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:02.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 --2- 172.21.15.137:0/826719488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678102e00 0x7f6678103280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:02.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 --2- 172.21.15.137:0/826719488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6678101c00 0x7f6678102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 -- 172.21.15.137:0/826719488 >> 172.21.15.137:0/826719488 conn(0x7f66780fd390 msgr2=0x7f66780ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 -- 172.21.15.137:0/826719488 shutdown_connections 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.264+0000 7f667dedd640 1 -- 172.21.15.137:0/826719488 wait complete. 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 Processor -- start 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 -- start start 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678101c00 0x7f6678197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6678102e00 0x7f66781983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66781037c0 0x7f667819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f667810d860 con 0x7f6678101c00 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f667810d6e0 con 0x7f66781037c0 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f667dedd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f667810d9e0 con 0x7f6678102e00 2024-09-12T12:09:02.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f66777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678101c00 0x7f6678197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f66777fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678101c00 0x7f6678197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37494/0 (socket says 172.21.15.137:37494) 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f66777fe640 1 -- 172.21.15.137:0/1504707778 learned_addr learned my addr 172.21.15.137:0/1504707778 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f6676ffd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6678102e00 0x7f66781983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.265+0000 7f6677fff640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66781037c0 0x7f667819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f66777fe640 1 -- 172.21.15.137:0/1504707778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6678102e00 msgr2=0x7f66781983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f66777fe640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6678102e00 0x7f66781983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f66777fe640 1 -- 172.21.15.137:0/1504707778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66781037c0 msgr2=0x7f667819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f66777fe640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66781037c0 0x7f667819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f66777fe640 1 -- 172.21.15.137:0/1504707778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667819fab0 con 0x7f6678101c00 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f66777fe640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678101c00 0x7f6678197e80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f666800c970 tx=0x7f666800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f6674ff9640 1 -- 172.21.15.137:0/1504707778 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668007c10 con 0x7f6678101c00 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f6674ff9640 1 -- 172.21.15.137:0/1504707778 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6668007db0 con 0x7f6678101c00 2024-09-12T12:09:02.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667819fda0 con 0x7f6678101c00 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f6674ff9640 1 -- 172.21.15.137:0/1504707778 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666800f940 con 0x7f6678101c00 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.266+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66781a01f0 con 0x7f6678101c00 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.267+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6678108fd0 con 0x7f6678101c00 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.268+0000 7f6674ff9640 1 -- 172.21.15.137:0/1504707778 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f666800fae0 con 0x7f6678101c00 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.269+0000 7f6674ff9640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66600a2bb0 0x7f66600a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.269+0000 7f6676ffd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66600a2bb0 0x7f66600a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.269+0000 7f6674ff9640 1 -- 172.21.15.137:0/1504707778 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5862+0+0 (secure 0 0 0) 0x7f66680ceaf0 con 0x7f6678101c00 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.269+0000 7f6676ffd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66600a2bb0 0x7f66600a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f666c000c00 tx=0x7f666c0315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:09:02.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.273+0000 7f6674ff9640 1 -- 172.21.15.137:0/1504707778 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f666809c470 con 0x7f6678101c00 2024-09-12T12:09:02.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:02.450+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6678062630 con 0x7f66600a2bb0 2024-09-12T12:09:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:09:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:09:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi189 2024-09-12T12:09:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-12T12:09:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] boot 2024-09-12T12:09:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T12:09:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T12:09:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi189 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] boot 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:02.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi189 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi189 to 5696M 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: osd.8 [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] boot 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:03 smithi171 ceph-mon[35098]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:03 smithi171 ceph-mon[35098]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:03 smithi171 ceph-mon[35098]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T12:09:03.990 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:03 smithi137 ceph-mon[30950]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:03.990 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:03 smithi137 ceph-mon[30950]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:03.990 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:03 smithi137 ceph-mon[30950]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T12:09:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:03 smithi189 ceph-mon[35475]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:03 smithi189 ceph-mon[35475]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:03 smithi189 ceph-mon[35475]: osdmap e58: 9 total, 9 up, 9 in 2024-09-12T12:09:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:05 smithi137 ceph-mon[30950]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T12:09:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:05 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2601833192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6098dac-95f6-45c0-8e0c-2299eb98ca05"}]: dispatch 2024-09-12T12:09:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:05 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2601833192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6098dac-95f6-45c0-8e0c-2299eb98ca05"}]': finished 2024-09-12T12:09:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:05 smithi137 ceph-mon[30950]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T12:09:05.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:05 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:05.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:05 smithi137 ceph-mon[30950]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:05.694 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:05 smithi189 ceph-mon[35475]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T12:09:05.694 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:05 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2601833192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6098dac-95f6-45c0-8e0c-2299eb98ca05"}]: dispatch 2024-09-12T12:09:05.694 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:05 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2601833192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6098dac-95f6-45c0-8e0c-2299eb98ca05"}]': finished 2024-09-12T12:09:05.694 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:05 smithi189 ceph-mon[35475]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T12:09:05.694 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:05 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:05.694 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:05 smithi189 ceph-mon[35475]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:05.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:05 smithi171 ceph-mon[35098]: osdmap e59: 9 total, 9 up, 9 in 2024-09-12T12:09:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:05 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2601833192' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6098dac-95f6-45c0-8e0c-2299eb98ca05"}]: dispatch 2024-09-12T12:09:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:05 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2601833192' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6098dac-95f6-45c0-8e0c-2299eb98ca05"}]': finished 2024-09-12T12:09:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:05 smithi171 ceph-mon[35098]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T12:09:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:05 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:05 smithi171 ceph-mon[35098]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:06.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:06 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2298823711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:06.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:06 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2298823711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:06.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:06 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2298823711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:07.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:07 smithi171 ceph-mon[35098]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:07 smithi137 ceph-mon[30950]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:07.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:07 smithi189 ceph-mon[35475]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:10.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:10 smithi171 ceph-mon[35098]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T12:09:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:10 smithi137 ceph-mon[30950]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T12:09:10.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:10 smithi189 ceph-mon[35475]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T12:09:12.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:12 smithi171 ceph-mon[35098]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-12T12:09:12.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:12 smithi137 ceph-mon[30950]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-12T12:09:12.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:12 smithi189 ceph-mon[35475]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-12T12:09:14.409 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:14 smithi137 ceph-mon[30950]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T12:09:14.410 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T12:09:14.410 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:14 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:14 smithi171 ceph-mon[35098]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T12:09:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T12:09:14.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:14 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:14.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:14 smithi189 ceph-mon[35475]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-12T12:09:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T12:09:14.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:14 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:15.416 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:15 smithi137 ceph-mon[30950]: Deploying daemon osd.9 on smithi137 2024-09-12T12:09:15.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:15 smithi171 ceph-mon[35098]: Deploying daemon osd.9 on smithi137 2024-09-12T12:09:15.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:15 smithi189 ceph-mon[35475]: Deploying daemon osd.9 on smithi137 2024-09-12T12:09:16.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:16 smithi137 ceph-mon[30950]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T12:09:16.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:16 smithi171 ceph-mon[35098]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T12:09:16.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:16 smithi189 ceph-mon[35475]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T12:09:18.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:18 smithi137 ceph-mon[30950]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:09:18.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:18 smithi171 ceph-mon[35098]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:09:18.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:18 smithi189 ceph-mon[35475]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:09:20.438 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:20 smithi137 ceph-mon[30950]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:09:20.439 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:20.439 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:20.439 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:20 smithi171 ceph-mon[35098]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:09:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:20 smithi189 ceph-mon[35475]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:09:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:22.428 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:22 smithi137 ceph-mon[30950]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:22.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:22 smithi171 ceph-mon[35098]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:22.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:22 smithi189 ceph-mon[35475]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:23.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.530+0000 7f6674ff9640 1 -- 172.21.15.137:0/1504707778 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6678062630 con 0x7f66600a2bb0 2024-09-12T12:09:23.533 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 9 on host 'smithi137' 2024-09-12T12:09:23.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66600a2bb0 msgr2=0x7f66600a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:23.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66600a2bb0 0x7f66600a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f666c000c00 tx=0x7f666c0315d0 comp rx=0 tx=0).stop 2024-09-12T12:09:23.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678101c00 msgr2=0x7f6678197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:23.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678101c00 0x7f6678197e80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f666800c970 tx=0x7f666800ce30 comp rx=0 tx=0).stop 2024-09-12T12:09:23.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 shutdown_connections 2024-09-12T12:09:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66600a2bb0 0x7f66600a5070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66781037c0 0x7f667819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6678102e00 0x7f66781983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 --2- 172.21.15.137:0/1504707778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6678101c00 0x7f6678197e80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 >> 172.21.15.137:0/1504707778 conn(0x7f66780fd390 msgr2=0x7f66781061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:09:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.532+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 shutdown_connections 2024-09-12T12:09:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:09:23.533+0000 7f667dedd640 1 -- 172.21.15.137:0/1504707778 wait complete. 2024-09-12T12:09:24.244 DEBUG:teuthology.orchestra.run.smithi137:osd.9> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T12:09:24.247 INFO:tasks.cephadm:Deploying osd.10 on smithi171 with /dev/vg_nvme/lv_1... 2024-09-12T12:09:24.247 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T12:09:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:24 smithi171 ceph-mon[35098]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:24 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:24 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:24.516 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:09:24.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:24 smithi137 ceph-mon[30950]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:24.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:24 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:24.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:24 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:24.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:24 smithi189 ceph-mon[35475]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:24.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:24 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:24.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:24 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:25.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:25.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:25 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:25.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:25 smithi137 ceph-mon[30950]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:25.621 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:25 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:25.621 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:25 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:25.622 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:25 smithi189 ceph-mon[35475]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:25.664 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:25 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:25.664 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:25 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:25.664 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:25 smithi171 ceph-mon[35098]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:26.367 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:09:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:09:26.088+0000 7f6186de6640 -1 osd.9 0 log_to_monitors true 2024-09-12T12:09:26.694 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:26 smithi137 ceph-mon[30950]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T12:09:26.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:26 smithi171 ceph-mon[35098]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T12:09:26.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:26 smithi189 ceph-mon[35475]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T12:09:27.154 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-12T12:09:27.183 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-09-12T12:09:27.431 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:09:27.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:27 smithi171 ceph-mon[35098]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T12:09:27.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:27 smithi171 ceph-mon[35098]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T12:09:27.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:27 smithi171 ceph-mon[35098]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:09:27.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:27 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:27.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:27 smithi171 ceph-mon[35098]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:27 smithi137 ceph-mon[30950]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T12:09:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:27 smithi137 ceph-mon[30950]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T12:09:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:27 smithi137 ceph-mon[30950]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:09:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:27 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:27 smithi137 ceph-mon[30950]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:27.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:27 smithi189 ceph-mon[35475]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T12:09:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:27 smithi189 ceph-mon[35475]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T12:09:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:27 smithi189 ceph-mon[35475]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-09-12T12:09:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:27 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:27 smithi189 ceph-mon[35475]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 -- 172.21.15.171:0/2855871570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb94076220 msgr2=0x7fbb94074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 --2- 172.21.15.171:0/2855871570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb94076220 0x7fbb94074680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbb90009900 tx=0x7fbb9002a410 comp rx=0 tx=0).stop 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 -- 172.21.15.171:0/2855871570 shutdown_connections 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 --2- 172.21.15.171:0/2855871570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb9410e1e0 0x7fbb941105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 --2- 172.21.15.171:0/2855871570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb94074c50 0x7fbb940750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 --2- 172.21.15.171:0/2855871570 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb94076220 0x7fbb94074680 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 -- 172.21.15.171:0/2855871570 >> 172.21.15.171:0/2855871570 conn(0x7fbb940fd390 msgr2=0x7fbb940ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:09:27.963 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 -- 172.21.15.171:0/2855871570 shutdown_connections 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.929+0000 7fbb9b315640 1 -- 172.21.15.171:0/2855871570 wait complete. 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 Processor -- start 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 -- start start 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb94074c50 0x7fbb9419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb94076220 0x7fbb9419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9410e1e0 0x7fbb941a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94113f60 con 0x7fbb94074c50 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94113de0 con 0x7fbb9410e1e0 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb9b315640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941140e0 con 0x7fbb94076220 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb998b2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9410e1e0 0x7fbb941a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb990b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb94074c50 0x7fbb9419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:27.964 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb990b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb94074c50 0x7fbb9419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.171:36460/0 (socket says 172.21.15.171:36460) 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb990b1640 1 -- 172.21.15.171:0/1043305205 learned_addr learned my addr 172.21.15.171:0/1043305205 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.930+0000 7fbb998b2640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9410e1e0 0x7fbb941a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:38778/0 (socket says 172.21.15.171:38778) 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb988b0640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb94076220 0x7fbb9419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb998b2640 1 -- 172.21.15.171:0/1043305205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb94076220 msgr2=0x7fbb9419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb998b2640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb94076220 0x7fbb9419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb998b2640 1 -- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb94074c50 msgr2=0x7fbb9419e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb998b2640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb94074c50 0x7fbb9419e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb998b2640 1 -- 172.21.15.171:0/1043305205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb941a6270 con 0x7fbb9410e1e0 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb990b1640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb94074c50 0x7fbb9419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb998b2640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9410e1e0 0x7fbb941a5c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00be20 tx=0x7fbb8c009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb8a7fc640 1 -- 172.21.15.171:0/1043305205 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c019070 con 0x7fbb9410e1e0 2024-09-12T12:09:27.965 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb8a7fc640 1 -- 172.21.15.171:0/1043305205 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb8c00cd70 con 0x7fbb9410e1e0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb941a6560 con 0x7fbb9410e1e0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb8a7fc640 1 -- 172.21.15.171:0/1043305205 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0127f0 con 0x7fbb9410e1e0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.931+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a6ad0 con 0x7fbb9410e1e0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.932+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb94103b80 con 0x7fbb9410e1e0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.933+0000 7fbb8a7fc640 1 -- 172.21.15.171:0/1043305205 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbb8c01c070 con 0x7fbb9410e1e0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.934+0000 7fbb8a7fc640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbb800a2bb0 0x7fbb800a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.934+0000 7fbb8a7fc640 1 -- 172.21.15.171:0/1043305205 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7fbb8c0ce2c0 con 0x7fbb9410e1e0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.934+0000 7fbb990b1640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbb800a2bb0 0x7fbb800a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.935+0000 7fbb990b1640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbb800a2bb0 0x7fbb800a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb90000c00 tx=0x7fbb90005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:09:27.966 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:27.937+0000 7fbb8a7fc640 1 -- 172.21.15.171:0/1043305205 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb8c09b4f0 con 0x7fbb9410e1e0 2024-09-12T12:09:28.049 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:09:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:09:27.767+0000 7f6182d69640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T12:09:28.049 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:09:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:09:27.780+0000 7f617ab89640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:09:28.113 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:28.113+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbb94062630 con 0x7fbb800a2bb0 2024-09-12T12:09:28.670 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi137 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.671 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:28.672 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:09:28.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T12:09:28.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi137 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:28.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-09-12T12:09:28.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T12:09:28.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:28.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:28.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350]' entity='osd.9' 2024-09-12T12:09:28.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:28.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:28.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:28.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi137 2024-09-12T12:09:28.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:28.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi137 to 4272M 2024-09-12T12:09:28.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:28.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:28.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:28.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:28.846 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:28.846 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:29.587 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:29 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:09:29.587 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:29 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:09:29.588 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:29.588 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:29 smithi171 ceph-mon[35098]: osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] boot 2024-09-12T12:09:29.588 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:29 smithi171 ceph-mon[35098]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T12:09:29.588 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:29 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:29.588 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:29 smithi171 ceph-mon[35098]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:29 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:09:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:29 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:09:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:29 smithi137 ceph-mon[30950]: osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] boot 2024-09-12T12:09:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:29 smithi137 ceph-mon[30950]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T12:09:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:29 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:29.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:29 smithi137 ceph-mon[30950]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:29 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:09:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:29 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:09:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:29 smithi189 ceph-mon[35475]: osd.9 [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] boot 2024-09-12T12:09:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:29 smithi189 ceph-mon[35475]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T12:09:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:29 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:09:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:29 smithi189 ceph-mon[35475]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-12T12:09:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:30 smithi137 ceph-mon[30950]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T12:09:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:30 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/3437042590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]: dispatch 2024-09-12T12:09:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:30 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]: dispatch 2024-09-12T12:09:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:30 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]': finished 2024-09-12T12:09:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:30 smithi137 ceph-mon[30950]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T12:09:31.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:30 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:30 smithi189 ceph-mon[35475]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T12:09:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:30 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/3437042590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]: dispatch 2024-09-12T12:09:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:30 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]: dispatch 2024-09-12T12:09:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:30 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]': finished 2024-09-12T12:09:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:30 smithi189 ceph-mon[35475]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T12:09:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:30 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:31.196 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:30 smithi171 ceph-mon[35098]: osdmap e64: 10 total, 10 up, 10 in 2024-09-12T12:09:31.196 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:30 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/3437042590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]: dispatch 2024-09-12T12:09:31.196 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:30 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]: dispatch 2024-09-12T12:09:31.196 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:30 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "579cd8cf-bf21-4438-9476-72080c6e193a"}]': finished 2024-09-12T12:09:31.196 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:30 smithi171 ceph-mon[35098]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T12:09:31.196 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:30 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:31 smithi137 ceph-mon[30950]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:31 smithi137 ceph-mon[30950]: from='client.? 172.21.15.171:0/1173688727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:32.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:31 smithi189 ceph-mon[35475]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:32.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:31 smithi189 ceph-mon[35475]: from='client.? 172.21.15.171:0/1173688727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:32.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:31 smithi171 ceph-mon[35098]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:32.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:31 smithi171 ceph-mon[35098]: from='client.? 172.21.15.171:0/1173688727' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:34.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:34 smithi171 ceph-mon[35098]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:34.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:34 smithi137 ceph-mon[30950]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:34.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:34 smithi189 ceph-mon[35475]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:36.413 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:36 smithi171 ceph-mon[35098]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:36.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:36 smithi137 ceph-mon[30950]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:36.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:36 smithi189 ceph-mon[35475]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:38.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:38 smithi171 ceph-mon[35098]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:38.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:38 smithi137 ceph-mon[30950]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:38.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:38 smithi189 ceph-mon[35475]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:40.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:40 smithi171 ceph-mon[35098]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:40.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T12:09:40.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:40.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:40 smithi171 ceph-mon[35098]: Deploying daemon osd.10 on smithi171 2024-09-12T12:09:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:40 smithi137 ceph-mon[30950]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T12:09:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:40 smithi137 ceph-mon[30950]: Deploying daemon osd.10 on smithi171 2024-09-12T12:09:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:40 smithi189 ceph-mon[35475]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T12:09:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:40 smithi189 ceph-mon[35475]: Deploying daemon osd.10 on smithi171 2024-09-12T12:09:42.418 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:42 smithi171 ceph-mon[35098]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:42 smithi137 ceph-mon[30950]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:42.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:42 smithi189 ceph-mon[35475]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:44.431 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:44 smithi171 ceph-mon[35098]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:44 smithi137 ceph-mon[30950]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:44.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:44 smithi189 ceph-mon[35475]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:45.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:45.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:45.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:45 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:45 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:45 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:46.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:46 smithi171 ceph-mon[35098]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:46 smithi137 ceph-mon[30950]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:46.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:46 smithi189 ceph-mon[35475]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:48.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:48 smithi171 ceph-mon[35098]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:48.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:48 smithi137 ceph-mon[30950]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:48.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:48.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:48 smithi189 ceph-mon[35475]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:48.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:48.650 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 10 on host 'smithi171' 2024-09-12T12:09:48.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.649+0000 7fbb8a7fc640 1 -- 172.21.15.171:0/1043305205 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbb94062630 con 0x7fbb800a2bb0 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.651+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbb800a2bb0 msgr2=0x7fbb800a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.651+0000 7fbb9b315640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbb800a2bb0 0x7fbb800a5070 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb90000c00 tx=0x7fbb90005e30 comp rx=0 tx=0).stop 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.651+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9410e1e0 msgr2=0x7fbb941a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.651+0000 7fbb9b315640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9410e1e0 0x7fbb941a5c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00be20 tx=0x7fbb8c009ed0 comp rx=0 tx=0).stop 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.651+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 shutdown_connections 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.652+0000 7fbb9b315640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbb800a2bb0 0x7fbb800a5070 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.652+0000 7fbb9b315640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9410e1e0 0x7fbb941a5c00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:48.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.652+0000 7fbb9b315640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb94076220 0x7fbb9419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:48.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.652+0000 7fbb9b315640 1 --2- 172.21.15.171:0/1043305205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb94074c50 0x7fbb9419e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:48.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.652+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 >> 172.21.15.171:0/1043305205 conn(0x7fbb940fd390 msgr2=0x7fbb940ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:09:48.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.652+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 shutdown_connections 2024-09-12T12:09:48.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-12T12:09:48.652+0000 7fbb9b315640 1 -- 172.21.15.171:0/1043305205 wait complete. 2024-09-12T12:09:49.326 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.326 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:49.326 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.326 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.353 DEBUG:teuthology.orchestra.run.smithi171:osd.10> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T12:09:49.355 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-09-12T12:09:49.356 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T12:09:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:09:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:49.611 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:09:50.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:50 smithi189 ceph-mon[35475]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:50.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:50 smithi171 ceph-mon[35098]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:50.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:50 smithi137 ceph-mon[30950]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:51.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:51 smithi189 ceph-mon[35475]: from='osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T12:09:51.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:51 smithi189 ceph-mon[35475]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T12:09:51.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:51 smithi171 ceph-mon[35098]: from='osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T12:09:51.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:51 smithi171 ceph-mon[35098]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T12:09:51.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:09:51 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:09:51.042+0000 7fd18d72c640 -1 osd.10 0 log_to_monitors true 2024-09-12T12:09:51.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:51 smithi137 ceph-mon[30950]: from='osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T12:09:51.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:51 smithi137 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T12:09:52.199 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:09:52.188+0000 7fd1896af640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T12:09:52.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-mon[35098]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:52.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-mon[35098]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T12:09:52.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-mon[35098]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T12:09:52.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:52.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-mon[35098]: from='osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:09:52.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-mon[35098]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:09:52.449 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:09:52 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:09:52.198+0000 7fd1854d7640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:09:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:52 smithi137 ceph-mon[30950]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:52 smithi137 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T12:09:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:52 smithi137 ceph-mon[30950]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T12:09:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:52 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:52 smithi137 ceph-mon[30950]: from='osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:09:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:52 smithi137 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:09:52.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:52 smithi189 ceph-mon[35475]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:52.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:52 smithi189 ceph-mon[35475]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T12:09:52.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:52 smithi189 ceph-mon[35475]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T12:09:52.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:52 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:52.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:52 smithi189 ceph-mon[35475]: from='osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:09:52.565 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:52 smithi189 ceph-mon[35475]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-09-12T12:09:52.712 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-12T12:09:52.738 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-09-12T12:09:53.001 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:09:53.445 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] boot 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.446 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:53.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:53.447 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:53 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] boot 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:53.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:53.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:53 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-12T12:09:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: osd.10 [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] boot 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:09:53.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:53.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:09:53.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:53 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:09:53.570 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 -- 172.21.15.189:0/1107173068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368107920 msgr2=0x7f8368109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 --2- 172.21.15.189:0/1107173068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368107920 0x7f8368109d10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8360009ca0 tx=0x7f836002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 -- 172.21.15.189:0/1107173068 shutdown_connections 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 --2- 172.21.15.189:0/1107173068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368107920 0x7f8368109d10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 --2- 172.21.15.189:0/1107173068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8368104ff0 0x7f83681073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 --2- 172.21.15.189:0/1107173068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83681026c0 0x7f8368104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 -- 172.21.15.189:0/1107173068 >> 172.21.15.189:0/1107173068 conn(0x7f83680fc280 msgr2=0x7f83680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 -- 172.21.15.189:0/1107173068 shutdown_connections 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 -- 172.21.15.189:0/1107173068 wait complete. 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 Processor -- start 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 -- start start 2024-09-12T12:09:53.571 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83681026c0 0x7f8368102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368104ff0 0x7f8368100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.532+0000 7f836fe13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368107920 0x7f8368100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836fe13640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f836810d860 con 0x7f8368107920 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836fe13640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f836810d6e0 con 0x7f83681026c0 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836fe13640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f836810d9e0 con 0x7f8368104ff0 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368104ff0 0x7f8368100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368104ff0 0x7f8368100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:49044/0 (socket says 172.21.15.189:49044) 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 -- 172.21.15.189:0/2182268831 learned_addr learned my addr 172.21.15.189:0/2182268831 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836e3b0640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368107920 0x7f8368100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836dbaf640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83681026c0 0x7f8368102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 -- 172.21.15.189:0/2182268831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83681026c0 msgr2=0x7f8368102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:53.572 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83681026c0 0x7f8368102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 -- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368107920 msgr2=0x7f8368100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368107920 0x7f8368100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 -- 172.21.15.189:0/2182268831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8368101470 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836e3b0640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368107920 0x7f8368100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836d3ae640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368104ff0 0x7f8368100670 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9c0 tx=0x7f835800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f835effd640 1 -- 172.21.15.189:0/2182268831 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835800cd80 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83681a1ff0 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f835effd640 1 -- 172.21.15.189:0/2182268831 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8358005500 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f835effd640 1 -- 172.21.15.189:0/2182268831 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358010690 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.533+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83681a24a0 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.534+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83680fdd70 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.536+0000 7f835effd640 1 -- 172.21.15.189:0/2182268831 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f83580046c0 con 0x7f8368104ff0 2024-09-12T12:09:53.573 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.536+0000 7f835effd640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83540a2b80 0x7f83540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:09:53.574 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.536+0000 7f835effd640 1 -- 172.21.15.189:0/2182268831 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6723+0+0 (secure 0 0 0) 0x7f83580cf010 con 0x7f8368104ff0 2024-09-12T12:09:53.574 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.536+0000 7f836dbaf640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83540a2b80 0x7f83540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:09:53.574 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.537+0000 7f836dbaf640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83540a2b80 0x7f83540a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f836400bbe0 tx=0x7f836400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:09:53.574 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.543+0000 7f835effd640 1 -- 172.21.15.189:0/2182268831 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83580541b0 con 0x7f8368104ff0 2024-09-12T12:09:53.724 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:09:53.724+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8368062630 con 0x7f83540a2b80 2024-09-12T12:09:54.399 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi171 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi171 to 4272M 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:54.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi171 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi171 to 4272M 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:54.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi171 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi171 to 4272M 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T12:09:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T12:09:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:09:55.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:55 smithi171 ceph-mon[35098]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:55.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:55 smithi171 ceph-mon[35098]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T12:09:55.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:55 smithi189 ceph-mon[35475]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:55.454 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:55 smithi189 ceph-mon[35475]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T12:09:55.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:55 smithi137 ceph-mon[30950]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:09:55.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:55 smithi137 ceph-mon[30950]: osdmap e70: 11 total, 11 up, 11 in 2024-09-12T12:09:56.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:56 smithi189 ceph-mon[35475]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:09:56.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:56 smithi189 ceph-mon[35475]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T12:09:56.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:56 smithi171 ceph-mon[35098]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:09:56.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:56 smithi171 ceph-mon[35098]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T12:09:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:56 smithi137 ceph-mon[30950]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:09:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:56 smithi137 ceph-mon[30950]: osdmap e71: 11 total, 11 up, 11 in 2024-09-12T12:09:57.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:57 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/3346390776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]: dispatch 2024-09-12T12:09:57.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:57 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]: dispatch 2024-09-12T12:09:57.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:57 smithi171 ceph-mon[35098]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]': finished 2024-09-12T12:09:57.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:57 smithi171 ceph-mon[35098]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T12:09:57.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:09:57.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:57 smithi171 ceph-mon[35098]: from='client.? 172.21.15.189:0/2051995530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:57 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/3346390776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]: dispatch 2024-09-12T12:09:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:57 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]: dispatch 2024-09-12T12:09:57.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:57 smithi137 ceph-mon[30950]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]': finished 2024-09-12T12:09:57.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:57 smithi137 ceph-mon[30950]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T12:09:57.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:09:57.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:57 smithi137 ceph-mon[30950]: from='client.? 172.21.15.189:0/2051995530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:57 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/3346390776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]: dispatch 2024-09-12T12:09:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:57 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]: dispatch 2024-09-12T12:09:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:57 smithi189 ceph-mon[35475]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "26715dfc-c88e-42be-a958-ffdb37376a62"}]': finished 2024-09-12T12:09:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:57 smithi189 ceph-mon[35475]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T12:09:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:09:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:57 smithi189 ceph-mon[35475]: from='client.? 172.21.15.189:0/2051995530' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T12:09:58.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:09:58 smithi171 ceph-mon[35098]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:09:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:09:58 smithi137 ceph-mon[30950]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:09:58.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:09:58 smithi189 ceph-mon[35475]: pgmap v233: 1 pgs: 1 remapped+peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:00 smithi171 ceph-mon[35098]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T12:10:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:00 smithi171 ceph-mon[35098]: overall HEALTH_OK 2024-09-12T12:10:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:00 smithi137 ceph-mon[30950]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T12:10:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:00 smithi137 ceph-mon[30950]: overall HEALTH_OK 2024-09-12T12:10:00.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:00 smithi189 ceph-mon[35475]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-12T12:10:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:00 smithi189 ceph-mon[35475]: overall HEALTH_OK 2024-09-12T12:10:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:02 smithi137 ceph-mon[30950]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T12:10:02.568 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:02 smithi189 ceph-mon[35475]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T12:10:02.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:02 smithi171 ceph-mon[35098]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T12:10:04.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:04 smithi137 ceph-mon[30950]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T12:10:04.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:04 smithi189 ceph-mon[35475]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T12:10:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:04 smithi171 ceph-mon[35098]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T12:10:05.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:05 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T12:10:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:05 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:05.479 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:05 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T12:10:05.479 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:05 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:05 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T12:10:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:05 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:06 smithi137 ceph-mon[30950]: Deploying daemon osd.11 on smithi189 2024-09-12T12:10:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:06 smithi137 ceph-mon[30950]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:10:06.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:06 smithi189 ceph-mon[35475]: Deploying daemon osd.11 on smithi189 2024-09-12T12:10:06.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:06 smithi189 ceph-mon[35475]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:10:06.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:06 smithi171 ceph-mon[35098]: Deploying daemon osd.11 on smithi189 2024-09-12T12:10:06.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:06 smithi171 ceph-mon[35098]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T12:10:08.478 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:08 smithi189 ceph-mon[35475]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T12:10:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:08 smithi137 ceph-mon[30950]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T12:10:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:08 smithi171 ceph-mon[35098]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T12:10:10.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:10 smithi137 ceph-mon[30950]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:10:10.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:10 smithi189 ceph-mon[35475]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:10:10.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:10 smithi171 ceph-mon[35098]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T12:10:12.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:12 smithi137 ceph-mon[30950]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:12.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:12.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:12.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:12 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:12.569 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:12 smithi189 ceph-mon[35475]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:12.569 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:12.569 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:12.570 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:12 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:12.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:12 smithi171 ceph-mon[35098]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:12.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:12.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:12.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:12 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:14.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:14 smithi189 ceph-mon[35475]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:14.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:14 smithi137 ceph-mon[30950]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:14.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:14 smithi171 ceph-mon[35098]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:14.781 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.781+0000 7f835effd640 1 -- 172.21.15.189:0/2182268831 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8368062630 con 0x7f83540a2b80 2024-09-12T12:10:14.782 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-09-12T12:10:14.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83540a2b80 msgr2=0x7f83540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:14.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83540a2b80 0x7f83540a5040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f836400bbe0 tx=0x7f836400b390 comp rx=0 tx=0).stop 2024-09-12T12:10:14.783 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368104ff0 msgr2=0x7f8368100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368104ff0 0x7f8368100670 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9c0 tx=0x7f835800ee80 comp rx=0 tx=0).stop 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 shutdown_connections 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83540a2b80 0x7f83540a5040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8368107920 0x7f8368100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8368104ff0 0x7f8368100670 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 --2- 172.21.15.189:0/2182268831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83681026c0 0x7f8368102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 >> 172.21.15.189:0/2182268831 conn(0x7f83680fc280 msgr2=0x7f8368102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 shutdown_connections 2024-09-12T12:10:14.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:14.783+0000 7f836fe13640 1 -- 172.21.15.189:0/2182268831 wait complete. 2024-09-12T12:10:15.525 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T12:10:15.527 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T12:10:15.527 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T12:10:15.794 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:15.914 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:15.914 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:15.914 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:15 smithi137 ceph-mon[30950]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:15.914 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:15.914 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:15 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.029 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.029 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.029 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:15 smithi189 ceph-mon[35475]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:16.029 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.030 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:15 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:15 smithi171 ceph-mon[35098]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:16.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:15 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:16.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 -- 172.21.15.137:0/3053156675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4102690 msgr2=0x7f40b4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:16.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3053156675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4102690 0x7f40b4108f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009ca0 tx=0x7f40ac02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 -- 172.21.15.137:0/3053156675 shutdown_connections 2024-09-12T12:10:16.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3053156675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4102690 0x7f40b4108f20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3053156675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40b4101cd0 0x7f40b4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3053156675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40b4100ad0 0x7f40b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 -- 172.21.15.137:0/3053156675 >> 172.21.15.137:0/3053156675 conn(0x7f40b40fc280 msgr2=0x7f40b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 -- 172.21.15.137:0/3053156675 shutdown_connections 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.275+0000 7f40b8c44640 1 -- 172.21.15.137:0/3053156675 wait complete. 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 Processor -- start 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 -- start start 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40b4100ad0 0x7f40b406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4101cd0 0x7f40b419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40b4102690 0x7f40b419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f40b410c800 con 0x7f40b4101cd0 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f40b410c680 con 0x7f40b4100ad0 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b8c44640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40b410c980 con 0x7f40b4102690 2024-09-12T12:10:16.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4101cd0 0x7f40b419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4101cd0 0x7f40b419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33530/0 (socket says 172.21.15.137:33530) 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b1d9b640 1 -- 172.21.15.137:0/3281295900 learned_addr learned my addr 172.21.15.137:0/3281295900 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b2d9d640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40b4102690 0x7f40b419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.276+0000 7f40b259c640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40b4100ad0 0x7f40b406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b1d9b640 1 -- 172.21.15.137:0/3281295900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40b4102690 msgr2=0x7f40b419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b1d9b640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40b4102690 0x7f40b419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b1d9b640 1 -- 172.21.15.137:0/3281295900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40b4100ad0 msgr2=0x7f40b406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b1d9b640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40b4100ad0 0x7f40b406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b1d9b640 1 -- 172.21.15.137:0/3281295900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b419fbb0 con 0x7f40b4101cd0 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b1d9b640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4101cd0 0x7f40b419cc80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e9f0 tx=0x7f40a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40a37fe640 1 -- 172.21.15.137:0/3281295900 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a800ccc0 con 0x7f40b4101cd0 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40a37fe640 1 -- 172.21.15.137:0/3281295900 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a800ce60 con 0x7f40b4101cd0 2024-09-12T12:10:16.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40a37fe640 1 -- 172.21.15.137:0/3281295900 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8010690 con 0x7f40b4101cd0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b419fe40 con 0x7f40b4101cd0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.277+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a0320 con 0x7f40b4101cd0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.278+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b4107ea0 con 0x7f40b4101cd0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.279+0000 7f40a37fe640 1 -- 172.21.15.137:0/3281295900 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f40a80046c0 con 0x7f40b4101cd0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.280+0000 7f40a37fe640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f409c0a2b80 0x7f409c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.280+0000 7f40b259c640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f409c0a2b80 0x7f409c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.280+0000 7f40a37fe640 1 -- 172.21.15.137:0/3281295900 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7f40a80ceed0 con 0x7f40b4101cd0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.281+0000 7f40b259c640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f409c0a2b80 0x7f409c0a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f40b4101b30 tx=0x7f40a4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:16.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.284+0000 7f40a37fe640 1 -- 172.21.15.137:0/3281295900 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40a809c370 con 0x7f40b4101cd0 2024-09-12T12:10:16.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.466+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f40b4062df0 con 0x7f40b4101cd0 2024-09-12T12:10:16.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.467+0000 7f40a37fe640 1 -- 172.21.15.137:0/3281295900 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f40a809bc70 con 0x7f40b4101cd0 2024-09-12T12:10:16.466 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:16.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f409c0a2b80 msgr2=0x7f409c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:16.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f409c0a2b80 0x7f409c0a5040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f40b4101b30 tx=0x7f40a4009110 comp rx=0 tx=0).stop 2024-09-12T12:10:16.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4101cd0 msgr2=0x7f40b419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4101cd0 0x7f40b419cc80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f40a800e9f0 tx=0x7f40a800edf0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 shutdown_connections 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f409c0a2b80 0x7f409c0a5040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f40b4102690 0x7f40b419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40b4101cd0 0x7f40b419cc80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 --2- 172.21.15.137:0/3281295900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40b4100ad0 0x7f40b406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 >> 172.21.15.137:0/3281295900 conn(0x7f40b40fc280 msgr2=0x7f40b4105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 shutdown_connections 2024-09-12T12:10:16.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:16.469+0000 7f40b8c44640 1 -- 172.21.15.137:0/3281295900 wait complete. 2024-09-12T12:10:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:16 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3281295900' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:17.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:16 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3281295900' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:17.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:16 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3281295900' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:17.283 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726142992,"num_in_osds":12,"osd_in_since":1726142996,"num_remapped_pgs":0} 2024-09-12T12:10:17.481 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:10:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:10:17.131+0000 7f919003d640 -1 osd.11 0 log_to_monitors true 2024-09-12T12:10:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:17 smithi137 ceph-mon[30950]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:17 smithi137 ceph-mon[30950]: from='osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T12:10:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:17 smithi137 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T12:10:18.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:17 smithi189 ceph-mon[35475]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:18.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:17 smithi189 ceph-mon[35475]: from='osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T12:10:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:17 smithi189 ceph-mon[35475]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T12:10:18.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:17 smithi171 ceph-mon[35098]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:18.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:17 smithi171 ceph-mon[35098]: from='osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T12:10:18.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:17 smithi171 ceph-mon[35098]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T12:10:18.285 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T12:10:18.545 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:18.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:10:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:10:18.795+0000 7f918bfc0640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T12:10:18.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:10:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:10:18.808+0000 7f9187de8640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T12:10:18.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:18 smithi189 ceph-mon[35475]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T12:10:18.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:18 smithi189 ceph-mon[35475]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T12:10:18.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:18 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:18.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:18 smithi189 ceph-mon[35475]: from='osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:10:18.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:18 smithi189 ceph-mon[35475]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:10:18.958 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:18 smithi137 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T12:10:18.958 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:18 smithi137 ceph-mon[30950]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T12:10:18.958 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:18 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:18.959 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:18 smithi137 ceph-mon[30950]: from='osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:10:18.959 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:18 smithi137 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:10:19.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- 172.21.15.137:0/2313648452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d41026e0 msgr2=0x7f61d4104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/2313648452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d41026e0 0x7f61d4104ad0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009900 tx=0x7f61c402a410 comp rx=0 tx=0).stop 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- 172.21.15.137:0/2313648452 shutdown_connections 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/2313648452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d4107940 0x7f61d4109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/2313648452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d4105010 0x7f61d4107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/2313648452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d41026e0 0x7f61d4104ad0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- 172.21.15.137:0/2313648452 >> 172.21.15.137:0/2313648452 conn(0x7f61d40fc280 msgr2=0x7f61d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- 172.21.15.137:0/2313648452 shutdown_connections 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- 172.21.15.137:0/2313648452 wait complete. 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 Processor -- start 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- start start 2024-09-12T12:10:19.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d41026e0 0x7f61d4102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:19.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d4105010 0x7f61d4100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:19.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d4107940 0x7f61d4100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:19.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410d860 con 0x7f61d4107940 2024-09-12T12:10:19.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410d6e0 con 0x7f61d41026e0 2024-09-12T12:10:19.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.043+0000 7f61dc6bf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61d410d9e0 con 0x7f61d4105010 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d4107940 0x7f61d4100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d4107940 0x7f61d4100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33552/0 (socket says 172.21.15.137:33552) 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 -- 172.21.15.137:0/3239026446 learned_addr learned my addr 172.21.15.137:0/3239026446 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61da45b640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d41026e0 0x7f61d4102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61d9c5a640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d4105010 0x7f61d4100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 -- 172.21.15.137:0/3239026446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d4105010 msgr2=0x7f61d4100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d4105010 0x7f61d4100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 -- 172.21.15.137:0/3239026446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d41026e0 msgr2=0x7f61d4102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d41026e0 0x7f61d4102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 -- 172.21.15.137:0/3239026446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d4101490 con 0x7f61d4107940 2024-09-12T12:10:19.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61d9c5a640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d4105010 0x7f61d4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dac5c640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d4107940 0x7f61d4100bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009fd0 tx=0x7f61cc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61cb7fe640 1 -- 172.21.15.137:0/3239026446 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc017070 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61cb7fe640 1 -- 172.21.15.137:0/3239026446 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61cc003e50 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d41a20d0 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61cb7fe640 1 -- 172.21.15.137:0/3239026446 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0106c0 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.044+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d41a2590 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.045+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d40fdd70 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.046+0000 7f61cb7fe640 1 -- 172.21.15.137:0/3239026446 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f61cc003970 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.047+0000 7f61cb7fe640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c00a2b80 0x7f61c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.047+0000 7f61da45b640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c00a2b80 0x7f61c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.047+0000 7f61cb7fe640 1 -- 172.21.15.137:0/3239026446 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7f61cc0ce770 con 0x7f61d4107940 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.048+0000 7f61da45b640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c00a2b80 0x7f61c00a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f61c40314e0 tx=0x7f61c4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:19.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.051+0000 7f61cb7fe640 1 -- 172.21.15.137:0/3239026446 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61cc09bc90 con 0x7f61d4107940 2024-09-12T12:10:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:18 smithi171 ceph-mon[35098]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T12:10:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:18 smithi171 ceph-mon[35098]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T12:10:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:18 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:19.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:18 smithi171 ceph-mon[35098]: from='osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:10:19.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:18 smithi171 ceph-mon[35098]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-09-12T12:10:19.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.239+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f61d4062df0 con 0x7f61d4107940 2024-09-12T12:10:19.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.240+0000 7f61cb7fe640 1 -- 172.21.15.137:0/3239026446 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f61cc09b590 con 0x7f61d4107940 2024-09-12T12:10:19.240 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:19.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c00a2b80 msgr2=0x7f61c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c00a2b80 0x7f61c00a5040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f61c40314e0 tx=0x7f61c4000bc0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d4107940 msgr2=0x7f61d4100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d4107940 0x7f61d4100bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009fd0 tx=0x7f61cc00b260 comp rx=0 tx=0).stop 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 shutdown_connections 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f61c00a2b80 0x7f61c00a5040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61d4107940 0x7f61d4100bd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61d4105010 0x7f61d4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 --2- 172.21.15.137:0/3239026446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61d41026e0 0x7f61d4102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 >> 172.21.15.137:0/3239026446 conn(0x7f61d40fc280 msgr2=0x7f61d4103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 shutdown_connections 2024-09-12T12:10:19.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:19.242+0000 7f61dc6bf640 1 -- 172.21.15.137:0/3239026446 wait complete. 2024-09-12T12:10:20.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:10:20.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: Detected new or changed devices on smithi189 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:19 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3239026446' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:20.093 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726142992,"num_in_osds":12,"osd_in_since":1726142996,"num_remapped_pgs":0} 2024-09-12T12:10:20.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:10:20.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T12:10:20.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:20.094 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:20.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: Detected new or changed devices on smithi189 2024-09-12T12:10:20.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.095 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.096 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.096 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.096 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-12T12:10:20.096 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.096 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:20.096 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:20.097 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.097 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:20.097 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:20.097 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:20.098 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:20.098 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.098 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:19 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3239026446' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: Detected new or changed devices on smithi189 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi189 to 4272M 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:20.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:19 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3239026446' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:20 smithi189 ceph-mon[35475]: purged_snaps scrub starts 2024-09-12T12:10:21.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:20 smithi189 ceph-mon[35475]: purged_snaps scrub ok 2024-09-12T12:10:21.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:20 smithi189 ceph-mon[35475]: osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] boot 2024-09-12T12:10:21.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:20 smithi189 ceph-mon[35475]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T12:10:21.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:20 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:21.094 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T12:10:21.121 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:20 smithi137 ceph-mon[30950]: purged_snaps scrub starts 2024-09-12T12:10:21.122 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:20 smithi137 ceph-mon[30950]: purged_snaps scrub ok 2024-09-12T12:10:21.122 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:20 smithi137 ceph-mon[30950]: osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] boot 2024-09-12T12:10:21.122 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:20 smithi137 ceph-mon[30950]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T12:10:21.122 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:20 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:21.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:20 smithi171 ceph-mon[35098]: purged_snaps scrub starts 2024-09-12T12:10:21.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:20 smithi171 ceph-mon[35098]: purged_snaps scrub ok 2024-09-12T12:10:21.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:20 smithi171 ceph-mon[35098]: osd.11 [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] boot 2024-09-12T12:10:21.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:20 smithi171 ceph-mon[35098]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T12:10:21.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:20 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:10:21.354 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.797+0000 7f4208c34640 1 -- 172.21.15.137:0/2229948122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 msgr2=0x7f420410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.797+0000 7f4208c34640 1 --2- 172.21.15.137:0/2229948122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f420410a050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009ca0 tx=0x7f41fc02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.797+0000 7f4208c34640 1 -- 172.21.15.137:0/2229948122 shutdown_connections 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.797+0000 7f4208c34640 1 --2- 172.21.15.137:0/2229948122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f420410a050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.797+0000 7f4208c34640 1 --2- 172.21.15.137:0/2229948122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4204102e00 0x7f4204103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.797+0000 7f4208c34640 1 --2- 172.21.15.137:0/2229948122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4204101c00 0x7f4204102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.797+0000 7f4208c34640 1 -- 172.21.15.137:0/2229948122 >> 172.21.15.137:0/2229948122 conn(0x7f42040fd390 msgr2=0x7f42040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.798+0000 7f4208c34640 1 -- 172.21.15.137:0/2229948122 shutdown_connections 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.798+0000 7f4208c34640 1 -- 172.21.15.137:0/2229948122 wait complete. 2024-09-12T12:10:21.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.798+0000 7f4208c34640 1 Processor -- start 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.798+0000 7f4208c34640 1 -- start start 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4204101c00 0x7f4204075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4204102e00 0x7f42040745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f4204074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f420410d980 con 0x7f42041037c0 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f420410d800 con 0x7f4204102e00 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f420410db00 con 0x7f4204101c00 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f4204074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f4204074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41112/0 (socket says 172.21.15.137:41112) 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 -- 172.21.15.137:0/4155817440 learned_addr learned my addr 172.21.15.137:0/4155817440 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:21.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f41fbfff640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4204102e00 0x7f42040745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:21.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f420259c640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4204101c00 0x7f4204075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:21.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 -- 172.21.15.137:0/4155817440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4204101c00 msgr2=0x7f4204075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:21.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4204101c00 0x7f4204075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 -- 172.21.15.137:0/4155817440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4204102e00 msgr2=0x7f42040745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:21.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4204102e00 0x7f42040745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 -- 172.21.15.137:0/4155817440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42040753e0 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4202d9d640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f4204074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0098d0 tx=0x7f41fc02f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f41fb7fe640 1 -- 172.21.15.137:0/4155817440 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc02fc30 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f41fb7fe640 1 -- 172.21.15.137:0/4155817440 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc003e50 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42041a2130 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f41fb7fe640 1 -- 172.21.15.137:0/4155817440 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc0305c0 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.799+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42041a26c0 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.800+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4204109050 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.801+0000 7f41fb7fe640 1 -- 172.21.15.137:0/4155817440 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f41fc003970 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.802+0000 7f41fb7fe640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f41f00a2b80 0x7f41f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.803+0000 7f420259c640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f41f00a2b80 0x7f41f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.803+0000 7f41fb7fe640 1 -- 172.21.15.137:0/4155817440 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f41fc0f0080 con 0x7f42041037c0 2024-09-12T12:10:21.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.803+0000 7f420259c640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f41f00a2b80 0x7f41f00a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4204102c60 tx=0x7f41f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:21.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.806+0000 7f41fb7fe640 1 -- 172.21.15.137:0/4155817440 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41fc0bd500 con 0x7f42041037c0 2024-09-12T12:10:21.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.973+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4204062df0 con 0x7f42041037c0 2024-09-12T12:10:21.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.974+0000 7f41fb7fe640 1 -- 172.21.15.137:0/4155817440 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f41fc0bce00 con 0x7f42041037c0 2024-09-12T12:10:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:21.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f41f00a2b80 msgr2=0x7f41f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:21.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f41f00a2b80 0x7f41f00a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4204102c60 tx=0x7f41f4008040 comp rx=0 tx=0).stop 2024-09-12T12:10:21.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 msgr2=0x7f4204074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:21.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f4204074b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0098d0 tx=0x7f41fc02f840 comp rx=0 tx=0).stop 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 shutdown_connections 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f41f00a2b80 0x7f41f00a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42041037c0 0x7f4204074b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4204102e00 0x7f42040745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 --2- 172.21.15.137:0/4155817440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4204101c00 0x7f4204075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 >> 172.21.15.137:0/4155817440 conn(0x7f42040fd390 msgr2=0x7f4204106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 shutdown_connections 2024-09-12T12:10:21.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:21.976+0000 7f4208c34640 1 -- 172.21.15.137:0/4155817440 wait complete. 2024-09-12T12:10:22.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:21 smithi137 ceph-mon[30950]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T12:10:22.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:21 smithi137 ceph-mon[30950]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:22.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:21 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4155817440' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:22.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:21 smithi189 ceph-mon[35475]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T12:10:22.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:21 smithi189 ceph-mon[35475]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:22.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:21 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4155817440' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:21 smithi171 ceph-mon[35098]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T12:10:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:21 smithi171 ceph-mon[35098]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:22.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:21 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4155817440' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T12:10:22.811 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726143019,"num_in_osds":12,"osd_in_since":1726142996,"num_remapped_pgs":0} 2024-09-12T12:10:22.812 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:10:23.094 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:23.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 -- 172.21.15.137:0/1089389517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c106a80 msgr2=0x7fe26c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:23.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 --2- 172.21.15.137:0/1089389517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c106a80 0x7fe26c106e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009930 tx=0x7fe25c02a420 comp rx=0 tx=0).stop 2024-09-12T12:10:23.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 -- 172.21.15.137:0/1089389517 shutdown_connections 2024-09-12T12:10:23.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 --2- 172.21.15.137:0/1089389517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe26c101420 0x7fe26c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 --2- 172.21.15.137:0/1089389517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe26c100a80 0x7fe26c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 --2- 172.21.15.137:0/1089389517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c106a80 0x7fe26c106e60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 -- 172.21.15.137:0/1089389517 >> 172.21.15.137:0/1089389517 conn(0x7fe26c0fc7a0 msgr2=0x7fe26c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 -- 172.21.15.137:0/1089389517 shutdown_connections 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 -- 172.21.15.137:0/1089389517 wait complete. 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.582+0000 7fe2717d9640 1 Processor -- start 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 -- start start 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe26c100a80 0x7fe26c197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c101420 0x7fe26c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe26c106a80 0x7fe26c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c110ea0 con 0x7fe26c101420 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c110d20 con 0x7fe26c100a80 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c111020 con 0x7fe26c106a80 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c101420 0x7fe26c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:23.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c101420 0x7fe26c197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41142/0 (socket says 172.21.15.137:41142) 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 -- 172.21.15.137:0/3649518264 learned_addr learned my addr 172.21.15.137:0/3649518264 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe26b7fe640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe26c106a80 0x7fe26c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe26affd640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe26c100a80 0x7fe26c197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 -- 172.21.15.137:0/3649518264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe26c106a80 msgr2=0x7fe26c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe26c106a80 0x7fe26c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 -- 172.21.15.137:0/3649518264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe26c100a80 msgr2=0x7fe26c197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe26c100a80 0x7fe26c197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 -- 172.21.15.137:0/3649518264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c1a69b0 con 0x7fe26c101420 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe26b7fe640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe26c106a80 0x7fe26c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe263fff640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c101420 0x7fe26c197680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe25400b4a0 tx=0x7fe25400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe268ff9640 1 -- 172.21.15.137:0/3649518264 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254014070 con 0x7fe26c101420 2024-09-12T12:10:23.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe268ff9640 1 -- 172.21.15.137:0/3649518264 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe254010af0 con 0x7fe26c101420 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c1a6ca0 con 0x7fe26c101420 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.583+0000 7fe268ff9640 1 -- 172.21.15.137:0/3649518264 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254017d10 con 0x7fe26c101420 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.584+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c1a7190 con 0x7fe26c101420 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.585+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c100ee0 con 0x7fe26c101420 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.586+0000 7fe268ff9640 1 -- 172.21.15.137:0/3649518264 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe254017070 con 0x7fe26c101420 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.586+0000 7fe268ff9640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2580a2b80 0x7fe2580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.587+0000 7fe26affd640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2580a2b80 0x7fe2580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.587+0000 7fe268ff9640 1 -- 172.21.15.137:0/3649518264 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe2540cdfc0 con 0x7fe26c101420 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.587+0000 7fe26affd640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2580a2b80 0x7fe2580a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe25c000c00 tx=0x7fe25c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:23.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.591+0000 7fe268ff9640 1 -- 172.21.15.137:0/3649518264 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe25409b3c0 con 0x7fe26c101420 2024-09-12T12:10:23.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.756+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe26c108a60 con 0x7fe26c101420 2024-09-12T12:10:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.757+0000 7fe268ff9640 1 -- 172.21.15.137:0/3649518264 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18537 (secure 0 0 0) 0x7fe25409acc0 con 0x7fe26c101420 2024-09-12T12:10:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:23.758 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":76,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:10:20.986497+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:10:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2580a2b80 msgr2=0x7fe2580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2580a2b80 0x7fe2580a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe25c000c00 tx=0x7fe25c005a20 comp rx=0 tx=0).stop 2024-09-12T12:10:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c101420 msgr2=0x7fe26c197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c101420 0x7fe26c197680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe25400b4a0 tx=0x7fe25400b960 comp rx=0 tx=0).stop 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 shutdown_connections 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2580a2b80 0x7fe2580a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe26c106a80 0x7fe26c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe26c101420 0x7fe26c197680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 --2- 172.21.15.137:0/3649518264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe26c100a80 0x7fe26c197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 >> 172.21.15.137:0/3649518264 conn(0x7fe26c0fc7a0 msgr2=0x7fe26c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 shutdown_connections 2024-09-12T12:10:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:23.760+0000 7fe2717d9640 1 -- 172.21.15.137:0/3649518264 wait complete. 2024-09-12T12:10:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:24 smithi171 ceph-mon[35098]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:24.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:24 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3649518264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:10:24.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:24 smithi137 ceph-mon[30950]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:24.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:24 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3649518264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:10:24.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:24 smithi189 ceph-mon[35475]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:24.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:24 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3649518264' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:10:24.614 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:06:29.335398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:10:24.615 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T12:10:24.858 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:25.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 -- 172.21.15.137:0/1160457885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14100e20 msgr2=0x7fcf14101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/1160457885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14100e20 0x7fcf14101200 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009900 tx=0x7fcf0402a480 comp rx=0 tx=0).stop 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 -- 172.21.15.137:0/1160457885 shutdown_connections 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/1160457885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14102ef0 0x7fcf1410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/1160457885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf14102570 0x7fcf141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/1160457885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14100e20 0x7fcf14101200 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 -- 172.21.15.137:0/1160457885 >> 172.21.15.137:0/1160457885 conn(0x7fcf140fc7d0 msgr2=0x7fcf140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 -- 172.21.15.137:0/1160457885 shutdown_connections 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 -- 172.21.15.137:0/1160457885 wait complete. 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 Processor -- start 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.320+0000 7fcf18fbe640 1 -- start start 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf14100e20 0x7fcf14198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14102570 0x7fcf14198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14102ef0 0x7fcf1419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14110fc0 con 0x7fcf14102ef0 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14110e40 con 0x7fcf14100e20 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14111140 con 0x7fcf14102570 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14102ef0 0x7fcf1419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14102ef0 0x7fcf1419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41160/0 (socket says 172.21.15.137:41160) 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 -- 172.21.15.137:0/3340539417 learned_addr learned my addr 172.21.15.137:0/3340539417 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf0bfff640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14102570 0x7fcf14198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf1259c640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf14100e20 0x7fcf14198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 -- 172.21.15.137:0/3340539417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14102570 msgr2=0x7fcf14198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14102570 0x7fcf14198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 -- 172.21.15.137:0/3340539417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf14100e20 msgr2=0x7fcf14198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf14100e20 0x7fcf14198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 -- 172.21.15.137:0/3340539417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1419d4b0 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf1259c640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf14100e20 0x7fcf14198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf12d9d640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14102ef0 0x7fcf1419cdd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00eaa0 tx=0x7fcf0c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf0b7fe640 1 -- 172.21.15.137:0/3340539417 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c004d20 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1419d7a0 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf0b7fe640 1 -- 172.21.15.137:0/3340539417 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c005020 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf0b7fe640 1 -- 172.21.15.137:0/3340539417 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0116d0 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.321+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a5080 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.322+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf14101200 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.324+0000 7fcf0b7fe640 1 -- 172.21.15.137:0/3340539417 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcf0c0051d0 con 0x7fcf14102ef0 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.324+0000 7fcf0b7fe640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcf000a2b80 0x7fcf000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.324+0000 7fcf1259c640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcf000a2b80 0x7fcf000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.324+0000 7fcf0b7fe640 1 -- 172.21.15.137:0/3340539417 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fcf0c0cf1a0 con 0x7fcf14102ef0 2024-09-12T12:10:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.325+0000 7fcf1259c640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcf000a2b80 0x7fcf000a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf04000c00 tx=0x7fcf04004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.327+0000 7fcf0b7fe640 1 -- 172.21.15.137:0/3340539417 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf0c016030 con 0x7fcf14102ef0 2024-09-12T12:10:25.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.492+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf1410a0a0 con 0x7fcf14102ef0 2024-09-12T12:10:25.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.492+0000 7fcf0b7fe640 1 -- 172.21.15.137:0/3340539417 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fcf0c09c5a0 con 0x7fcf14102ef0 2024-09-12T12:10:25.493 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-09-12T12:10:25.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcf000a2b80 msgr2=0x7fcf000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcf000a2b80 0x7fcf000a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf04000c00 tx=0x7fcf04004100 comp rx=0 tx=0).stop 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14102ef0 msgr2=0x7fcf1419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14102ef0 0x7fcf1419cdd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00eaa0 tx=0x7fcf0c00eea0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 shutdown_connections 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcf000a2b80 0x7fcf000a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf14102ef0 0x7fcf1419cdd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf14102570 0x7fcf14198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 --2- 172.21.15.137:0/3340539417 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf14100e20 0x7fcf14198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 >> 172.21.15.137:0/3340539417 conn(0x7fcf140fc7d0 msgr2=0x7fcf1406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 shutdown_connections 2024-09-12T12:10:25.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:25.495+0000 7fcf18fbe640 1 -- 172.21.15.137:0/3340539417 wait complete. 2024-09-12T12:10:26.182 INFO:tasks.cephadm:Adding mds.a on smithi137 2024-09-12T12:10:26.182 INFO:tasks.cephadm:Adding mds.d on smithi137 2024-09-12T12:10:26.183 INFO:tasks.cephadm:Adding mds.g on smithi137 2024-09-12T12:10:26.183 INFO:tasks.cephadm:Adding mds.j on smithi137 2024-09-12T12:10:26.183 INFO:tasks.cephadm:Adding mds.b on smithi171 2024-09-12T12:10:26.183 INFO:tasks.cephadm:Adding mds.e on smithi171 2024-09-12T12:10:26.183 INFO:tasks.cephadm:Adding mds.h on smithi171 2024-09-12T12:10:26.183 INFO:tasks.cephadm:Adding mds.k on smithi171 2024-09-12T12:10:26.184 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-09-12T12:10:26.184 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-09-12T12:10:26.184 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-09-12T12:10:26.184 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-09-12T12:10:26.184 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-09-12T12:10:26.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:26 smithi137 ceph-mon[30950]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:26.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:26 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3340539417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T12:10:26.432 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:10:26.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:26 smithi171 ceph-mon[35098]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:26.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:26 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3340539417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T12:10:26.553 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:26 smithi189 ceph-mon[35475]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:26.553 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:26 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3340539417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 -- 172.21.15.189:0/2515200328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98101d10 msgr2=0x7f9c98102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/2515200328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98101d10 0x7f9c98102190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009900 tx=0x7f9c9402a410 comp rx=0 tx=0).stop 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 -- 172.21.15.189:0/2515200328 shutdown_connections 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/2515200328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c981026d0 0x7f9c98108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/2515200328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98101d10 0x7f9c98102190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/2515200328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c98100b10 0x7f9c98100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 -- 172.21.15.189:0/2515200328 >> 172.21.15.189:0/2515200328 conn(0x7f9c980fc2a0 msgr2=0x7f9c980fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 -- 172.21.15.189:0/2515200328 shutdown_connections 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.932+0000 7f9c9f51a640 1 -- 172.21.15.189:0/2515200328 wait complete. 2024-09-12T12:10:26.975 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 Processor -- start 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 -- start start 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98100b10 0x7f9c98196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c98101d10 0x7f9c98197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c981026d0 0x7f9c9819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9810c790 con 0x7f9c981026d0 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9810c610 con 0x7f9c98101d10 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9f51a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9810c910 con 0x7f9c98100b10 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9d2b6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98100b10 0x7f9c98196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9d2b6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98100b10 0x7f9c98196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:40180/0 (socket says 172.21.15.189:40180) 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9d2b6640 1 -- 172.21.15.189:0/4153783139 learned_addr learned my addr 172.21.15.189:0/4153783139 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.933+0000 7f9c9dab7640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c981026d0 0x7f9c9819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9cab5640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c98101d10 0x7f9c98197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9d2b6640 1 -- 172.21.15.189:0/4153783139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c98101d10 msgr2=0x7f9c98197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:26.976 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9d2b6640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c98101d10 0x7f9c98197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9d2b6640 1 -- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c981026d0 msgr2=0x7f9c9819e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9d2b6640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c981026d0 0x7f9c9819e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9d2b6640 1 -- 172.21.15.189:0/4153783139 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9819ea40 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9dab7640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c981026d0 0x7f9c9819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9d2b6640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98100b10 0x7f9c98196e10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009970 tx=0x7f9c8c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c8a7fc640 1 -- 172.21.15.189:0/4153783139 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00d030 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c8a7fc640 1 -- 172.21.15.189:0/4153783139 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c8c007d40 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9806a7f0 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c8a7fc640 1 -- 172.21.15.189:0/4153783139 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00cc50 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.934+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9806ad50 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.935+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c98107980 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.937+0000 7f9c8a7fc640 1 -- 172.21.15.189:0/4153783139 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9c8c007860 con 0x7f9c98100b10 2024-09-12T12:10:26.977 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.937+0000 7f9c8a7fc640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c840a2c30 0x7f9c840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:26.978 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.938+0000 7f9c8a7fc640 1 -- 172.21.15.189:0/4153783139 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f9c8c0d0610 con 0x7f9c98100b10 2024-09-12T12:10:26.978 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.938+0000 7f9c9cab5640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c840a2c30 0x7f9c840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:26.978 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.938+0000 7f9c9cab5640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c840a2c30 0x7f9c840a50f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c940315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:26.978 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:26.941+0000 7f9c8a7fc640 1 -- 172.21.15.189:0/4153783139 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c8c09da10 con 0x7f9c98100b10 2024-09-12T12:10:27.120 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.120+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) -- 0x7f9c9806b110 con 0x7f9c840a2c30 2024-09-12T12:10:27.127 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.127+0000 7f9c8a7fc640 1 -- 172.21.15.189:0/4153783139 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9c9806b110 con 0x7f9c840a2c30 2024-09-12T12:10:27.127 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-09-12T12:10:27.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c840a2c30 msgr2=0x7f9c840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:27.129 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c840a2c30 0x7f9c840a50f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c940315d0 comp rx=0 tx=0).stop 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98100b10 msgr2=0x7f9c98196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98100b10 0x7f9c98196e10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009970 tx=0x7f9c8c009e30 comp rx=0 tx=0).stop 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 shutdown_connections 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c840a2c30 0x7f9c840a50f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c981026d0 0x7f9c9819e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c98101d10 0x7f9c98197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 --2- 172.21.15.189:0/4153783139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c98100b10 0x7f9c98196e10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 >> 172.21.15.189:0/4153783139 conn(0x7f9c980fc2a0 msgr2=0x7f9c981050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 shutdown_connections 2024-09-12T12:10:27.130 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-12T12:10:27.129+0000 7f9c9f51a640 1 -- 172.21.15.189:0/4153783139 wait complete. 2024-09-12T12:10:27.973 DEBUG:teuthology.orchestra.run.smithi137:mds.a> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T12:10:27.976 DEBUG:teuthology.orchestra.run.smithi137:mds.d> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T12:10:28.018 DEBUG:teuthology.orchestra.run.smithi137:mds.g> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T12:10:28.062 DEBUG:teuthology.orchestra.run.smithi137:mds.j> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T12:10:28.107 DEBUG:teuthology.orchestra.run.smithi171:mds.b> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T12:10:28.111 DEBUG:teuthology.orchestra.run.smithi171:mds.e> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T12:10:28.155 DEBUG:teuthology.orchestra.run.smithi171:mds.h> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T12:10:28.196 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: Saving service mds.all spec with placement smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:28.197 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:28 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:28.199 DEBUG:teuthology.orchestra.run.smithi171:mds.k> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T12:10:28.242 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T12:10:28.245 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T12:10:28.288 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T12:10:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:10:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: Saving service mds.all spec with placement smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-12T12:10:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:28.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:28.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:28 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:28.331 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T12:10:28.373 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T12:10:28.373 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: Saving service mds.all spec with placement smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:10:28.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:28.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:28.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:28.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:28 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:28.615 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:29.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 -- 172.21.15.137:0/3804857407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400102ef0 msgr2=0x7f940010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:29.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 --2- 172.21.15.137:0/3804857407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400102ef0 0x7f940010da40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009900 tx=0x7f93e802a480 comp rx=0 tx=0).stop 2024-09-12T12:10:29.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 -- 172.21.15.137:0/3804857407 shutdown_connections 2024-09-12T12:10:29.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 --2- 172.21.15.137:0/3804857407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400102ef0 0x7f940010da40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 --2- 172.21.15.137:0/3804857407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9400102570 0x7f94001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 --2- 172.21.15.137:0/3804857407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9400100e20 0x7f9400101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 -- 172.21.15.137:0/3804857407 >> 172.21.15.137:0/3804857407 conn(0x7f94000fc7d0 msgr2=0x7f94000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 -- 172.21.15.137:0/3804857407 shutdown_connections 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.058+0000 7f9406b82640 1 -- 172.21.15.137:0/3804857407 wait complete. 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 Processor -- start 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 -- start start 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400100e20 0x7f9400199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9400102570 0x7f94001998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9400102ef0 0x7f94001a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9400110f80 con 0x7f9400100e20 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9400110e00 con 0x7f9400102570 2024-09-12T12:10:29.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f9406b82640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9400111100 con 0x7f9400102ef0 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400100e20 0x7f9400199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400100e20 0x7f9400199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41174/0 (socket says 172.21.15.137:41174) 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 -- 172.21.15.137:0/2886044428 learned_addr learned my addr 172.21.15.137:0/2886044428 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940511f640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9400102ef0 0x7f94001a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f93fffff640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9400102570 0x7f94001998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 -- 172.21.15.137:0/2886044428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9400102ef0 msgr2=0x7f94001a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9400102ef0 0x7f94001a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 -- 172.21.15.137:0/2886044428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9400102570 msgr2=0x7f94001998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9400102570 0x7f94001998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 -- 172.21.15.137:0/2886044428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94001a4930 con 0x7f9400100e20 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f93fffff640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9400102570 0x7f94001998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:10:29.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940511f640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9400102ef0 0x7f94001a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.059+0000 7f940491e640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400100e20 0x7f9400199370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93f800d950 tx=0x7f93f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.060+0000 7f93fdffb640 1 -- 172.21.15.137:0/2886044428 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8017070 con 0x7f9400100e20 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.060+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94001a4b00 con 0x7f9400100e20 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.060+0000 7f93fdffb640 1 -- 172.21.15.137:0/2886044428 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8009c30 con 0x7f9400100e20 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.060+0000 7f93fdffb640 1 -- 172.21.15.137:0/2886044428 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f800f6b0 con 0x7f9400100e20 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.060+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94001a5010 con 0x7f9400100e20 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.061+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9400101200 con 0x7f9400100e20 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.062+0000 7f93fdffb640 1 -- 172.21.15.137:0/2886044428 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f93f80109d0 con 0x7f9400100e20 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.062+0000 7f93fdffb640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93ec0a2b80 0x7f93ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:29.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.063+0000 7f93fffff640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93ec0a2b80 0x7f93ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:29.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.063+0000 7f93fdffb640 1 -- 172.21.15.137:0/2886044428 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f93f80ce940 con 0x7f9400100e20 2024-09-12T12:10:29.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.063+0000 7f93fffff640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93ec0a2b80 0x7f93ec0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f93f00092f0 tx=0x7f93f0009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:29.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.067+0000 7f93fdffb640 1 -- 172.21.15.137:0/2886044428 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93f8009de0 con 0x7f9400100e20 2024-09-12T12:10:29.204 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:29 smithi189 ceph-mon[35475]: Deploying daemon mds.l on smithi189 2024-09-12T12:10:29.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.270+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94001a5210 con 0x7f9400100e20 2024-09-12T12:10:29.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.271+0000 7f93fdffb640 1 -- 172.21.15.137:0/2886044428 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f93f809bca0 con 0x7f9400100e20 2024-09-12T12:10:29.271 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:29.271 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":1,"btime":"2024-09-12T12:03:22:696421+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T12:10:29.272 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 1 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93ec0a2b80 msgr2=0x7f93ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93ec0a2b80 0x7f93ec0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f93f00092f0 tx=0x7f93f0009280 comp rx=0 tx=0).stop 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400100e20 msgr2=0x7f9400199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400100e20 0x7f9400199370 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93f800d950 tx=0x7f93f800de10 comp rx=0 tx=0).stop 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 shutdown_connections 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93ec0a2b80 0x7f93ec0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9400102ef0 0x7f94001a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9400102570 0x7f94001998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 --2- 172.21.15.137:0/2886044428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9400100e20 0x7f9400199370 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 >> 172.21.15.137:0/2886044428 conn(0x7f94000fc7d0 msgr2=0x7f9400106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:29.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 shutdown_connections 2024-09-12T12:10:29.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:29.273+0000 7f9406b82640 1 -- 172.21.15.137:0/2886044428 wait complete. 2024-09-12T12:10:29.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:29 smithi171 ceph-mon[35098]: Deploying daemon mds.l on smithi189 2024-09-12T12:10:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:29 smithi137 ceph-mon[30950]: Deploying daemon mds.l on smithi189 2024-09-12T12:10:29.841 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:29 smithi189 systemd[1]: Starting Ceph mds.l for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:30.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:30 smithi189 ceph-mon[35475]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:30.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:30 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2886044428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:30.151 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:29 smithi189 podman[65219]: 2024-09-12 12:10:29.789029034 +0000 UTC m=+0.015788185 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:30.151 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:29 smithi189 podman[65219]: 2024-09-12 12:10:29.915693608 +0000 UTC m=+0.142452798 container create f5d5c33814cce466709e5d6312e76167eb86b59da803a3838713b44781a70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:10:30.151 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:30 smithi189 podman[65219]: 2024-09-12 12:10:30.151664676 +0000 UTC m=+0.378423828 container init f5d5c33814cce466709e5d6312e76167eb86b59da803a3838713b44781a70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T12:10:30.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:30 smithi171 ceph-mon[35098]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:30.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:30 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2886044428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:30.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:30 smithi137 ceph-mon[30950]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:30.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:30 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2886044428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:30.591 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:30 smithi189 podman[65219]: 2024-09-12 12:10:30.154613141 +0000 UTC m=+0.381372291 container start f5d5c33814cce466709e5d6312e76167eb86b59da803a3838713b44781a70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T12:10:30.591 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:30 smithi189 ceph-mds[65238]: -- 172.21.15.189:0/2022121267 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ac8a9d860 con 0x563ac8a7a800 2024-09-12T12:10:30.591 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l[65234]: starting mds.l at 2024-09-12T12:10:30.591 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:30 smithi189 bash[65219]: f5d5c33814cce466709e5d6312e76167eb86b59da803a3838713b44781a70588 2024-09-12T12:10:30.591 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:10:30 smithi189 systemd[1]: Started Ceph mds.l for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6832/4107944214,v1:172.21.15.189:6833/4107944214] up:boot 2024-09-12T12:10:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: fsmap 1 up:standby 2024-09-12T12:10:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:10:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:31.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:31.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:31 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6832/4107944214,v1:172.21.15.189:6833/4107944214] up:boot 2024-09-12T12:10:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: fsmap 1 up:standby 2024-09-12T12:10:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:10:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:31.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:31.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:31 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6832/4107944214,v1:172.21.15.189:6833/4107944214] up:boot 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: fsmap 1 up:standby 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:31 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:32.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:32.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:32 smithi171 ceph-mon[35098]: Deploying daemon mds.k on smithi171 2024-09-12T12:10:32.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:32 smithi171 ceph-mon[35098]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:32.450 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 systemd[1]: Starting Ceph mds.k for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:32.485 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:32 smithi137 ceph-mon[30950]: Deploying daemon mds.k on smithi171 2024-09-12T12:10:32.486 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:32 smithi137 ceph-mon[30950]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:32.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:32 smithi189 ceph-mon[35475]: Deploying daemon mds.k on smithi171 2024-09-12T12:10:32.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:32 smithi189 ceph-mon[35475]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:32.789 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 podman[64020]: 2024-09-12 12:10:32.431404718 +0000 UTC m=+0.015773440 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:32.789 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 podman[64020]: 2024-09-12 12:10:32.542733045 +0000 UTC m=+0.127101758 container create dd5bf0673ac56a852eaedb63d13a40e6f1d4352b687e0ff431af370675d48d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T12:10:32.790 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 podman[64020]: 2024-09-12 12:10:32.786833574 +0000 UTC m=+0.371202295 container init dd5bf0673ac56a852eaedb63d13a40e6f1d4352b687e0ff431af370675d48d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:10:32.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.866+0000 7f4018f58640 1 -- 172.21.15.137:0/1515560309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014101de0 msgr2=0x7f40141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:32.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.866+0000 7f4018f58640 1 --2- 172.21.15.137:0/1515560309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014101de0 0x7f40141021c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4004009900 tx=0x7f400402a410 comp rx=0 tx=0).stop 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 -- 172.21.15.137:0/1515560309 shutdown_connections 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 --2- 172.21.15.137:0/1515560309 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014106f10 0x7f401410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 --2- 172.21.15.137:0/1515560309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4014102790 0x7f40141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 --2- 172.21.15.137:0/1515560309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014101de0 0x7f40141021c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 -- 172.21.15.137:0/1515560309 >> 172.21.15.137:0/1515560309 conn(0x7f40140fd8c0 msgr2=0x7f40140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 -- 172.21.15.137:0/1515560309 shutdown_connections 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 -- 172.21.15.137:0/1515560309 wait complete. 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 Processor -- start 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.867+0000 7f4018f58640 1 -- start start 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101de0 0x7f40140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4014102790 0x7f4014074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014106f10 0x7f4014074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:32.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4014111f70 con 0x7f4014106f10 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4014111df0 con 0x7f4014102790 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40141120f0 con 0x7f4014101de0 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014106f10 0x7f4014074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014106f10 0x7f4014074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41256/0 (socket says 172.21.15.137:41256) 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 -- 172.21.15.137:0/164794626 learned_addr learned my addr 172.21.15.137:0/164794626 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4011d9b640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4014102790 0x7f4014074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f401259c640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101de0 0x7f40140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 -- 172.21.15.137:0/164794626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101de0 msgr2=0x7f40140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101de0 0x7f40140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:32.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 -- 172.21.15.137:0/164794626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4014102790 msgr2=0x7f4014074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4014102790 0x7f4014074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 -- 172.21.15.137:0/164794626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40141a68d0 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4011d9b640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4014102790 0x7f4014074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4012d9d640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014106f10 0x7f4014074f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f400c00eaa0 tx=0x7f400c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f40037fe640 1 -- 172.21.15.137:0/164794626 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c00cdb0 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f40037fe640 1 -- 172.21.15.137:0/164794626 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f400c005020 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141a6bc0 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f40037fe640 1 -- 172.21.15.137:0/164794626 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c0116b0 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.868+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a7100 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.870+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40141021c0 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.870+0000 7f40037fe640 1 -- 172.21.15.137:0/164794626 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f400c005560 con 0x7f4014106f10 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.871+0000 7f40037fe640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3ffc0a2b80 0x7f3ffc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:32.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.871+0000 7f401259c640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3ffc0a2b80 0x7f3ffc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:32.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.871+0000 7f40037fe640 1 -- 172.21.15.137:0/164794626 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f400c0cee30 con 0x7f4014106f10 2024-09-12T12:10:32.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.872+0000 7f401259c640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3ffc0a2b80 0x7f3ffc0a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40040096c0 tx=0x7f4004000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:32.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:32.875+0000 7f40037fe640 1 -- 172.21.15.137:0/164794626 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f400c016030 con 0x7f4014106f10 2024-09-12T12:10:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.087+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f401410a0f0 con 0x7f4014106f10 2024-09-12T12:10:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.087+0000 7f40037fe640 1 -- 172.21.15.137:0/164794626 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f400c09c230 con 0x7f4014106f10 2024-09-12T12:10:33.088 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:33.088 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":2,"btime":"2024-09-12T12:10:30:182445+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T12:10:33.089 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 2 2024-09-12T12:10:33.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3ffc0a2b80 msgr2=0x7f3ffc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:33.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3ffc0a2b80 0x7f3ffc0a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f40040096c0 tx=0x7f4004000bc0 comp rx=0 tx=0).stop 2024-09-12T12:10:33.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014106f10 msgr2=0x7f4014074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:33.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014106f10 0x7f4014074f70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f400c00eaa0 tx=0x7f400c00ef60 comp rx=0 tx=0).stop 2024-09-12T12:10:33.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 shutdown_connections 2024-09-12T12:10:33.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3ffc0a2b80 0x7f3ffc0a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:33.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4014106f10 0x7f4014074f70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:33.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4014102790 0x7f4014074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:33.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 --2- 172.21.15.137:0/164794626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4014101de0 0x7f40140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:33.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 >> 172.21.15.137:0/164794626 conn(0x7f40140fd8c0 msgr2=0x7f40140fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:33.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 shutdown_connections 2024-09-12T12:10:33.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:33.090+0000 7f4018f58640 1 -- 172.21.15.137:0/164794626 wait complete. 2024-09-12T12:10:33.199 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 podman[64020]: 2024-09-12 12:10:32.789928439 +0000 UTC m=+0.374297153 container start dd5bf0673ac56a852eaedb63d13a40e6f1d4352b687e0ff431af370675d48d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T12:10:33.200 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 ceph-mds[64039]: -- 172.21.15.171:0/2199110781 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561967683860 con 0x561967661400 2024-09-12T12:10:33.200 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k[64035]: starting mds.k at 2024-09-12T12:10:33.200 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 bash[64020]: dd5bf0673ac56a852eaedb63d13a40e6f1d4352b687e0ff431af370675d48d9c 2024-09-12T12:10:33.200 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:10:32 smithi171 systemd[1]: Started Ceph mds.k for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: Deploying daemon mds.j on smithi137 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/164794626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:34.130 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:34.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.171:6832/1998961324,v1:172.21.15.171:6833/1998961324] up:boot 2024-09-12T12:10:34.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: fsmap 2 up:standby 2024-09-12T12:10:34.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:33 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: Deploying daemon mds.j on smithi137 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/164794626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.171:6832/1998961324,v1:172.21.15.171:6833/1998961324] up:boot 2024-09-12T12:10:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: fsmap 2 up:standby 2024-09-12T12:10:34.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:33 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T12:10:34.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: Deploying daemon mds.j on smithi137 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/164794626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.171:6832/1998961324,v1:172.21.15.171:6833/1998961324] up:boot 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: fsmap 2 up:standby 2024-09-12T12:10:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:33 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T12:10:35.891 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:36.040 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:10:36 smithi137 podman[67555]: 2024-09-12 12:10:36.040379175 +0000 UTC m=+0.131498903 container create 19c51bf267dd5acb66926786cf7beceffe311aa04fdc75c32780cccbc9e0764a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T12:10:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:36 smithi137 ceph-mon[30950]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:36.300 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:10:36 smithi137 podman[67555]: 2024-09-12 12:10:36.251375672 +0000 UTC m=+0.342495400 container init 19c51bf267dd5acb66926786cf7beceffe311aa04fdc75c32780cccbc9e0764a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T12:10:36.300 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:10:36 smithi137 podman[67555]: 2024-09-12 12:10:36.255074877 +0000 UTC m=+0.346194607 container start 19c51bf267dd5acb66926786cf7beceffe311aa04fdc75c32780cccbc9e0764a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T12:10:36.300 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:10:36 smithi137 ceph-mds[67608]: -- 172.21.15.137:0/2323596420 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b776a3f860 con 0x55b776a1c800 2024-09-12T12:10:36.300 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:10:36 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j[67598]: starting mds.j at 2024-09-12T12:10:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:36.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:36 smithi171 ceph-mon[35098]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:36.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:36 smithi189 ceph-mon[35475]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:36.658 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:10:36 smithi137 bash[67555]: 19c51bf267dd5acb66926786cf7beceffe311aa04fdc75c32780cccbc9e0764a 2024-09-12T12:10:36.658 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:10:36 smithi137 systemd[1]: Started Ceph mds.j for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:36.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1986045767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 msgr2=0x7f9b04103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1986045767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b04103180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009900 tx=0x7f9af802a480 comp rx=0 tx=0).stop 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1986045767 shutdown_connections 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1986045767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b0410c180 0x7f9b0410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1986045767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b041036c0 0x7f9b0410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1986045767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b04103180 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1986045767 >> 172.21.15.137:0/1986045767 conn(0x7f9b040fc7a0 msgr2=0x7f9b040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1986045767 shutdown_connections 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1986045767 wait complete. 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 Processor -- start 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.903+0000 7f9b0aa65640 1 -- start start 2024-09-12T12:10:36.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b0aa65640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b041984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b0aa65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b041036c0 0x7f9b04198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b0aa65640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b0410c180 0x7f9b0419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b0aa65640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04111f60 con 0x7f9b04102da0 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b0aa65640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b04111de0 con 0x7f9b0410c180 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b0aa65640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9b041120e0 con 0x7f9b041036c0 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b041984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b041984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41296/0 (socket says 172.21.15.137:41296) 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 -- 172.21.15.137:0/1273862051 learned_addr learned my addr 172.21.15.137:0/1273862051 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b09002640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b0410c180 0x7f9b0419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b03fff640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b041036c0 0x7f9b04198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 -- 172.21.15.137:0/1273862051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b041036c0 msgr2=0x7f9b04198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:36.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b041036c0 0x7f9b04198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 -- 172.21.15.137:0/1273862051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b0410c180 msgr2=0x7f9b0419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b0410c180 0x7f9b0419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 -- 172.21.15.137:0/1273862051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0419d490 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b09002640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b0410c180 0x7f9b0419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b08801640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b041984e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9af8006010 tx=0x7f9af8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b01ffb640 1 -- 172.21.15.137:0/1273862051 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af803f030 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b01ffb640 1 -- 172.21.15.137:0/1273862051 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af8004440 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0419d720 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.904+0000 7f9b01ffb640 1 -- 172.21.15.137:0/1273862051 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8035070 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.905+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b041a5060 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.905+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b04103180 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.907+0000 7f9b01ffb640 1 -- 172.21.15.137:0/1273862051 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9af8042030 con 0x7f9b04102da0 2024-09-12T12:10:36.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.907+0000 7f9b01ffb640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9af40a2c00 0x7f9af40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:36.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.907+0000 7f9b03fff640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9af40a2c00 0x7f9af40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:36.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.907+0000 7f9b01ffb640 1 -- 172.21.15.137:0/1273862051 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f9af80eefd0 con 0x7f9b04102da0 2024-09-12T12:10:36.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.908+0000 7f9b03fff640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9af40a2c00 0x7f9af40a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b040fedc0 tx=0x7f9af0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:36.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:36.911+0000 7f9b01ffb640 1 -- 172.21.15.137:0/1273862051 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9af80bc3d0 con 0x7f9b04102da0 2024-09-12T12:10:37.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.120+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b04107220 con 0x7f9b04102da0 2024-09-12T12:10:37.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.120+0000 7f9b01ffb640 1 -- 172.21.15.137:0/1273862051 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f9af80bbcd0 con 0x7f9b04102da0 2024-09-12T12:10:37.121 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:37.121 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":3,"btime":"2024-09-12T12:10:33:186325+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T12:10:37.122 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 3 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9af40a2c00 msgr2=0x7f9af40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9af40a2c00 0x7f9af40a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9b040fedc0 tx=0x7f9af0005f90 comp rx=0 tx=0).stop 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 msgr2=0x7f9b041984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b041984e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9af8006010 tx=0x7f9af8004040 comp rx=0 tx=0).stop 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 shutdown_connections 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9af40a2c00 0x7f9af40a50c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b0410c180 0x7f9b0419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:37.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b041036c0 0x7f9b04198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:37.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 --2- 172.21.15.137:0/1273862051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b04102da0 0x7f9b041984e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:37.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 >> 172.21.15.137:0/1273862051 conn(0x7f9b040fc7a0 msgr2=0x7f9b040fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:37.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 shutdown_connections 2024-09-12T12:10:37.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:37.123+0000 7f9b0aa65640 1 -- 172.21.15.137:0/1273862051 wait complete. 2024-09-12T12:10:37.501 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.501 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.501 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.501 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:37.501 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:37.501 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:37.501 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: Deploying daemon mds.d on smithi137 2024-09-12T12:10:37.502 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:37.502 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1273862051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:37.502 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.137:6834/2258495281,v1:172.21.15.137:6835/2258495281] up:boot 2024-09-12T12:10:37.502 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: fsmap 3 up:standby 2024-09-12T12:10:37.502 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:37 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: Deploying daemon mds.d on smithi137 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1273862051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.137:6834/2258495281,v1:172.21.15.137:6835/2258495281] up:boot 2024-09-12T12:10:37.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: fsmap 3 up:standby 2024-09-12T12:10:37.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:37 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: Deploying daemon mds.d on smithi137 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1273862051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:37.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.137:6834/2258495281,v1:172.21.15.137:6835/2258495281] up:boot 2024-09-12T12:10:37.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: fsmap 3 up:standby 2024-09-12T12:10:37.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:37 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T12:10:39.549 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 systemd[1]: Starting Ceph mds.d for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:39.916 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 podman[67960]: 2024-09-12 12:10:39.550400663 +0000 UTC m=+0.016218056 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:39.917 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 podman[67960]: 2024-09-12 12:10:39.654544997 +0000 UTC m=+0.120362386 container create 4d9bdea95fb4794a1b531e3f4bd7b15719902d222063a5c46488cc286f6f10b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T12:10:39.917 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 podman[67960]: 2024-09-12 12:10:39.890781445 +0000 UTC m=+0.356598834 container init 4d9bdea95fb4794a1b531e3f4bd7b15719902d222063a5c46488cc286f6f10b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:10:39.917 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 podman[67960]: 2024-09-12 12:10:39.893595314 +0000 UTC m=+0.359412713 container start 4d9bdea95fb4794a1b531e3f4bd7b15719902d222063a5c46488cc286f6f10b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:10:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:40 smithi137 ceph-mon[30950]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:40 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:40.300 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 ceph-mds[67979]: -- 172.21.15.137:0/1913905352 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f69253f860 con 0x55f69251d400 2024-09-12T12:10:40.300 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d[67975]: starting mds.d at 2024-09-12T12:10:40.300 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 bash[67960]: 4d9bdea95fb4794a1b531e3f4bd7b15719902d222063a5c46488cc286f6f10b2 2024-09-12T12:10:40.300 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:10:39 smithi137 systemd[1]: Started Ceph mds.d for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:40 smithi171 ceph-mon[35098]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:40 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:40.490 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:40 smithi189 ceph-mon[35475]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:40.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:40 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:40.739 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 -- 172.21.15.137:0/3220009191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 msgr2=0x7fe1a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/3220009191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01021c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe198009900 tx=0x7fe19802a410 comp rx=0 tx=0).stop 2024-09-12T12:10:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 -- 172.21.15.137:0/3220009191 shutdown_connections 2024-09-12T12:10:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/3220009191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a0106f10 0x7fe1a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/3220009191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a0102790 0x7fe1a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/3220009191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01021c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 -- 172.21.15.137:0/3220009191 >> 172.21.15.137:0/3220009191 conn(0x7fe1a00fd8c0 msgr2=0x7fe1a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 -- 172.21.15.137:0/3220009191 shutdown_connections 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.208+0000 7fe1a83a6640 1 -- 172.21.15.137:0/3220009191 wait complete. 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 Processor -- start 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 -- start start 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a0102790 0x7fe1a0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a0106f10 0x7fe1a019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0112010 con 0x7fe1a0101de0 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0111e90 con 0x7fe1a0102790 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a83a6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a0112190 con 0x7fe1a0106f10 2024-09-12T12:10:41.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55902/0 (socket says 172.21.15.137:55902) 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 -- 172.21.15.137:0/4085466212 learned_addr learned my addr 172.21.15.137:0/4085466212 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a5941640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a0102790 0x7fe1a0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 -- 172.21.15.137:0/4085466212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a0106f10 msgr2=0x7fe1a019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a0106f10 0x7fe1a019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 -- 172.21.15.137:0/4085466212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a0102790 msgr2=0x7fe1a0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a0102790 0x7fe1a0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a6142640 1 -- 172.21.15.137:0/4085466212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a019d560 con 0x7fe1a0101de0 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.209+0000 7fe1a5941640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a0102790 0x7fe1a0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.210+0000 7fe1a6142640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01985b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe198005ab0 tx=0x7fe1980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.210+0000 7fe19f7fe640 1 -- 172.21.15.137:0/4085466212 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19803f030 con 0x7fe1a0101de0 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.210+0000 7fe19f7fe640 1 -- 172.21.15.137:0/4085466212 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe198003ce0 con 0x7fe1a0101de0 2024-09-12T12:10:41.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.210+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a019d7f0 con 0x7fe1a0101de0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.210+0000 7fe19f7fe640 1 -- 172.21.15.137:0/4085466212 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe198035070 con 0x7fe1a0101de0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.210+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a01a5130 con 0x7fe1a0101de0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.211+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a01032c0 con 0x7fe1a0101de0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.212+0000 7fe19f7fe640 1 -- 172.21.15.137:0/4085466212 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe198042030 con 0x7fe1a0101de0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.212+0000 7fe19f7fe640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe1880a2b80 0x7fe1880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.213+0000 7fe1a5941640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe1880a2b80 0x7fe1880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.213+0000 7fe19f7fe640 1 -- 172.21.15.137:0/4085466212 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe1980ef540 con 0x7fe1a0101de0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.213+0000 7fe1a5941640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe1880a2b80 0x7fe1880a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe194006fd0 tx=0x7fe194008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:41.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.217+0000 7fe19f7fe640 1 -- 172.21.15.137:0/4085466212 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1980bc940 con 0x7fe1a0101de0 2024-09-12T12:10:41.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.419+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1a010a0f0 con 0x7fe1a0101de0 2024-09-12T12:10:41.420 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:41 smithi137 ceph-mon[30950]: Deploying daemon mds.e on smithi171 2024-09-12T12:10:41.420 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:41 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.137:6836/1845807788,v1:172.21.15.137:6837/1845807788] up:boot 2024-09-12T12:10:41.421 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:41 smithi137 ceph-mon[30950]: fsmap 4 up:standby 2024-09-12T12:10:41.421 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:41 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T12:10:41.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.420+0000 7fe19f7fe640 1 -- 172.21.15.137:0/4085466212 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7fe1980bc240 con 0x7fe1a0101de0 2024-09-12T12:10:41.421 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:41.421 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":5,"btime":"2024-09-12T12:10:40:155509+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T12:10:41.422 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 5 2024-09-12T12:10:41.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe1880a2b80 msgr2=0x7fe1880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:41.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe1880a2b80 0x7fe1880a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe194006fd0 tx=0x7fe194008040 comp rx=0 tx=0).stop 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 msgr2=0x7fe1a01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01985b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe198005ab0 tx=0x7fe1980042f0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 shutdown_connections 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe1880a2b80 0x7fe1880a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1a0106f10 0x7fe1a019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a0102790 0x7fe1a0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 --2- 172.21.15.137:0/4085466212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1a0101de0 0x7fe1a01985b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 >> 172.21.15.137:0/4085466212 conn(0x7fe1a00fd8c0 msgr2=0x7fe1a00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 shutdown_connections 2024-09-12T12:10:41.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:41.423+0000 7fe1a83a6640 1 -- 172.21.15.137:0/4085466212 wait complete. 2024-09-12T12:10:41.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:41 smithi171 ceph-mon[35098]: Deploying daemon mds.e on smithi171 2024-09-12T12:10:41.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:41 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.137:6836/1845807788,v1:172.21.15.137:6837/1845807788] up:boot 2024-09-12T12:10:41.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:41 smithi171 ceph-mon[35098]: fsmap 4 up:standby 2024-09-12T12:10:41.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:41 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T12:10:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:41 smithi189 ceph-mon[35475]: Deploying daemon mds.e on smithi171 2024-09-12T12:10:41.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:41 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.137:6836/1845807788,v1:172.21.15.137:6837/1845807788] up:boot 2024-09-12T12:10:41.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:41 smithi189 ceph-mon[35475]: fsmap 4 up:standby 2024-09-12T12:10:41.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:41 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T12:10:42.169 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 systemd[1]: Starting Ceph mds.e for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:42.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:42 smithi171 ceph-mon[35098]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:42.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:42 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4085466212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:42.450 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 podman[64330]: 2024-09-12 12:10:42.196701848 +0000 UTC m=+0.015851615 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:42.450 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 podman[64330]: 2024-09-12 12:10:42.314580433 +0000 UTC m=+0.133730184 container create 204a2a4d658528ef1229ed0c239515d9b5bcf20f5bb7b4aee8d3bff136263e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-09-12T12:10:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:42 smithi137 ceph-mon[30950]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:42 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4085466212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:42.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:42 smithi189 ceph-mon[35475]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:42.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:42 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4085466212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:42.950 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 podman[64330]: 2024-09-12 12:10:42.59229563 +0000 UTC m=+0.411445383 container init 204a2a4d658528ef1229ed0c239515d9b5bcf20f5bb7b4aee8d3bff136263e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T12:10:42.950 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 podman[64330]: 2024-09-12 12:10:42.595438088 +0000 UTC m=+0.414587846 container start 204a2a4d658528ef1229ed0c239515d9b5bcf20f5bb7b4aee8d3bff136263e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T12:10:42.950 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 ceph-mds[64349]: -- 172.21.15.171:0/3452154776 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc7fbedc20 con 0x55cc7fbcb400 2024-09-12T12:10:42.950 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e[64345]: starting mds.e at 2024-09-12T12:10:42.950 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 bash[64330]: 204a2a4d658528ef1229ed0c239515d9b5bcf20f5bb7b4aee8d3bff136263e63 2024-09-12T12:10:42.950 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:10:42 smithi171 systemd[1]: Started Ceph mds.e for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:44.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:44.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:44.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:44.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: Deploying daemon mds.f on smithi189 2024-09-12T12:10:44.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:44.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.171:6834/971785585,v1:172.21.15.171:6835/971785585] up:boot 2024-09-12T12:10:44.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: fsmap 5 up:standby 2024-09-12T12:10:44.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:43 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T12:10:44.088 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: Deploying daemon mds.f on smithi189 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.171:6834/971785585,v1:172.21.15.171:6835/971785585] up:boot 2024-09-12T12:10:44.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: fsmap 5 up:standby 2024-09-12T12:10:44.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:43 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: Deploying daemon mds.f on smithi189 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.171:6834/971785585,v1:172.21.15.171:6835/971785585] up:boot 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: fsmap 5 up:standby 2024-09-12T12:10:44.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:43 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T12:10:44.350 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:44.841 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:44 smithi189 systemd[1]: Starting Ceph mds.f for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:44.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.825+0000 7fa56af06640 1 -- 172.21.15.137:0/1933089988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 msgr2=0x7fa5641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.825+0000 7fa56af06640 1 --2- 172.21.15.137:0/1933089988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641021c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009900 tx=0x7fa55c02a410 comp rx=0 tx=0).stop 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 -- 172.21.15.137:0/1933089988 shutdown_connections 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 --2- 172.21.15.137:0/1933089988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa564106f10 0x7fa56410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 --2- 172.21.15.137:0/1933089988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa564102790 0x7fa5641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 --2- 172.21.15.137:0/1933089988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641021c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 -- 172.21.15.137:0/1933089988 >> 172.21.15.137:0/1933089988 conn(0x7fa5640fd8c0 msgr2=0x7fa5640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 -- 172.21.15.137:0/1933089988 shutdown_connections 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 -- 172.21.15.137:0/1933089988 wait complete. 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 Processor -- start 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.826+0000 7fa56af06640 1 -- start start 2024-09-12T12:10:44.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa56af06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa56af06640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa564102790 0x7fa564198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa56af06640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa564106f10 0x7fa56419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa56af06640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa564111f70 con 0x7fa564101de0 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa56af06640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa564111df0 con 0x7fa564106f10 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa56af06640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641120f0 con 0x7fa564102790 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55922/0 (socket says 172.21.15.137:55922) 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 -- 172.21.15.137:0/3610136874 learned_addr learned my addr 172.21.15.137:0/3610136874 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa5694a3640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa564106f10 0x7fa56419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa563fff640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa564102790 0x7fa564198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 -- 172.21.15.137:0/3610136874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa564102790 msgr2=0x7fa564198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa564102790 0x7fa564198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:44.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 -- 172.21.15.137:0/3610136874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa564106f10 msgr2=0x7fa56419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa564106f10 0x7fa56419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 -- 172.21.15.137:0/3610136874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56419d380 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa563fff640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa564102790 0x7fa564198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa5694a3640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa564106f10 0x7fa56419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa568ca2640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641983d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006010 tx=0x7fa55c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa561ffb640 1 -- 172.21.15.137:0/3610136874 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c03f030 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa561ffb640 1 -- 172.21.15.137:0/3610136874 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa55c003ce0 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56419d610 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.827+0000 7fa561ffb640 1 -- 172.21.15.137:0/3610136874 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c035070 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.828+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5641a4f50 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.829+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5641021c0 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.830+0000 7fa561ffb640 1 -- 172.21.15.137:0/3610136874 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa55c042030 con 0x7fa564101de0 2024-09-12T12:10:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.830+0000 7fa561ffb640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5500a2c00 0x7fa5500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.831+0000 7fa563fff640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5500a2c00 0x7fa5500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.831+0000 7fa561ffb640 1 -- 172.21.15.137:0/3610136874 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa55c0ef430 con 0x7fa564101de0 2024-09-12T12:10:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.831+0000 7fa563fff640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5500a2c00 0x7fa5500a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa5640fdce0 tx=0x7fa5540093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:44.834+0000 7fa561ffb640 1 -- 172.21.15.137:0/3610136874 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa55c0bc8f0 con 0x7fa564101de0 2024-09-12T12:10:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.037+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa564199730 con 0x7fa564101de0 2024-09-12T12:10:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.037+0000 7fa561ffb640 1 -- 172.21.15.137:0/3610136874 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7fa55c0bc1f0 con 0x7fa564101de0 2024-09-12T12:10:45.038 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:45.038 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":6,"btime":"2024-09-12T12:10:43:169338+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T12:10:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 6 2024-09-12T12:10:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5500a2c00 msgr2=0x7fa5500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5500a2c00 0x7fa5500a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa5640fdce0 tx=0x7fa5540093e0 comp rx=0 tx=0).stop 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 msgr2=0x7fa5641983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641983d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa55c006010 tx=0x7fa55c0042f0 comp rx=0 tx=0).stop 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 shutdown_connections 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5500a2c00 0x7fa5500a50c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa564106f10 0x7fa56419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa564102790 0x7fa564198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 --2- 172.21.15.137:0/3610136874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa564101de0 0x7fa5641983d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 >> 172.21.15.137:0/3610136874 conn(0x7fa5640fd8c0 msgr2=0x7fa5640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 shutdown_connections 2024-09-12T12:10:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:45.040+0000 7fa56af06640 1 -- 172.21.15.137:0/3610136874 wait complete. 2024-09-12T12:10:45.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:45 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3610136874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:45.156 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:44 smithi189 podman[65521]: 2024-09-12 12:10:44.793093603 +0000 UTC m=+0.015531373 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:45.156 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:44 smithi189 podman[65521]: 2024-09-12 12:10:44.912090482 +0000 UTC m=+0.134528298 container create 621c43d9b50d547e2fe40dfeffe7df7efea64ff3df542c64ddb0e277761aed3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T12:10:45.156 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:45 smithi189 podman[65521]: 2024-09-12 12:10:45.156439847 +0000 UTC m=+0.378877621 container init 621c43d9b50d547e2fe40dfeffe7df7efea64ff3df542c64ddb0e277761aed3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T12:10:45.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:45 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3610136874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:45.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:45 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3610136874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:45.590 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:45 smithi189 podman[65521]: 2024-09-12 12:10:45.159632114 +0000 UTC m=+0.382069888 container start 621c43d9b50d547e2fe40dfeffe7df7efea64ff3df542c64ddb0e277761aed3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T12:10:45.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:45 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/513201130 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7a2495860 con 0x55e7a2472800 2024-09-12T12:10:45.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:45 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:10:45.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:45 smithi189 bash[65521]: 621c43d9b50d547e2fe40dfeffe7df7efea64ff3df542c64ddb0e277761aed3d 2024-09-12T12:10:45.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:10:45 smithi189 systemd[1]: Started Ceph mds.f for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:46.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:46 smithi189 ceph-mon[35475]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:46.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:46.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:46.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:46 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:46 smithi171 ceph-mon[35098]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:46.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:46 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:46 smithi137 ceph-mon[30950]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:46.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:46 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:47.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:47 smithi171 ceph-mon[35098]: Deploying daemon mds.c on smithi189 2024-09-12T12:10:47.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:47 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/2972240277,v1:172.21.15.189:6835/2972240277] up:boot 2024-09-12T12:10:47.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:47 smithi171 ceph-mon[35098]: fsmap 6 up:standby 2024-09-12T12:10:47.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:47 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:10:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:47 smithi137 ceph-mon[30950]: Deploying daemon mds.c on smithi189 2024-09-12T12:10:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:47 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/2972240277,v1:172.21.15.189:6835/2972240277] up:boot 2024-09-12T12:10:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:47 smithi137 ceph-mon[30950]: fsmap 6 up:standby 2024-09-12T12:10:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:47 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:10:47.555 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:47 smithi189 ceph-mon[35475]: Deploying daemon mds.c on smithi189 2024-09-12T12:10:47.555 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:47 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/2972240277,v1:172.21.15.189:6835/2972240277] up:boot 2024-09-12T12:10:47.555 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:47 smithi189 ceph-mon[35475]: fsmap 6 up:standby 2024-09-12T12:10:47.555 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:47 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:10:47.555 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 systemd[1]: Starting Ceph mds.c for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:47.727 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:47.829 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 podman[65822]: 2024-09-12 12:10:47.458196125 +0000 UTC m=+0.015901849 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:47.829 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 podman[65822]: 2024-09-12 12:10:47.58490544 +0000 UTC m=+0.142611219 container create 794fd1523170b41d17651c0fb6d273c6626d54ea027bd025db0e4486df28438b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T12:10:47.829 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 podman[65822]: 2024-09-12 12:10:47.829802766 +0000 UTC m=+0.387508491 container init 794fd1523170b41d17651c0fb6d273c6626d54ea027bd025db0e4486df28438b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:10:47.991 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:48.091 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 podman[65822]: 2024-09-12 12:10:47.832793614 +0000 UTC m=+0.390499343 container start 794fd1523170b41d17651c0fb6d273c6626d54ea027bd025db0e4486df28438b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T12:10:48.091 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 ceph-mds[65841]: -- 172.21.15.189:0/1773020929 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629dbcc9860 con 0x5629dbca6800 2024-09-12T12:10:48.091 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c[65837]: starting mds.c at 2024-09-12T12:10:48.091 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 bash[65822]: 794fd1523170b41d17651c0fb6d273c6626d54ea027bd025db0e4486df28438b 2024-09-12T12:10:48.092 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:10:47 smithi189 systemd[1]: Started Ceph mds.c for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:48.380 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:48 smithi137 ceph-mon[30950]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:48.380 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.380 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.380 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.380 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:48.381 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:48.381 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:48 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:48.429 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:48 smithi171 ceph-mon[35098]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:48.430 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.430 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.430 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.430 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:48.430 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:48.430 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:48 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:48.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 -- 172.21.15.137:0/1516748903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804100e20 msgr2=0x7f4804101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:48.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 --2- 172.21.15.137:0/1516748903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804100e20 0x7f4804101200 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009900 tx=0x7f47f802a480 comp rx=0 tx=0).stop 2024-09-12T12:10:48.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 -- 172.21.15.137:0/1516748903 shutdown_connections 2024-09-12T12:10:48.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 --2- 172.21.15.137:0/1516748903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4804102ef0 0x7f480410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 --2- 172.21.15.137:0/1516748903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4804102570 0x7f48041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 --2- 172.21.15.137:0/1516748903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804100e20 0x7f4804101200 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 -- 172.21.15.137:0/1516748903 >> 172.21.15.137:0/1516748903 conn(0x7f48040fc7d0 msgr2=0x7f48040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 -- 172.21.15.137:0/1516748903 shutdown_connections 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.463+0000 7f480b016640 1 -- 172.21.15.137:0/1516748903 wait complete. 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 Processor -- start 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 -- start start 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4804100e20 0x7f4804199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804102570 0x7f4804199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4804102ef0 0x7f48041a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4804110f80 con 0x7f4804102570 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4804110e00 con 0x7f4804102ef0 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f480b016640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4804111100 con 0x7f4804100e20 2024-09-12T12:10:48.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f4803fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804102570 0x7f4804199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f4803fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804102570 0x7f4804199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55938/0 (socket says 172.21.15.137:55938) 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f4803fff640 1 -- 172.21.15.137:0/2686589983 learned_addr learned my addr 172.21.15.137:0/2686589983 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f4808db2640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4804100e20 0x7f4804199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.464+0000 7f48095b3640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4804102ef0 0x7f48041a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4803fff640 1 -- 172.21.15.137:0/2686589983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4804100e20 msgr2=0x7f4804199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4803fff640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4804100e20 0x7f4804199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4803fff640 1 -- 172.21.15.137:0/2686589983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4804102ef0 msgr2=0x7f48041a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4803fff640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4804102ef0 0x7f48041a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4803fff640 1 -- 172.21.15.137:0/2686589983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48041a48f0 con 0x7f4804102570 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f48095b3640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4804102ef0 0x7f48041a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4803fff640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804102570 0x7f4804199870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47f000e9f0 tx=0x7f47f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4801ffb640 1 -- 172.21.15.137:0/2686589983 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f000cd80 con 0x7f4804102570 2024-09-12T12:10:48.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4804197120 con 0x7f4804102570 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4801ffb640 1 -- 172.21.15.137:0/2686589983 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47f0010910 con 0x7f4804102570 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f4801ffb640 1 -- 172.21.15.137:0/2686589983 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f00188f0 con 0x7f4804102570 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.465+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4804197610 con 0x7f4804102570 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.467+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4804101200 con 0x7f4804102570 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.467+0000 7f4801ffb640 1 -- 172.21.15.137:0/2686589983 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f47f0010ac0 con 0x7f4804102570 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.468+0000 7f4801ffb640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f47f40a2b80 0x7f47f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.469+0000 7f4808db2640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f47f40a2b80 0x7f47f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.469+0000 7f4801ffb640 1 -- 172.21.15.137:0/2686589983 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f47f00cef30 con 0x7f4804102570 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.469+0000 7f4808db2640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f47f40a2b80 0x7f47f40a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009230 tx=0x7f47f80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:48.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.472+0000 7f4801ffb640 1 -- 172.21.15.137:0/2686589983 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47f009bce0 con 0x7f4804102570 2024-09-12T12:10:48.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:48 smithi189 ceph-mon[35475]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:48.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:48 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:48.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.704+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4804109090 con 0x7f4804102570 2024-09-12T12:10:48.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.705+0000 7f4801ffb640 1 -- 172.21.15.137:0/2686589983 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7f47f009ffe0 con 0x7f4804102570 2024-09-12T12:10:48.705 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:48.706 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":8,"btime":"2024-09-12T12:10:48:098515+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T12:10:48.706 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 8 2024-09-12T12:10:48.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f47f40a2b80 msgr2=0x7f47f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:48.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f47f40a2b80 0x7f47f40a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009230 tx=0x7f47f80058d0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804102570 msgr2=0x7f4804199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804102570 0x7f4804199870 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f47f000e9f0 tx=0x7f47f000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 shutdown_connections 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f47f40a2b80 0x7f47f40a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4804102ef0 0x7f48041a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4804102570 0x7f4804199870 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 --2- 172.21.15.137:0/2686589983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4804100e20 0x7f4804199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 >> 172.21.15.137:0/2686589983 conn(0x7f48040fc7d0 msgr2=0x7f48040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 shutdown_connections 2024-09-12T12:10:48.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:48.707+0000 7f480b016640 1 -- 172.21.15.137:0/2686589983 wait complete. 2024-09-12T12:10:49.376 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:49 smithi137 ceph-mon[30950]: Deploying daemon mds.h on smithi171 2024-09-12T12:10:49.376 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:49 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] up:boot 2024-09-12T12:10:49.376 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:49 smithi137 ceph-mon[30950]: fsmap 7 up:standby 2024-09-12T12:10:49.376 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:49 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:10:49.376 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:49 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2686589983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:49.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:49 smithi171 ceph-mon[35098]: Deploying daemon mds.h on smithi171 2024-09-12T12:10:49.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:49 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] up:boot 2024-09-12T12:10:49.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:49 smithi171 ceph-mon[35098]: fsmap 7 up:standby 2024-09-12T12:10:49.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:49 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:10:49.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:49 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2686589983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:49 smithi189 ceph-mon[35475]: Deploying daemon mds.h on smithi171 2024-09-12T12:10:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:49 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] up:boot 2024-09-12T12:10:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:49 smithi189 ceph-mon[35475]: fsmap 7 up:standby 2024-09-12T12:10:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:49 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:10:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:49 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2686589983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:50.110 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 systemd[1]: Starting Ceph mds.h for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:50.366 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:50 smithi171 ceph-mon[35098]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:50.366 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 podman[64628]: 2024-09-12 12:10:50.146198358 +0000 UTC m=+0.016225034 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:50.366 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 podman[64628]: 2024-09-12 12:10:50.244643203 +0000 UTC m=+0.114669866 container create 143f0538919948cb8d5c2efc487745c7955a7c8560ef455b395ac8aefd016025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T12:10:50.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:50 smithi137 ceph-mon[30950]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:50.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:50 smithi189 ceph-mon[35475]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:50.950 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 podman[64628]: 2024-09-12 12:10:50.497403842 +0000 UTC m=+0.367430505 container init 143f0538919948cb8d5c2efc487745c7955a7c8560ef455b395ac8aefd016025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T12:10:50.950 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 podman[64628]: 2024-09-12 12:10:50.500255659 +0000 UTC m=+0.370282323 container start 143f0538919948cb8d5c2efc487745c7955a7c8560ef455b395ac8aefd016025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T12:10:50.950 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 ceph-mds[64647]: -- 172.21.15.171:0/1768468073 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639ab437860 con 0x5639ab414800 2024-09-12T12:10:50.950 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h[64643]: starting mds.h at 2024-09-12T12:10:50.950 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 bash[64628]: 143f0538919948cb8d5c2efc487745c7955a7c8560ef455b395ac8aefd016025 2024-09-12T12:10:50.950 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:10:50 smithi171 systemd[1]: Started Ceph mds.h for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:51.412 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:51.676 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:51.798 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:51.798 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:51.798 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:51.798 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: Deploying daemon mds.i on smithi189 2024-09-12T12:10:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.171:6836/1673181827,v1:172.21.15.171:6837/1673181827] up:boot 2024-09-12T12:10:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: fsmap 8 up:standby 2024-09-12T12:10:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:51 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:10:51.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: Deploying daemon mds.i on smithi189 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.171:6836/1673181827,v1:172.21.15.171:6837/1673181827] up:boot 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: fsmap 8 up:standby 2024-09-12T12:10:51.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:51 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:10:52.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: Deploying daemon mds.i on smithi189 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.171:6836/1673181827,v1:172.21.15.171:6837/1673181827] up:boot 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: fsmap 8 up:standby 2024-09-12T12:10:52.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:51 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:10:52.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 -- 172.21.15.137:0/3209577155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a10103660 msgr2=0x7f1a1010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:52.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 --2- 172.21.15.137:0/3209577155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a10103660 0x7f1a1010bb50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009900 tx=0x7f19fc02a480 comp rx=0 tx=0).stop 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 -- 172.21.15.137:0/3209577155 shutdown_connections 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 --2- 172.21.15.137:0/3209577155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1010c170 0x7f1a1010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 --2- 172.21.15.137:0/3209577155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a10103660 0x7f1a1010bb50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 --2- 172.21.15.137:0/3209577155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a10102d40 0x7f1a10103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 -- 172.21.15.137:0/3209577155 >> 172.21.15.137:0/3209577155 conn(0x7f1a100fc7a0 msgr2=0x7f1a100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 -- 172.21.15.137:0/3209577155 shutdown_connections 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.156+0000 7f1a16c95640 1 -- 172.21.15.137:0/3209577155 wait complete. 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 Processor -- start 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 -- start start 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a10102d40 0x7f1a101009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:52.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a10103660 0x7f1a10100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1010c170 0x7f1a10101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10111f70 con 0x7f1a1010c170 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1a10111df0 con 0x7f1a10103660 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a16c95640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1a101120f0 con 0x7f1a10102d40 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1010c170 0x7f1a10101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1010c170 0x7f1a10101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53802/0 (socket says 172.21.15.137:53802) 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 -- 172.21.15.137:0/52303780 learned_addr learned my addr 172.21.15.137:0/52303780 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a14a31640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a10102d40 0x7f1a101009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a0ffff640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a10103660 0x7f1a10100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 -- 172.21.15.137:0/52303780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a10102d40 msgr2=0x7f1a101009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a10102d40 0x7f1a101009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 -- 172.21.15.137:0/52303780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a10103660 msgr2=0x7f1a10100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a10103660 0x7f1a10100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 -- 172.21.15.137:0/52303780 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a101a68f0 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.157+0000 7f1a15232640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1010c170 0x7f1a10101420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800eaa0 tx=0x7f1a0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.158+0000 7f1a0dffb640 1 -- 172.21.15.137:0/52303780 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800cdb0 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.158+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a101a6be0 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.158+0000 7f1a0dffb640 1 -- 172.21.15.137:0/52303780 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a08005020 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.158+0000 7f1a0dffb640 1 -- 172.21.15.137:0/52303780 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a080116b0 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.158+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a101a70f0 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.159+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a10103120 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.160+0000 7f1a0dffb640 1 -- 172.21.15.137:0/52303780 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1a08005560 con 0x7f1a1010c170 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.160+0000 7f1a0dffb640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1a000a2b80 0x7f1a000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.160+0000 7f1a14a31640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1a000a2b80 0x7f1a000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:52.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.161+0000 7f1a0dffb640 1 -- 172.21.15.137:0/52303780 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f1a080cecf0 con 0x7f1a1010c170 2024-09-12T12:10:52.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.161+0000 7f1a14a31640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1a000a2b80 0x7f1a000a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400bbe0 tx=0x7f1a0400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:52.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.164+0000 7f1a0dffb640 1 -- 172.21.15.137:0/52303780 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a080d3070 con 0x7f1a1010c170 2024-09-12T12:10:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.367+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a101071e0 con 0x7f1a1010c170 2024-09-12T12:10:52.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.368+0000 7f1a0dffb640 1 -- 172.21.15.137:0/52303780 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7f1a08016030 con 0x7f1a1010c170 2024-09-12T12:10:52.369 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:52.369 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":9,"btime":"2024-09-12T12:10:51:109083+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T12:10:52.369 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 9 2024-09-12T12:10:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1a000a2b80 msgr2=0x7f1a000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1a000a2b80 0x7f1a000a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400bbe0 tx=0x7f1a0400b390 comp rx=0 tx=0).stop 2024-09-12T12:10:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1010c170 msgr2=0x7f1a10101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1010c170 0x7f1a10101420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800eaa0 tx=0x7f1a0800ef60 comp rx=0 tx=0).stop 2024-09-12T12:10:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 shutdown_connections 2024-09-12T12:10:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1a000a2b80 0x7f1a000a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a1010c170 0x7f1a10101420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a10103660 0x7f1a10100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 --2- 172.21.15.137:0/52303780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a10102d40 0x7f1a101009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.370+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 >> 172.21.15.137:0/52303780 conn(0x7f1a100fc7a0 msgr2=0x7f1a100fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.371+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 shutdown_connections 2024-09-12T12:10:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:52.371+0000 7f1a16c95640 1 -- 172.21.15.137:0/52303780 wait complete. 2024-09-12T12:10:52.831 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:52 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/52303780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:52.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:52 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/52303780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:53.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:52 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/52303780' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:53.590 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:10:53 smithi189 systemd[1]: Started Ceph mds.i for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: Deploying daemon mds.g on smithi137 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] up:boot 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: fsmap 9 up:standby 2024-09-12T12:10:54.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:54 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:10:54.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: Deploying daemon mds.g on smithi137 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] up:boot 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: fsmap 9 up:standby 2024-09-12T12:10:54.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:54 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:10:54.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:54.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: Deploying daemon mds.g on smithi137 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] up:boot 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: fsmap 9 up:standby 2024-09-12T12:10:54.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:54 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:10:55.063 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:55.299 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 systemd[1]: Starting Ceph mds.g for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:55.689 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 podman[68754]: 2024-09-12 12:10:55.334387874 +0000 UTC m=+0.016222042 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:55.689 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 podman[68754]: 2024-09-12 12:10:55.461939545 +0000 UTC m=+0.143773745 container create 04bed9820499b68a47d5e6fa6d77ad74e63e3bc5f7d134bcf162495a564f4409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T12:10:55.690 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 podman[68754]: 2024-09-12 12:10:55.689718142 +0000 UTC m=+0.371552304 container init 04bed9820499b68a47d5e6fa6d77ad74e63e3bc5f7d134bcf162495a564f4409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T12:10:55.852 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:10:55.975 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 podman[68754]: 2024-09-12 12:10:55.693234595 +0000 UTC m=+0.375068763 container start 04bed9820499b68a47d5e6fa6d77ad74e63e3bc5f7d134bcf162495a564f4409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T12:10:55.976 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 ceph-mds[68798]: -- 172.21.15.137:0/3106593599 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0dd695860 con 0x55f0dd673000 2024-09-12T12:10:55.976 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g[68782]: starting mds.g at 2024-09-12T12:10:55.976 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 bash[68754]: 04bed9820499b68a47d5e6fa6d77ad74e63e3bc5f7d134bcf162495a564f4409 2024-09-12T12:10:55.976 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:10:55 smithi137 systemd[1]: Started Ceph mds.g for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:56.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.374+0000 7fcc23e65640 1 -- 172.21.15.137:0/1337842421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c102790 msgr2=0x7fcc1c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:56.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.374+0000 7fcc23e65640 1 --2- 172.21.15.137:0/1337842421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c102790 0x7fcc1c1067e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009ca0 tx=0x7fcc1002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 -- 172.21.15.137:0/1337842421 shutdown_connections 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 --2- 172.21.15.137:0/1337842421 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c106f10 0x7fcc1c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 --2- 172.21.15.137:0/1337842421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c102790 0x7fcc1c1067e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 --2- 172.21.15.137:0/1337842421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1c101de0 0x7fcc1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 -- 172.21.15.137:0/1337842421 >> 172.21.15.137:0/1337842421 conn(0x7fcc1c0fd8c0 msgr2=0x7fcc1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 -- 172.21.15.137:0/1337842421 shutdown_connections 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 -- 172.21.15.137:0/1337842421 wait complete. 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 Processor -- start 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.375+0000 7fcc23e65640 1 -- start start 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c101de0 0x7fcc1c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1c102790 0x7fcc1c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:56.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c106f10 0x7fcc1c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c111fc0 con 0x7fcc1c101de0 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c111e40 con 0x7fcc1c106f10 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c112140 con 0x7fcc1c102790 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c101de0 0x7fcc1c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c101de0 0x7fcc1c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53836/0 (socket says 172.21.15.137:53836) 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 -- 172.21.15.137:0/4267938592 learned_addr learned my addr 172.21.15.137:0/4267938592 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21400640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1c102790 0x7fcc1c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc22402640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c106f10 0x7fcc1c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 -- 172.21.15.137:0/4267938592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1c102790 msgr2=0x7fcc1c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1c102790 0x7fcc1c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 -- 172.21.15.137:0/4267938592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c106f10 msgr2=0x7fcc1c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c106f10 0x7fcc1c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 -- 172.21.15.137:0/4267938592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c19d3e0 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc21c01640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c101de0 0x7fcc1c198430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c970 tx=0x7fcc0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc1affd640 1 -- 172.21.15.137:0/4267938592 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c007bb0 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc1affd640 1 -- 172.21.15.137:0/4267938592 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc0c007d50 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc1affd640 1 -- 172.21.15.137:0/4267938592 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c00f450 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c19d6d0 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.376+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c1a4fb0 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.377+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c1021c0 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.378+0000 7fcc1affd640 1 -- 172.21.15.137:0/4267938592 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcc0c005020 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.379+0000 7fcc1affd640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcc040a2c00 0x7fcc040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.379+0000 7fcc21400640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcc040a2c00 0x7fcc040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.380+0000 7fcc1affd640 1 -- 172.21.15.137:0/4267938592 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fcc0c0ce830 con 0x7fcc1c101de0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.380+0000 7fcc21400640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcc040a2c00 0x7fcc040a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc10006010 tx=0x7fcc10009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:10:56.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.384+0000 7fcc1affd640 1 -- 172.21.15.137:0/4267938592 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc0c014030 con 0x7fcc1c101de0 2024-09-12T12:10:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:56 smithi137 ceph-mon[30950]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:56.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:56 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:56 smithi189 ceph-mon[35475]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:56 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:56.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.592+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc1c199790 con 0x7fcc1c101de0 2024-09-12T12:10:56.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.593+0000 7fcc1affd640 1 -- 172.21.15.137:0/4267938592 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8890 (secure 0 0 0) 0x7fcc0c09b5e0 con 0x7fcc1c101de0 2024-09-12T12:10:56.593 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:10:56.593 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":11,"btime":"2024-09-12T12:10:56:261035+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T12:10:56.594 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 11 2024-09-12T12:10:56.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcc040a2c00 msgr2=0x7fcc040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:56.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcc040a2c00 0x7fcc040a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc10006010 tx=0x7fcc10009820 comp rx=0 tx=0).stop 2024-09-12T12:10:56.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c101de0 msgr2=0x7fcc1c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:10:56.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c101de0 0x7fcc1c198430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c970 tx=0x7fcc0c00ce30 comp rx=0 tx=0).stop 2024-09-12T12:10:56.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 shutdown_connections 2024-09-12T12:10:56.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcc040a2c00 0x7fcc040a50c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc1c106f10 0x7fcc1c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc1c102790 0x7fcc1c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 --2- 172.21.15.137:0/4267938592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c101de0 0x7fcc1c198430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:10:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.595+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 >> 172.21.15.137:0/4267938592 conn(0x7fcc1c0fd8c0 msgr2=0x7fcc1c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:10:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.596+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 shutdown_connections 2024-09-12T12:10:56.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:10:56.596+0000 7fcc23e65640 1 -- 172.21.15.137:0/4267938592 wait complete. 2024-09-12T12:10:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:56 smithi171 ceph-mon[35098]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:10:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:10:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:10:56.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:56 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:10:57.275 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:57 smithi137 ceph-mon[30950]: Deploying daemon mds.a on smithi137 2024-09-12T12:10:57.275 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:57 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.137:6838/3700003405,v1:172.21.15.137:6839/3700003405] up:boot 2024-09-12T12:10:57.275 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:57 smithi137 ceph-mon[30950]: fsmap 10 up:standby 2024-09-12T12:10:57.275 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:57 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T12:10:57.275 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:57 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4267938592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:57.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:57 smithi189 ceph-mon[35475]: Deploying daemon mds.a on smithi137 2024-09-12T12:10:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:57 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.137:6838/3700003405,v1:172.21.15.137:6839/3700003405] up:boot 2024-09-12T12:10:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:57 smithi189 ceph-mon[35475]: fsmap 10 up:standby 2024-09-12T12:10:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:57 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T12:10:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:57 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4267938592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:57 smithi171 ceph-mon[35098]: Deploying daemon mds.a on smithi137 2024-09-12T12:10:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:57 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.137:6838/3700003405,v1:172.21.15.137:6839/3700003405] up:boot 2024-09-12T12:10:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:57 smithi171 ceph-mon[35098]: fsmap 10 up:standby 2024-09-12T12:10:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:57 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T12:10:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:57 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4267938592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:10:58.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:10:58 smithi137 ceph-mon[30950]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:58.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:10:58 smithi189 ceph-mon[35475]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:58.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:10:58 smithi171 ceph-mon[35098]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:10:58.799 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:58 smithi137 systemd[1]: Starting Ceph mds.a for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:10:59.205 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:58 smithi137 podman[69151]: 2024-09-12 12:10:58.824531336 +0000 UTC m=+0.016130318 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:10:59.205 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:58 smithi137 podman[69151]: 2024-09-12 12:10:58.951383072 +0000 UTC m=+0.142982045 container create 4db1b9d79bd2ea269b1b58bd1efd1c46f86c5821cc95b80c267a79a85643cbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T12:10:59.205 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:59 smithi137 podman[69151]: 2024-09-12 12:10:59.204337842 +0000 UTC m=+0.395936817 container init 4db1b9d79bd2ea269b1b58bd1efd1c46f86c5821cc95b80c267a79a85643cbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T12:10:59.355 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:10:59.549 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:59 smithi137 podman[69151]: 2024-09-12 12:10:59.207369362 +0000 UTC m=+0.398968343 container start 4db1b9d79bd2ea269b1b58bd1efd1c46f86c5821cc95b80c267a79a85643cbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T12:10:59.549 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:59 smithi137 ceph-mds[69170]: -- 172.21.15.137:0/1100029534 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558f86c45860 con 0x558f86c23000 2024-09-12T12:10:59.550 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a[69166]: starting mds.a at 2024-09-12T12:10:59.550 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:59 smithi137 bash[69151]: 4db1b9d79bd2ea269b1b58bd1efd1c46f86c5821cc95b80c267a79a85643cbbe 2024-09-12T12:10:59.550 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:10:59 smithi137 systemd[1]: Started Ceph mds.a for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:10:59.609 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 -- 172.21.15.137:0/2011281772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508106f00 msgr2=0x7fa50810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 --2- 172.21.15.137:0/2011281772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508106f00 0x7fa50810ea80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa500009ca0 tx=0x7fa50002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 -- 172.21.15.137:0/2011281772 shutdown_connections 2024-09-12T12:11:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 --2- 172.21.15.137:0/2011281772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508106f00 0x7fa50810ea80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 --2- 172.21.15.137:0/2011281772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa508102780 0x7fa5081067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 --2- 172.21.15.137:0/2011281772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa508101dd0 0x7fa5081021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 -- 172.21.15.137:0/2011281772 >> 172.21.15.137:0/2011281772 conn(0x7fa5080fd8b0 msgr2=0x7fa5080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 -- 172.21.15.137:0/2011281772 shutdown_connections 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 -- 172.21.15.137:0/2011281772 wait complete. 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 Processor -- start 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.088+0000 7fa50db1a640 1 -- start start 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa50db1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508101dd0 0x7fa5081983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa50db1a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa508102780 0x7fa508198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa50db1a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa508106f00 0x7fa50819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa50db1a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa508111f60 con 0x7fa508101dd0 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa50db1a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa508111de0 con 0x7fa508106f00 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa50db1a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5081120e0 con 0x7fa508102780 2024-09-12T12:11:00.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508101dd0 0x7fa5081983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508101dd0 0x7fa5081983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36778/0 (socket says 172.21.15.137:36778) 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 -- 172.21.15.137:0/3963925650 learned_addr learned my addr 172.21.15.137:0/3963925650 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa4feffd640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa508102780 0x7fa508198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa507fff640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa508106f00 0x7fa50819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 -- 172.21.15.137:0/3963925650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa508102780 msgr2=0x7fa508198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa508102780 0x7fa508198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 -- 172.21.15.137:0/3963925650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa508106f00 msgr2=0x7fa50819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa508106f00 0x7fa50819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 -- 172.21.15.137:0/3963925650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50819d370 con 0x7fa508101dd0 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5077fe640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508101dd0 0x7fa5081983c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800d770 tx=0x7fa4f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.089+0000 7fa5057fa640 1 -- 172.21.15.137:0/3963925650 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8017070 con 0x7fa508101dd0 2024-09-12T12:11:00.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.090+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50819d660 con 0x7fa508101dd0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.090+0000 7fa5057fa640 1 -- 172.21.15.137:0/3963925650 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8005500 con 0x7fa508101dd0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.090+0000 7fa5057fa640 1 -- 172.21.15.137:0/3963925650 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004e00 con 0x7fa508101dd0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.090+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5081a4f40 con 0x7fa508101dd0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.090+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5081021b0 con 0x7fa508101dd0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.092+0000 7fa5057fa640 1 -- 172.21.15.137:0/3963925650 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa4f8004fa0 con 0x7fa508101dd0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.093+0000 7fa5057fa640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4f40a2b80 0x7fa4f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.093+0000 7fa4feffd640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4f40a2b80 0x7fa4f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.093+0000 7fa5057fa640 1 -- 172.21.15.137:0/3963925650 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa4f80ce9a0 con 0x7fa508101dd0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.094+0000 7fa4feffd640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4f40a2b80 0x7fa4f40a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5081088d0 tx=0x7fa4f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:00.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.097+0000 7fa5057fa640 1 -- 172.21.15.137:0/3963925650 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4f809b6b0 con 0x7fa508101dd0 2024-09-12T12:11:00.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.306+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa508199720 con 0x7fa508101dd0 2024-09-12T12:11:00.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.306+0000 7fa5057fa640 1 -- 172.21.15.137:0/3963925650 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9719 (secure 0 0 0) 0x7fa4f809f9b0 con 0x7fa508101dd0 2024-09-12T12:11:00.307 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:00.307 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":12,"btime":"2024-09-12T12:10:59:277444+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T12:11:00.307 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 12 2024-09-12T12:11:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4f40a2b80 msgr2=0x7fa4f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4f40a2b80 0x7fa4f40a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5081088d0 tx=0x7fa4f000b410 comp rx=0 tx=0).stop 2024-09-12T12:11:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508101dd0 msgr2=0x7fa5081983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508101dd0 0x7fa5081983c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800d770 tx=0x7fa4f800dc30 comp rx=0 tx=0).stop 2024-09-12T12:11:00.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 shutdown_connections 2024-09-12T12:11:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa4f40a2b80 0x7fa4f40a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa508106f00 0x7fa50819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa508102780 0x7fa508198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 --2- 172.21.15.137:0/3963925650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa508101dd0 0x7fa5081983c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 >> 172.21.15.137:0/3963925650 conn(0x7fa5080fd8b0 msgr2=0x7fa5080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 shutdown_connections 2024-09-12T12:11:00.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:00.310+0000 7fa50db1a640 1 -- 172.21.15.137:0/3963925650 wait complete. 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.137:6840/2375268827,v1:172.21.15.137:6841/2375268827] up:boot 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: fsmap 11 up:standby 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:11:00.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:00 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.137:6840/2375268827,v1:172.21.15.137:6841/2375268827] up:boot 2024-09-12T12:11:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: fsmap 11 up:standby 2024-09-12T12:11:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T12:11:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:11:00.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:11:00.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:00 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:00.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.137:6840/2375268827,v1:172.21.15.137:6841/2375268827] up:boot 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: fsmap 11 up:standby 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T12:11:00.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:00 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:01 smithi137 ceph-mon[30950]: Deploying daemon mds.b on smithi171 2024-09-12T12:11:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:01 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3963925650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:01.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:01 smithi189 ceph-mon[35475]: Deploying daemon mds.b on smithi171 2024-09-12T12:11:01.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:01 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3963925650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:01.653 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:01 smithi171 ceph-mon[35098]: Deploying daemon mds.b on smithi171 2024-09-12T12:11:01.653 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:01 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3963925650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:01.653 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:01 smithi171 systemd[1]: Starting Ceph mds.b for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:11:01.927 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:01 smithi171 podman[64943]: 2024-09-12 12:11:01.555640669 +0000 UTC m=+0.016101607 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:11:01.927 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:01 smithi171 podman[64943]: 2024-09-12 12:11:01.700159825 +0000 UTC m=+0.160620755 container create be8aa567e0a4f938c8c4cecb1279f2760f5ba5fe1cd0f537c19c59cab456f6f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T12:11:01.927 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:01 smithi171 podman[64943]: 2024-09-12 12:11:01.927806947 +0000 UTC m=+0.388267875 container init be8aa567e0a4f938c8c4cecb1279f2760f5ba5fe1cd0f537c19c59cab456f6f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T12:11:02.200 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:01 smithi171 podman[64943]: 2024-09-12 12:11:01.9307917 +0000 UTC m=+0.391252633 container start be8aa567e0a4f938c8c4cecb1279f2760f5ba5fe1cd0f537c19c59cab456f6f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:11:02.200 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:01 smithi171 ceph-mds[64962]: -- 172.21.15.171:0/2324921454 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f1776f860 con 0x561f1774d000 2024-09-12T12:11:02.200 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: starting mds.b at 2024-09-12T12:11:02.200 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:02 smithi171 bash[64943]: be8aa567e0a4f938c8c4cecb1279f2760f5ba5fe1cd0f537c19c59cab456f6f4 2024-09-12T12:11:02.200 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:02 smithi171 systemd[1]: Started Ceph mds.b for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:11:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:02 smithi137 ceph-mon[30950]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:02 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:11:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:02 smithi189 ceph-mon[35475]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:02 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:11:02.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:02 smithi171 ceph-mon[35098]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:02.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:02.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:02 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:11:03.024 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:11:03.367 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:03.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:03 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] up:boot 2024-09-12T12:11:03.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:03 smithi137 ceph-mon[30950]: fsmap 12 up:standby 2024-09-12T12:11:03.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:03 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:11:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:03 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] up:boot 2024-09-12T12:11:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:03 smithi189 ceph-mon[35475]: fsmap 12 up:standby 2024-09-12T12:11:03.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:03 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:11:03.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:03 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] up:boot 2024-09-12T12:11:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:03 smithi171 ceph-mon[35098]: fsmap 12 up:standby 2024-09-12T12:11:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:03 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:11:03.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 -- 172.21.15.137:0/3954867706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c102790 msgr2=0x7f562c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:03.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 --2- 172.21.15.137:0/3954867706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c102790 0x7f562c1067e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5620009900 tx=0x7f562002a410 comp rx=0 tx=0).stop 2024-09-12T12:11:03.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 -- 172.21.15.137:0/3954867706 shutdown_connections 2024-09-12T12:11:03.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 --2- 172.21.15.137:0/3954867706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f562c106f10 0x7f562c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:03.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 --2- 172.21.15.137:0/3954867706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c102790 0x7f562c1067e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:03.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 --2- 172.21.15.137:0/3954867706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c101de0 0x7f562c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:03.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 -- 172.21.15.137:0/3954867706 >> 172.21.15.137:0/3954867706 conn(0x7f562c0fd8c0 msgr2=0x7f562c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 -- 172.21.15.137:0/3954867706 shutdown_connections 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.829+0000 7f563227f640 1 -- 172.21.15.137:0/3954867706 wait complete. 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.830+0000 7f563227f640 1 Processor -- start 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.830+0000 7f563227f640 1 -- start start 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.830+0000 7f563227f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c101de0 0x7f562c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563227f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f562c102790 0x7f562c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563227f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c106f10 0x7f562c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563227f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f562c111f70 con 0x7f562c106f10 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563227f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f562c111df0 con 0x7f562c101de0 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563227f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1120f0 con 0x7f562c102790 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c106f10 0x7f562c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c106f10 0x7f562c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36792/0 (socket says 172.21.15.137:36792) 2024-09-12T12:11:03.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 -- 172.21.15.137:0/2546957204 learned_addr learned my addr 172.21.15.137:0/2546957204 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f562b7fe640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f562c102790 0x7f562c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f562bfff640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c101de0 0x7f562c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 -- 172.21.15.137:0/2546957204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f562c102790 msgr2=0x7f562c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f562c102790 0x7f562c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 -- 172.21.15.137:0/2546957204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c101de0 msgr2=0x7f562c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c101de0 0x7f562c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 -- 172.21.15.137:0/2546957204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c19d480 con 0x7f562c106f10 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f562bfff640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c101de0 0x7f562c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563081c640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c106f10 0x7f562c19cda0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5624009fd0 tx=0x7f562400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f56297fa640 1 -- 172.21.15.137:0/2546957204 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624017070 con 0x7f562c106f10 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c19d770 con 0x7f562c106f10 2024-09-12T12:11:03.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f56297fa640 1 -- 172.21.15.137:0/2546957204 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5624003e50 con 0x7f562c106f10 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f56297fa640 1 -- 172.21.15.137:0/2546957204 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56240106c0 con 0x7f562c106f10 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.831+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c1a5050 con 0x7f562c106f10 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.832+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562c1032c0 con 0x7f562c106f10 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.833+0000 7f56297fa640 1 -- 172.21.15.137:0/2546957204 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5624003970 con 0x7f562c106f10 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.834+0000 7f56297fa640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f56140a2b80 0x7f56140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.834+0000 7f562bfff640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f56140a2b80 0x7f56140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.834+0000 7f56297fa640 1 -- 172.21.15.137:0/2546957204 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f56240ce8b0 con 0x7f562c106f10 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.834+0000 7f562bfff640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f56140a2b80 0x7f56140a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f561c005d00 tx=0x7f561c009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:03.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:03.837+0000 7f56297fa640 1 -- 172.21.15.137:0/2546957204 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f562409bcb0 con 0x7f562c106f10 2024-09-12T12:11:04.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.067+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f562c10a150 con 0x7f562c106f10 2024-09-12T12:11:04.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.067+0000 7f56297fa640 1 -- 172.21.15.137:0/2546957204 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10548 (secure 0 0 0) 0x7f562409b5b0 con 0x7f562c106f10 2024-09-12T12:11:04.067 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:04.067 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":13,"btime":"2024-09-12T12:11:02:290785+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24454,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24475,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-12T12:11:04.068 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 13 2024-09-12T12:11:04.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f56140a2b80 msgr2=0x7f56140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:04.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f56140a2b80 0x7f56140a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f561c005d00 tx=0x7f561c009550 comp rx=0 tx=0).stop 2024-09-12T12:11:04.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c106f10 msgr2=0x7f562c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:04.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c106f10 0x7f562c19cda0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5624009fd0 tx=0x7f562400b260 comp rx=0 tx=0).stop 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 shutdown_connections 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f56140a2b80 0x7f56140a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f562c106f10 0x7f562c19cda0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f562c102790 0x7f562c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 --2- 172.21.15.137:0/2546957204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f562c101de0 0x7f562c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 >> 172.21.15.137:0/2546957204 conn(0x7f562c0fd8c0 msgr2=0x7f562c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 shutdown_connections 2024-09-12T12:11:04.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:04.070+0000 7f563227f640 1 -- 172.21.15.137:0/2546957204 wait complete. 2024-09-12T12:11:04.308 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:04.308 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.308 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.308 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.308 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.308 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2546957204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:04.308 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:04.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:11:04.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:04 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:11:04.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2546957204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:04 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:11:04.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2546957204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:04 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:11:04.772 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T12:11:05.698 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 -- 172.21.15.137:0/1488664887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 msgr2=0x7fdc7c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/1488664887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c100ec0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009930 tx=0x7fdc7002a420 comp rx=0 tx=0).stop 2024-09-12T12:11:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 -- 172.21.15.137:0/1488664887 shutdown_connections 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/1488664887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc7c101400 0x7fdc7c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/1488664887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c100ec0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/1488664887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc7c106a60 0x7fdc7c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 -- 172.21.15.137:0/1488664887 >> 172.21.15.137:0/1488664887 conn(0x7fdc7c0fc780 msgr2=0x7fdc7c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 -- 172.21.15.137:0/1488664887 shutdown_connections 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 -- 172.21.15.137:0/1488664887 wait complete. 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 Processor -- start 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.282+0000 7fdc82a3b640 1 -- start start 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc82a3b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc82a3b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc7c101400 0x7fdc7c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc82a3b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc7c106a60 0x7fdc7c1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc82a3b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c110e60 con 0x7fdc7c100a60 2024-09-12T12:11:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc82a3b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c110ce0 con 0x7fdc7c106a60 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc82a3b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c110fe0 con 0x7fdc7c101400 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36812/0 (socket says 172.21.15.137:36812) 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 -- 172.21.15.137:0/580699308 learned_addr learned my addr 172.21.15.137:0/580699308 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7b7fe640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc7c101400 0x7fdc7c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc80fd8640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc7c106a60 0x7fdc7c1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 -- 172.21.15.137:0/580699308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc7c101400 msgr2=0x7fdc7c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc7c101400 0x7fdc7c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 -- 172.21.15.137:0/580699308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc7c106a60 msgr2=0x7fdc7c1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc7c106a60 0x7fdc7c1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 -- 172.21.15.137:0/580699308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c1a48f0 con 0x7fdc7c100a60 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc80fd8640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc7c106a60 0x7fdc7c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc7bfff640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c199330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d770 tx=0x7fdc6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.283+0000 7fdc797fa640 1 -- 172.21.15.137:0/580699308 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c017070 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.284+0000 7fdc797fa640 1 -- 172.21.15.137:0/580699308 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc6c005500 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.284+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c1a4ac0 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.284+0000 7fdc797fa640 1 -- 172.21.15.137:0/580699308 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c004e00 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.284+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c1a4f80 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.285+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c100ec0 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.286+0000 7fdc797fa640 1 -- 172.21.15.137:0/580699308 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdc6c004fa0 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.286+0000 7fdc797fa640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc640a2b80 0x7fdc640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.286+0000 7fdc7b7fe640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc640a2b80 0x7fdc640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.286+0000 7fdc797fa640 1 -- 172.21.15.137:0/580699308 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fdc6c0ce9a0 con 0x7fdc7c100a60 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.287+0000 7fdc7b7fe640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc640a2b80 0x7fdc640a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009900 tx=0x7fdc700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:06.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.289+0000 7fdc797fa640 1 -- 172.21.15.137:0/580699308 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc6c09bda0 con 0x7fdc7c100a60 2024-09-12T12:11:06.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:06 smithi171 ceph-mon[35098]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:06 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.480+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdc7c108fd0 con 0x7fdc7c100a60 2024-09-12T12:11:06.480 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.480 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:06 smithi137 ceph-mon[30950]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:06.480 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.480 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.480 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.481 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:06 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.481+0000 7fdc797fa640 1 -- 172.21.15.137:0/580699308 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7fdc6c09b6a0 con 0x7fdc7c100a60 2024-09-12T12:11:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:06.481 INFO:teuthology.orchestra.run.smithi137.stdout:[ 2024-09-12T12:11:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:11:06.481 INFO:teuthology.orchestra.run.smithi137.stdout: "poolnum": 1, 2024-09-12T12:11:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: "poolname": ".mgr" 2024-09-12T12:11:06.482 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:11:06.482 INFO:teuthology.orchestra.run.smithi137.stdout:] 2024-09-12T12:11:06.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc640a2b80 msgr2=0x7fdc640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:06.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc640a2b80 0x7fdc640a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc70009900 tx=0x7fdc700058d0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 msgr2=0x7fdc7c199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c199330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00d770 tx=0x7fdc6c00dc30 comp rx=0 tx=0).stop 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 shutdown_connections 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc640a2b80 0x7fdc640a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc7c106a60 0x7fdc7c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc7c101400 0x7fdc7c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 --2- 172.21.15.137:0/580699308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc7c100a60 0x7fdc7c199330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 >> 172.21.15.137:0/580699308 conn(0x7fdc7c0fc780 msgr2=0x7fdc7c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 shutdown_connections 2024-09-12T12:11:06.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:06.484+0000 7fdc82a3b640 1 -- 172.21.15.137:0/580699308 wait complete. 2024-09-12T12:11:06.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:06 smithi189 ceph-mon[35475]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:06.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:06 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:07.148 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T12:11:07.148 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T12:11:07.414 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:07.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:07 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/580699308' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T12:11:07.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:07 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/580699308' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T12:11:07.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:07 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/580699308' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T12:11:07.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 -- 172.21.15.137:0/1742378605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4081036c0 msgr2=0x7fa40810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:07.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 --2- 172.21.15.137:0/1742378605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4081036c0 0x7fa40810bbb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009900 tx=0x7fa3fc02a480 comp rx=0 tx=0).stop 2024-09-12T12:11:07.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 -- 172.21.15.137:0/1742378605 shutdown_connections 2024-09-12T12:11:07.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 --2- 172.21.15.137:0/1742378605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa40810c180 0x7fa40810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:07.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 --2- 172.21.15.137:0/1742378605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4081036c0 0x7fa40810bbb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 --2- 172.21.15.137:0/1742378605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa408102da0 0x7fa408103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 -- 172.21.15.137:0/1742378605 >> 172.21.15.137:0/1742378605 conn(0x7fa4080fc7a0 msgr2=0x7fa4080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 -- 172.21.15.137:0/1742378605 shutdown_connections 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.928+0000 7fa40e231640 1 -- 172.21.15.137:0/1742378605 wait complete. 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 Processor -- start 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 -- start start 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa408102da0 0x7fa4081985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4081036c0 0x7fa408198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40810c180 0x7fa40819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa408112020 con 0x7fa40810c180 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa408111ea0 con 0x7fa408102da0 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa40e231640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa4081121a0 con 0x7fa4081036c0 2024-09-12T12:11:07.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa407fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40810c180 0x7fa40819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa407fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40810c180 0x7fa40819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36838/0 (socket says 172.21.15.137:36838) 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa407fff640 1 -- 172.21.15.137:0/4139318281 learned_addr learned my addr 172.21.15.137:0/4139318281 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.929+0000 7fa4077fe640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa408102da0 0x7fa4081985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa406ffd640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4081036c0 0x7fa408198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa407fff640 1 -- 172.21.15.137:0/4139318281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4081036c0 msgr2=0x7fa408198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa407fff640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4081036c0 0x7fa408198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa407fff640 1 -- 172.21.15.137:0/4139318281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa408102da0 msgr2=0x7fa4081985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa407fff640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa408102da0 0x7fa4081985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa407fff640 1 -- 172.21.15.137:0/4139318281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40819d550 con 0x7fa40810c180 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa407fff640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40810c180 0x7fa40819ce70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa40000b970 tx=0x7fa40000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa404ff9640 1 -- 172.21.15.137:0/4139318281 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400019070 con 0x7fa40810c180 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40819d840 con 0x7fa40810c180 2024-09-12T12:11:07.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa404ff9640 1 -- 172.21.15.137:0/4139318281 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa40000cb40 con 0x7fa40810c180 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa404ff9640 1 -- 172.21.15.137:0/4139318281 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400007650 con 0x7fa40810c180 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.930+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081a5120 con 0x7fa40810c180 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.931+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa408103200 con 0x7fa40810c180 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.932+0000 7fa404ff9640 1 -- 172.21.15.137:0/4139318281 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa40001c070 con 0x7fa40810c180 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.933+0000 7fa404ff9640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa3f00a2b80 0x7fa3f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.933+0000 7fa4077fe640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa3f00a2b80 0x7fa3f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.933+0000 7fa404ff9640 1 -- 172.21.15.137:0/4139318281 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa4000ce6f0 con 0x7fa40810c180 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.933+0000 7fa4077fe640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa3f00a2b80 0x7fa3f00a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009890 tx=0x7fa3f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:07.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:07.936+0000 7fa404ff9640 1 -- 172.21.15.137:0/4139318281 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa40009baf0 con 0x7fa40810c180 2024-09-12T12:11:08.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.102+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa40802d7c0 con 0x7fa40810c180 2024-09-12T12:11:08.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.459+0000 7fa404ff9640 1 -- 172.21.15.137:0/4139318281 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fa40009b3f0 con 0x7fa40810c180 2024-09-12T12:11:08.459 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_metadata' created 2024-09-12T12:11:08.459 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:08 smithi137 ceph-mon[30950]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:08.459 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.459 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.459 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:08.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:11:08.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa3f00a2b80 msgr2=0x7fa3f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:08.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa3f00a2b80 0x7fa3f00a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009890 tx=0x7fa3f8008040 comp rx=0 tx=0).stop 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40810c180 msgr2=0x7fa40819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40810c180 0x7fa40819ce70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa40000b970 tx=0x7fa40000be30 comp rx=0 tx=0).stop 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 shutdown_connections 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa3f00a2b80 0x7fa3f00a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa40810c180 0x7fa40819ce70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4081036c0 0x7fa408198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 --2- 172.21.15.137:0/4139318281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa408102da0 0x7fa4081985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 >> 172.21.15.137:0/4139318281 conn(0x7fa4080fc7a0 msgr2=0x7fa4080fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 shutdown_connections 2024-09-12T12:11:08.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:08.462+0000 7fa40e231640 1 -- 172.21.15.137:0/4139318281 wait complete. 2024-09-12T12:11:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:08 smithi189 ceph-mon[35475]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:11:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:08 smithi171 ceph-mon[35098]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:11:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:11:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:11:09.172 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T12:11:09.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:09 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4139318281' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T12:11:09.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:09 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4139318281' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T12:11:09.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:09 smithi137 ceph-mon[30950]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T12:11:09.551 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:09 smithi137 ceph-mon[30950]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:09.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:09 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4139318281' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T12:11:09.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:09 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4139318281' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T12:11:09.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:09 smithi189 ceph-mon[35475]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T12:11:09.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:09 smithi189 ceph-mon[35475]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:09.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:09 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4139318281' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T12:11:09.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:09 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4139318281' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T12:11:09.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:09 smithi171 ceph-mon[35098]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T12:11:09.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:09 smithi171 ceph-mon[35098]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:09.802 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:10.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 -- 172.21.15.137:0/4011093137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0101420 msgr2=0x7efde010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:10.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 --2- 172.21.15.137:0/4011093137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0101420 0x7efde010d950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009cd0 tx=0x7efdd802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:11:10.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 -- 172.21.15.137:0/4011093137 shutdown_connections 2024-09-12T12:11:10.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 --2- 172.21.15.137:0/4011093137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0101420 0x7efde010d950 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:10.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 --2- 172.21.15.137:0/4011093137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0100a80 0x7efde0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:10.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 --2- 172.21.15.137:0/4011093137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efde0106a80 0x7efde0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 -- 172.21.15.137:0/4011093137 >> 172.21.15.137:0/4011093137 conn(0x7efde00fc7a0 msgr2=0x7efde00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 -- 172.21.15.137:0/4011093137 shutdown_connections 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.296+0000 7efde5675640 1 -- 172.21.15.137:0/4011093137 wait complete. 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 Processor -- start 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 -- start start 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0100a80 0x7efde0197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efde0101420 0x7efde01978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0106a80 0x7efde019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efde0110ea0 con 0x7efde0106a80 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efde0110d20 con 0x7efde0101420 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efde5675640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efde0111020 con 0x7efde0100a80 2024-09-12T12:11:10.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0106a80 0x7efde019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efddf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0106a80 0x7efde019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33840/0 (socket says 172.21.15.137:33840) 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.297+0000 7efddf7fe640 1 -- 172.21.15.137:0/150125616 learned_addr learned my addr 172.21.15.137:0/150125616 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efdd67fc640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efde0101420 0x7efde01978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddeffd640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0100a80 0x7efde0197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddf7fe640 1 -- 172.21.15.137:0/150125616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0100a80 msgr2=0x7efde0197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddf7fe640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0100a80 0x7efde0197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddf7fe640 1 -- 172.21.15.137:0/150125616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efde0101420 msgr2=0x7efde01978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddf7fe640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efde0101420 0x7efde01978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddf7fe640 1 -- 172.21.15.137:0/150125616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde019c330 con 0x7efde0106a80 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddf7fe640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0106a80 0x7efde019bc50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005cc0 tx=0x7efdd8031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddcff9640 1 -- 172.21.15.137:0/150125616 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd804a070 con 0x7efde0106a80 2024-09-12T12:11:10.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddcff9640 1 -- 172.21.15.137:0/150125616 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd8004040 con 0x7efde0106a80 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde019c5c0 con 0x7efde0106a80 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efddcff9640 1 -- 172.21.15.137:0/150125616 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8030800 con 0x7efde0106a80 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.298+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde006a790 con 0x7efde0106a80 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.299+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde01021c0 con 0x7efde0106a80 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.300+0000 7efddcff9640 1 -- 172.21.15.137:0/150125616 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7efdd80041f0 con 0x7efde0106a80 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.301+0000 7efddcff9640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7efdcc0a2b80 0x7efdcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.301+0000 7efddeffd640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7efdcc0a2b80 0x7efdcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.301+0000 7efddcff9640 1 -- 172.21.15.137:0/150125616 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7efdd802e020 con 0x7efde0106a80 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.302+0000 7efddeffd640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7efdcc0a2b80 0x7efdcc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009b60 tx=0x7efdd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:10.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.305+0000 7efddcff9640 1 -- 172.21.15.137:0/150125616 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdd8053020 con 0x7efde0106a80 2024-09-12T12:11:10.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:10.474+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efde0100ee0 con 0x7efde0106a80 2024-09-12T12:11:10.474 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:10 smithi137 ceph-mon[30950]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T12:11:10.474 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:10 smithi137 ceph-mon[30950]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T12:11:10.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:10 smithi171 ceph-mon[35098]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T12:11:10.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:10 smithi171 ceph-mon[35098]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T12:11:10.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:10 smithi189 ceph-mon[35475]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T12:11:10.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:10 smithi189 ceph-mon[35475]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T12:11:11.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.478+0000 7efddcff9640 1 -- 172.21.15.137:0/150125616 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7efde0100ee0 con 0x7efde0106a80 2024-09-12T12:11:11.478 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_data' created 2024-09-12T12:11:11.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7efdcc0a2b80 msgr2=0x7efdcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:11.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7efdcc0a2b80 0x7efdcc0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009b60 tx=0x7efdd0005f90 comp rx=0 tx=0).stop 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0106a80 msgr2=0x7efde019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0106a80 0x7efde019bc50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005cc0 tx=0x7efdd8031e60 comp rx=0 tx=0).stop 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 shutdown_connections 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7efdcc0a2b80 0x7efdcc0a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efde0106a80 0x7efde019bc50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efde0101420 0x7efde01978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 --2- 172.21.15.137:0/150125616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0100a80 0x7efde0197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.481+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 >> 172.21.15.137:0/150125616 conn(0x7efde00fc7a0 msgr2=0x7efde00fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.482+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 shutdown_connections 2024-09-12T12:11:11.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:11.482+0000 7efde5675640 1 -- 172.21.15.137:0/150125616 wait complete. 2024-09-12T12:11:11.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:11 smithi137 ceph-mon[30950]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T12:11:11.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:11 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/150125616' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T12:11:11.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:11 smithi137 ceph-mon[30950]: pgmap v277: 33 pgs: 5 creating+peering, 21 unknown, 7 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:11.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:11 smithi189 ceph-mon[35475]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T12:11:11.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:11 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/150125616' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T12:11:11.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:11 smithi189 ceph-mon[35475]: pgmap v277: 33 pgs: 5 creating+peering, 21 unknown, 7 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:11.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:11 smithi171 ceph-mon[35098]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T12:11:11.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:11 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/150125616' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T12:11:11.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:11 smithi171 ceph-mon[35098]: pgmap v277: 33 pgs: 5 creating+peering, 21 unknown, 7 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:12.208 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T12:11:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:12.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:12 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/150125616' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T12:11:12.487 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:12 smithi137 ceph-mon[30950]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T12:11:12.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:12 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/150125616' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T12:11:12.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:12 smithi189 ceph-mon[35475]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T12:11:12.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:12 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/150125616' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T12:11:12.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:12 smithi171 ceph-mon[35098]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T12:11:13.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 -- 172.21.15.137:0/2583432931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090106f20 msgr2=0x7fe09010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 --2- 172.21.15.137:0/2583432931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090106f20 0x7fe09010eaa0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe088009ca0 tx=0x7fe08802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 -- 172.21.15.137:0/2583432931 shutdown_connections 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 --2- 172.21.15.137:0/2583432931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090106f20 0x7fe09010eaa0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 --2- 172.21.15.137:0/2583432931 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0901027a0 0x7fe0901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 --2- 172.21.15.137:0/2583432931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe090101df0 0x7fe0901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 -- 172.21.15.137:0/2583432931 >> 172.21.15.137:0/2583432931 conn(0x7fe0900fd8b0 msgr2=0x7fe0900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 -- 172.21.15.137:0/2583432931 shutdown_connections 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.984+0000 7fe09725c640 1 -- 172.21.15.137:0/2583432931 wait complete. 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 Processor -- start 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 -- start start 2024-09-12T12:11:13.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090101df0 0x7fe090198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0901027a0 0x7fe090198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe090106f20 0x7fe09019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe090111f80 con 0x7fe090101df0 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe090111e00 con 0x7fe0901027a0 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe09725c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe090112100 con 0x7fe090106f20 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe094ff8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090101df0 0x7fe090198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe094ff8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090101df0 0x7fe090198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33854/0 (socket says 172.21.15.137:33854) 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe094ff8640 1 -- 172.21.15.137:0/634122591 learned_addr learned my addr 172.21.15.137:0/634122591 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe0957f9640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe090106f20 0x7fe09019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.985+0000 7fe08ffff640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0901027a0 0x7fe090198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe094ff8640 1 -- 172.21.15.137:0/634122591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe090106f20 msgr2=0x7fe09019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe094ff8640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe090106f20 0x7fe09019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe094ff8640 1 -- 172.21.15.137:0/634122591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0901027a0 msgr2=0x7fe090198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe094ff8640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0901027a0 0x7fe090198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe094ff8640 1 -- 172.21.15.137:0/634122591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09019d3f0 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe0957f9640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe090106f20 0x7fe09019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe094ff8640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090101df0 0x7fe090198440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe08400c890 tx=0x7fe08400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe08dffb640 1 -- 172.21.15.137:0/634122591 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084004220 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09019d6e0 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe08dffb640 1 -- 172.21.15.137:0/634122591 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0840043c0 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe08dffb640 1 -- 172.21.15.137:0/634122591 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084004a00 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.986+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0901a4fc0 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.987+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0901021d0 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.988+0000 7fe08dffb640 1 -- 172.21.15.137:0/634122591 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe0840079b0 con 0x7fe090101df0 2024-09-12T12:11:13.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.989+0000 7fe08dffb640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe0800a2b80 0x7fe0800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:13.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.989+0000 7fe08ffff640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe0800a2b80 0x7fe0800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:13.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.989+0000 7fe08dffb640 1 -- 172.21.15.137:0/634122591 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7fe0840cec40 con 0x7fe090101df0 2024-09-12T12:11:13.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.989+0000 7fe08ffff640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe0800a2b80 0x7fe0800a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe07c005d70 tx=0x7fe07c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:13.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:12.993+0000 7fe08dffb640 1 -- 172.21.15.137:0/634122591 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe084014030 con 0x7fe090101df0 2024-09-12T12:11:13.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.194+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe09010a100 con 0x7fe090101df0 2024-09-12T12:11:13.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.487+0000 7fe08dffb640 1 -- 172.21.15.137:0/634122591 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe08409bd50 con 0x7fe090101df0 2024-09-12T12:11:13.488 INFO:teuthology.orchestra.run.smithi137.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T12:11:13.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe0800a2b80 msgr2=0x7fe0800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe0800a2b80 0x7fe0800a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe07c005d70 tx=0x7fe07c005d00 comp rx=0 tx=0).stop 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090101df0 msgr2=0x7fe090198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090101df0 0x7fe090198440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe08400c890 tx=0x7fe08400cd50 comp rx=0 tx=0).stop 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 shutdown_connections 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe0800a2b80 0x7fe0800a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe090106f20 0x7fe09019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0901027a0 0x7fe090198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 --2- 172.21.15.137:0/634122591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe090101df0 0x7fe090198440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 >> 172.21.15.137:0/634122591 conn(0x7fe0900fd8b0 msgr2=0x7fe0900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 shutdown_connections 2024-09-12T12:11:13.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:13.490+0000 7fe09725c640 1 -- 172.21.15.137:0/634122591 wait complete. 2024-09-12T12:11:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:13 smithi137 ceph-mon[30950]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T12:11:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:13 smithi137 ceph-mon[30950]: pgmap v280: 97 pgs: 5 creating+peering, 51 unknown, 41 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:13 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/634122591' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T12:11:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:13 smithi137 ceph-mon[30950]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T12:11:13.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:13 smithi189 ceph-mon[35475]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T12:11:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:13 smithi189 ceph-mon[35475]: pgmap v280: 97 pgs: 5 creating+peering, 51 unknown, 41 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:13 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/634122591' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T12:11:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:13 smithi189 ceph-mon[35475]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T12:11:13.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:13 smithi171 ceph-mon[35098]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T12:11:13.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:13 smithi171 ceph-mon[35098]: pgmap v280: 97 pgs: 5 creating+peering, 51 unknown, 41 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:11:13.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:13 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/634122591' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T12:11:13.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:13 smithi171 ceph-mon[35098]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-12T12:11:14.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:11:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:14.745 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:14 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/634122591' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T12:11:14.745 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:14 smithi137 ceph-mon[30950]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T12:11:14.745 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:14 smithi137 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T12:11:14.745 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:14 smithi137 ceph-mon[30950]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T12:11:14.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:14 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/634122591' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T12:11:14.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:14 smithi189 ceph-mon[35475]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T12:11:14.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:14 smithi189 ceph-mon[35475]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T12:11:14.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:14 smithi189 ceph-mon[35475]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T12:11:14.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:14 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/634122591' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T12:11:14.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:14 smithi171 ceph-mon[35098]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T12:11:14.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:14 smithi171 ceph-mon[35098]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-12T12:11:14.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:14 smithi171 ceph-mon[35098]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-12T12:11:14.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 -- 172.21.15.137:0/65203871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8112b90 msgr2=0x7f6ff8114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:14.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 --2- 172.21.15.137:0/65203871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8112b90 0x7f6ff8114f80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009ca0 tx=0x7f6ff002a950 comp rx=0 tx=0).stop 2024-09-12T12:11:14.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 -- 172.21.15.137:0/65203871 shutdown_connections 2024-09-12T12:11:14.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 --2- 172.21.15.137:0/65203871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8112b90 0x7f6ff8114f80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:14.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 --2- 172.21.15.137:0/65203871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ff8074eb0 0x7f6ff8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:14.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 --2- 172.21.15.137:0/65203871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ff8074590 0x7f6ff8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:14.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 -- 172.21.15.137:0/65203871 >> 172.21.15.137:0/65203871 conn(0x7f6ff80fd8c0 msgr2=0x7f6ff80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:14.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 -- 172.21.15.137:0/65203871 shutdown_connections 2024-09-12T12:11:14.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.956+0000 7f6ffef80640 1 -- 172.21.15.137:0/65203871 wait complete. 2024-09-12T12:11:14.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 Processor -- start 2024-09-12T12:11:14.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 -- start start 2024-09-12T12:11:14.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8074590 0x7f6ff81a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:14.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ff8074eb0 0x7f6ff81a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:14.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ff8112b90 0x7f6ff819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:14.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81185c0 con 0x7f6ff8074590 2024-09-12T12:11:14.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8118440 con 0x7f6ff8074eb0 2024-09-12T12:11:14.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffef80640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8118740 con 0x7f6ff8112b90 2024-09-12T12:11:14.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8074590 0x7f6ff81a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:14.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8074590 0x7f6ff81a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33866/0 (socket says 172.21.15.137:33866) 2024-09-12T12:11:14.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 -- 172.21.15.137:0/3004899066 learned_addr learned my addr 172.21.15.137:0/3004899066 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:14.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffd51d640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ff8112b90 0x7f6ff819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:14.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ff7fff640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ff8074eb0 0x7f6ff81a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 -- 172.21.15.137:0/3004899066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ff8112b90 msgr2=0x7f6ff819e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ff8112b90 0x7f6ff819e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 -- 172.21.15.137:0/3004899066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ff8074eb0 msgr2=0x7f6ff81a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ff8074eb0 0x7f6ff81a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 -- 172.21.15.137:0/3004899066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff819f010 con 0x7f6ff8074590 2024-09-12T12:11:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ff7fff640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ff8074eb0 0x7f6ff81a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:11:14.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.957+0000 7f6ffcd1c640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8074590 0x7f6ff81a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b4a0 tx=0x7f6fec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:14.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.958+0000 7f6ff5ffb640 1 -- 172.21.15.137:0/3004899066 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec014070 con 0x7f6ff8074590 2024-09-12T12:11:14.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.958+0000 7f6ff5ffb640 1 -- 172.21.15.137:0/3004899066 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fec010af0 con 0x7f6ff8074590 2024-09-12T12:11:14.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.958+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff819f300 con 0x7f6ff8074590 2024-09-12T12:11:14.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.958+0000 7f6ff5ffb640 1 -- 172.21.15.137:0/3004899066 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec017d10 con 0x7f6ff8074590 2024-09-12T12:11:14.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.958+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81ab5f0 con 0x7f6ff8074590 2024-09-12T12:11:14.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.959+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff8074970 con 0x7f6ff8074590 2024-09-12T12:11:14.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.960+0000 7f6ff5ffb640 1 -- 172.21.15.137:0/3004899066 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6fec017070 con 0x7f6ff8074590 2024-09-12T12:11:14.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.961+0000 7f6ff5ffb640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6fe80a2c00 0x7f6fe80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:14.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.961+0000 7f6ff7fff640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6fe80a2c00 0x7f6fe80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:14.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.961+0000 7f6ff5ffb640 1 -- 172.21.15.137:0/3004899066 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f6fec0ce290 con 0x7f6ff8074590 2024-09-12T12:11:14.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.961+0000 7f6ff7fff640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6fe80a2c00 0x7f6fe80a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8114c40 tx=0x7f6fe4005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:14.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:14.965+0000 7f6ff5ffb640 1 -- 172.21.15.137:0/3004899066 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fec09b340 con 0x7f6ff8074590 2024-09-12T12:11:15.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.133+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ff81ab8e0 con 0x7f6ff8074590 2024-09-12T12:11:15.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.135+0000 7f6ff5ffb640 1 -- 172.21.15.137:0/3004899066 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22348 (secure 0 0 0) 0x7f6fec09ac40 con 0x7f6ff8074590 2024-09-12T12:11:15.136 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:15.137 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:11:13.196473+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:11:15.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6fe80a2c00 msgr2=0x7f6fe80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6fe80a2c00 0x7f6fe80a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8114c40 tx=0x7f6fe4005d30 comp rx=0 tx=0).stop 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8074590 msgr2=0x7f6ff81a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8074590 0x7f6ff81a4720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b4a0 tx=0x7f6fec00b960 comp rx=0 tx=0).stop 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 shutdown_connections 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6fe80a2c00 0x7f6fe80a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ff8112b90 0x7f6ff819e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ff8074eb0 0x7f6ff81a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 --2- 172.21.15.137:0/3004899066 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff8074590 0x7f6ff81a4720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 >> 172.21.15.137:0/3004899066 conn(0x7f6ff80fd8c0 msgr2=0x7f6ff80ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 shutdown_connections 2024-09-12T12:11:15.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:15.138+0000 7f6ffef80640 1 -- 172.21.15.137:0/3004899066 wait complete. 2024-09-12T12:11:15.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:15 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] up:active 2024-09-12T12:11:15.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:15 smithi137 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:15.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:15 smithi137 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:15.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:15 smithi137 ceph-mon[30950]: pgmap v282: 97 pgs: 20 unknown, 77 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 220 B/s wr, 0 op/s 2024-09-12T12:11:15.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:15 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3004899066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:15.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:15 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] up:active 2024-09-12T12:11:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:15 smithi189 ceph-mon[35475]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:15 smithi189 ceph-mon[35475]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:15 smithi189 ceph-mon[35475]: pgmap v282: 97 pgs: 20 unknown, 77 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 220 B/s wr, 0 op/s 2024-09-12T12:11:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:15 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3004899066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:15.878 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:11:15.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:15 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] up:active 2024-09-12T12:11:15.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:15 smithi171 ceph-mon[35098]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:15.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:15 smithi171 ceph-mon[35098]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:15.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:15 smithi171 ceph-mon[35098]: pgmap v282: 97 pgs: 20 unknown, 77 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 220 B/s wr, 0 op/s 2024-09-12T12:11:15.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:15 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3004899066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:16.123 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1585620941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102ef0 msgr2=0x7fcae010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1585620941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102ef0 0x7fcae010da40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009ca0 tx=0x7fcad802a950 comp rx=0 tx=0).stop 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1585620941 shutdown_connections 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1585620941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102ef0 0x7fcae010da40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1585620941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcae0102570 0x7fcae01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1585620941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae0100e20 0x7fcae0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1585620941 >> 172.21.15.137:0/1585620941 conn(0x7fcae00fc7d0 msgr2=0x7fcae00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1585620941 shutdown_connections 2024-09-12T12:11:16.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1585620941 wait complete. 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 Processor -- start 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 -- start start 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae0100e20 0x7fcae01974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102570 0x7fcae0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.612+0000 7fcae7a0b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcae0102ef0 0x7fcae019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae7a0b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0111020 con 0x7fcae0102570 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae7a0b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcae0110ea0 con 0x7fcae0102ef0 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae7a0b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcae01111a0 con 0x7fcae0100e20 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102570 0x7fcae0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102570 0x7fcae0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33892/0 (socket says 172.21.15.137:33892) 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 -- 172.21.15.137:0/1057176413 learned_addr learned my addr 172.21.15.137:0/1057176413 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:16.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae57a7640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae0100e20 0x7fcae01974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae5fa8640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcae0102ef0 0x7fcae019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 -- 172.21.15.137:0/1057176413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae0100e20 msgr2=0x7fcae01974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae0100e20 0x7fcae01974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 -- 172.21.15.137:0/1057176413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcae0102ef0 msgr2=0x7fcae019bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcae0102ef0 0x7fcae019bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 -- 172.21.15.137:0/1057176413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae019c490 con 0x7fcae0102570 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae5fa8640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcae0102ef0 0x7fcae019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae57a7640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae0100e20 0x7fcae01974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae4fa6640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102570 0x7fcae0197a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcad000b4a0 tx=0x7fcad000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:16.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcad67fc640 1 -- 172.21.15.137:0/1057176413 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0014070 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae019c780 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcad67fc640 1 -- 172.21.15.137:0/1057176413 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad0010af0 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcad67fc640 1 -- 172.21.15.137:0/1057176413 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0017d10 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.613+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae01a4060 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.614+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcae0101280 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.615+0000 7fcad67fc640 1 -- 172.21.15.137:0/1057176413 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcad0017070 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.616+0000 7fcad67fc640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcacc0a2b80 0x7fcacc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.616+0000 7fcae57a7640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcacc0a2b80 0x7fcacc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.616+0000 7fcad67fc640 1 -- 172.21.15.137:0/1057176413 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fcad00ce7a0 con 0x7fcae0102570 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.617+0000 7fcae57a7640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcacc0a2b80 0x7fcacc0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:16.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.620+0000 7fcad67fc640 1 -- 172.21.15.137:0/1057176413 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcad009b7b0 con 0x7fcae0102570 2024-09-12T12:11:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.788+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcae0109090 con 0x7fcae0102570 2024-09-12T12:11:16.789 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:16 smithi137 ceph-mon[30950]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T12:11:16.790 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:16 smithi137 ceph-mon[30950]: Cluster is now healthy 2024-09-12T12:11:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.790+0000 7fcad67fc640 1 -- 172.21.15.137:0/1057176413 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22348 (secure 0 0 0) 0x7fcad009b0b0 con 0x7fcae0102570 2024-09-12T12:11:16.791 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:16.791 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:11:13.196473+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:11:16.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcacc0a2b80 msgr2=0x7fcacc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:16.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcacc0a2b80 0x7fcacc0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).stop 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102570 msgr2=0x7fcae0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102570 0x7fcae0197a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcad000b4a0 tx=0x7fcad000b960 comp rx=0 tx=0).stop 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 shutdown_connections 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcacc0a2b80 0x7fcacc0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcae0102ef0 0x7fcae019bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcae0102570 0x7fcae0197a20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 --2- 172.21.15.137:0/1057176413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcae0100e20 0x7fcae01974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 >> 172.21.15.137:0/1057176413 conn(0x7fcae00fc7d0 msgr2=0x7fcae00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:16.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 shutdown_connections 2024-09-12T12:11:16.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:16.792+0000 7fcae7a0b640 1 -- 172.21.15.137:0/1057176413 wait complete. 2024-09-12T12:11:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:16 smithi189 ceph-mon[35475]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T12:11:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:16 smithi189 ceph-mon[35475]: Cluster is now healthy 2024-09-12T12:11:16.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:16 smithi171 ceph-mon[35098]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T12:11:16.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:16 smithi171 ceph-mon[35098]: Cluster is now healthy 2024-09-12T12:11:17.480 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T12:11:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:17 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1057176413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:17 smithi137 ceph-mon[30950]: pgmap v283: 97 pgs: 97 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-09-12T12:11:17.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:17 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1057176413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:17.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:17 smithi171 ceph-mon[35098]: pgmap v283: 97 pgs: 97 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-09-12T12:11:17.700 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: 2024-09-12T12:11:17.308+0000 7fc9754f6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T12:11:17.727 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:17.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:17 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1057176413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:17.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:17 smithi189 ceph-mon[35475]: pgmap v283: 97 pgs: 97 active+clean; 577 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-09-12T12:11:18.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 -- 172.21.15.137:0/2842877438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac100a80 msgr2=0x7f8fac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:18.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2842877438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac100a80 0x7f8fac100ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009930 tx=0x7f8fa802a420 comp rx=0 tx=0).stop 2024-09-12T12:11:18.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 -- 172.21.15.137:0/2842877438 shutdown_connections 2024-09-12T12:11:18.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2842877438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fac101420 0x7f8fac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2842877438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac100a80 0x7f8fac100ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2842877438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fac106a80 0x7f8fac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 -- 172.21.15.137:0/2842877438 >> 172.21.15.137:0/2842877438 conn(0x7f8fac0fc7a0 msgr2=0x7f8fac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 -- 172.21.15.137:0/2842877438 shutdown_connections 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.202+0000 7f8fb47df640 1 -- 172.21.15.137:0/2842877438 wait complete. 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 Processor -- start 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 -- start start 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fac100a80 0x7f8fac1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fac101420 0x7f8fac197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac106a80 0x7f8fac19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac110ea0 con 0x7f8fac106a80 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac110d20 con 0x7f8fac101420 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb47df640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac111020 con 0x7f8fac100a80 2024-09-12T12:11:18.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb2d7c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac106a80 0x7f8fac19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb2d7c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac106a80 0x7f8fac19bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33910/0 (socket says 172.21.15.137:33910) 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb2d7c640 1 -- 172.21.15.137:0/2029216100 learned_addr learned my addr 172.21.15.137:0/2029216100 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.203+0000 7f8fb1d7a640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fac101420 0x7f8fac197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb257b640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fac100a80 0x7f8fac1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb2d7c640 1 -- 172.21.15.137:0/2029216100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fac100a80 msgr2=0x7f8fac1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb2d7c640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fac100a80 0x7f8fac1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb2d7c640 1 -- 172.21.15.137:0/2029216100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fac101420 msgr2=0x7f8fac197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb2d7c640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fac101420 0x7f8fac197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb2d7c640 1 -- 172.21.15.137:0/2029216100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac19c370 con 0x7f8fac106a80 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb2d7c640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac106a80 0x7f8fac19bc90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b970 tx=0x7f8fa400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8f9f7fe640 1 -- 172.21.15.137:0/2029216100 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4019070 con 0x7f8fac106a80 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8f9f7fe640 1 -- 172.21.15.137:0/2029216100 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa400cb40 con 0x7f8fac106a80 2024-09-12T12:11:18.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac19c660 con 0x7f8fac106a80 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8f9f7fe640 1 -- 172.21.15.137:0/2029216100 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4007650 con 0x7f8fac106a80 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.204+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1a3f40 con 0x7f8fac106a80 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.205+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fac100ee0 con 0x7f8fac106a80 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.206+0000 7f8f9f7fe640 1 -- 172.21.15.137:0/2029216100 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8fa401c070 con 0x7f8fac106a80 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.207+0000 7f8f9f7fe640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f8f980a2c00 0x7f8f980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.207+0000 7f8fb257b640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f8f980a2c00 0x7f8f980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.207+0000 7f8f9f7fe640 1 -- 172.21.15.137:0/2029216100 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f8fa40ce870 con 0x7f8fac106a80 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.207+0000 7f8fb257b640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f8f980a2c00 0x7f8f980a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005fd0 tx=0x7f8fa0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:18.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.210+0000 7f8f9f7fe640 1 -- 172.21.15.137:0/2029216100 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fa409b2d0 con 0x7f8fac106a80 2024-09-12T12:11:18.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.413+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8fac198720 con 0x7f8fac106a80 2024-09-12T12:11:18.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.515+0000 7f8f9f7fe640 1 -- 172.21.15.137:0/2029216100 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f8fac198720 con 0x7f8fac106a80 2024-09-12T12:11:18.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f8f980a2c00 msgr2=0x7f8f980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:18.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f8f980a2c00 0x7f8f980a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005fd0 tx=0x7f8fa0005850 comp rx=0 tx=0).stop 2024-09-12T12:11:18.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac106a80 msgr2=0x7f8fac19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac106a80 0x7f8fac19bc90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b970 tx=0x7f8fa400be30 comp rx=0 tx=0).stop 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 shutdown_connections 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f8f980a2c00 0x7f8f980a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8fac106a80 0x7f8fac19bc90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fac101420 0x7f8fac197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 --2- 172.21.15.137:0/2029216100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fac100a80 0x7f8fac1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 >> 172.21.15.137:0/2029216100 conn(0x7f8fac0fc7a0 msgr2=0x7f8fac0fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 shutdown_connections 2024-09-12T12:11:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:18.518+0000 7f8fb47df640 1 -- 172.21.15.137:0/2029216100 wait complete. 2024-09-12T12:11:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:18 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2029216100' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T12:11:18.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:18 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2029216100' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T12:11:18.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:18 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2029216100' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T12:11:19.220 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-12T12:11:19.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-12T12:11:19.488 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:19.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:19 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2029216100' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T12:11:19.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:19 smithi137 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:19.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:19 smithi137 ceph-mon[30950]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T12:11:19.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:19 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2029216100' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T12:11:19.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:19 smithi189 ceph-mon[35475]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:19.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:19 smithi189 ceph-mon[35475]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T12:11:19.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:19 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2029216100' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T12:11:19.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:19 smithi171 ceph-mon[35098]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:19.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:19 smithi171 ceph-mon[35098]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-12T12:11:20.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 -- 172.21.15.137:0/3785351606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 msgr2=0x7f3b24103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 --2- 172.21.15.137:0/3785351606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24103120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009900 tx=0x7f3b1402a480 comp rx=0 tx=0).stop 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 -- 172.21.15.137:0/3785351606 shutdown_connections 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 --2- 172.21.15.137:0/3785351606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b2410c170 0x7f3b2410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 --2- 172.21.15.137:0/3785351606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b24103660 0x7f3b2410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 --2- 172.21.15.137:0/3785351606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24103120 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 -- 172.21.15.137:0/3785351606 >> 172.21.15.137:0/3785351606 conn(0x7f3b240fc7a0 msgr2=0x7f3b240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 -- 172.21.15.137:0/3785351606 shutdown_connections 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.019+0000 7f3b29652640 1 -- 172.21.15.137:0/3785351606 wait complete. 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 Processor -- start 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 -- start start 2024-09-12T12:11:20.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b24103660 0x7f3b241989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b2410c170 0x7f3b2419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24111fb0 con 0x7f3b24102d40 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24111e30 con 0x7f3b2410c170 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b29652640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24112130 con 0x7f3b24103660 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b22ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b22ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55596/0 (socket says 172.21.15.137:55596) 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b22ffd640 1 -- 172.21.15.137:0/1682405947 learned_addr learned my addr 172.21.15.137:0/1682405947 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.020+0000 7f3b227fc640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b24103660 0x7f3b241989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b237fe640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b2410c170 0x7f3b2419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b22ffd640 1 -- 172.21.15.137:0/1682405947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b24103660 msgr2=0x7f3b241989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:20.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b22ffd640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b24103660 0x7f3b241989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b22ffd640 1 -- 172.21.15.137:0/1682405947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b2410c170 msgr2=0x7f3b2419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b22ffd640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b2410c170 0x7f3b2419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b22ffd640 1 -- 172.21.15.137:0/1682405947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2419d440 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b227fc640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b24103660 0x7f3b241989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b22ffd640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24198490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b14006010 tx=0x7f3b140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b0bfff640 1 -- 172.21.15.137:0/1682405947 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1403f030 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b0bfff640 1 -- 172.21.15.137:0/1682405947 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b14003ce0 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2419d6d0 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b0bfff640 1 -- 172.21.15.137:0/1682405947 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14035070 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.021+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b241a5010 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.022+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2406b070 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.023+0000 7f3b0bfff640 1 -- 172.21.15.137:0/1682405947 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3b14042030 con 0x7f3b24102d40 2024-09-12T12:11:20.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.024+0000 7f3b0bfff640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b0c0a2b80 0x7f3b0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:20.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.024+0000 7f3b227fc640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b0c0a2b80 0x7f3b0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:20.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.024+0000 7f3b0bfff640 1 -- 172.21.15.137:0/1682405947 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f3b140ef660 con 0x7f3b24102d40 2024-09-12T12:11:20.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.024+0000 7f3b227fc640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b0c0a2b80 0x7f3b0c0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b180058c0 tx=0x7f3b18005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:20.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.028+0000 7f3b0bfff640 1 -- 172.21.15.137:0/1682405947 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b140bc7d0 con 0x7f3b24102d40 2024-09-12T12:11:20.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.229+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f3b24107210 con 0x7f3b24102d40 2024-09-12T12:11:20.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.528+0000 7f3b0bfff640 1 -- 172.21.15.137:0/1682405947 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f3b140bc0d0 con 0x7f3b24102d40 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b0c0a2b80 msgr2=0x7f3b0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b0c0a2b80 0x7f3b0c0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3b180058c0 tx=0x7f3b18005850 comp rx=0 tx=0).stop 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 msgr2=0x7f3b24198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24198490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3b14006010 tx=0x7f3b140042f0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 shutdown_connections 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3b0c0a2b80 0x7f3b0c0a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b2410c170 0x7f3b2419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b24103660 0x7f3b241989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 --2- 172.21.15.137:0/1682405947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b24102d40 0x7f3b24198490 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.530+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 >> 172.21.15.137:0/1682405947 conn(0x7f3b240fc7a0 msgr2=0x7f3b240fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.531+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 shutdown_connections 2024-09-12T12:11:20.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:20.531+0000 7f3b29652640 1 -- 172.21.15.137:0/1682405947 wait complete. 2024-09-12T12:11:20.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:20 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1682405947' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T12:11:20.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:20 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1682405947' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T12:11:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:20 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1682405947' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-12T12:11:21.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T12:11:21.517 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:21.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:21 smithi137 ceph-mon[30950]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T12:11:21.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:21 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1682405947' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T12:11:21.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:21 smithi137 ceph-mon[30950]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T12:11:21.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:21 smithi137 ceph-mon[30950]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:21.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:21 smithi137 ceph-mon[30950]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T12:11:21.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:21 smithi137 ceph-mon[30950]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T12:11:21.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:21 smithi137 ceph-mon[30950]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T12:11:21.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:21 smithi189 ceph-mon[35475]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T12:11:21.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:21 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1682405947' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T12:11:21.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:21 smithi189 ceph-mon[35475]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T12:11:21.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:21 smithi189 ceph-mon[35475]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:21.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:21 smithi189 ceph-mon[35475]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T12:11:21.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:21 smithi189 ceph-mon[35475]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T12:11:21.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:21 smithi189 ceph-mon[35475]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T12:11:21.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:21 smithi171 ceph-mon[35098]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T12:11:21.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:21 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1682405947' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-12T12:11:21.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:21 smithi171 ceph-mon[35098]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T12:11:21.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:21 smithi171 ceph-mon[35098]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-12T12:11:21.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:21 smithi171 ceph-mon[35098]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-12T12:11:21.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:21 smithi171 ceph-mon[35098]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-12T12:11:21.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:21 smithi171 ceph-mon[35098]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 -- 172.21.15.137:0/4285347071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c103660 msgr2=0x7fdb2c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 --2- 172.21.15.137:0/4285347071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c103660 0x7fdb2c10bb50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009900 tx=0x7fdb2802a480 comp rx=0 tx=0).stop 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 -- 172.21.15.137:0/4285347071 shutdown_connections 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 --2- 172.21.15.137:0/4285347071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb2c10c170 0x7fdb2c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 --2- 172.21.15.137:0/4285347071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c103660 0x7fdb2c10bb50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 --2- 172.21.15.137:0/4285347071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb2c102d40 0x7fdb2c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 -- 172.21.15.137:0/4285347071 >> 172.21.15.137:0/4285347071 conn(0x7fdb2c0fc7a0 msgr2=0x7fdb2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 -- 172.21.15.137:0/4285347071 shutdown_connections 2024-09-12T12:11:22.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 -- 172.21.15.137:0/4285347071 wait complete. 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.993+0000 7fdb34323640 1 Processor -- start 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 -- start start 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb2c102d40 0x7fdb2c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb2c103660 0x7fdb2c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c10c170 0x7fdb2c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c111f70 con 0x7fdb2c10c170 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c111df0 con 0x7fdb2c102d40 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c1120f0 con 0x7fdb2c103660 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c10c170 0x7fdb2c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c10c170 0x7fdb2c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55614/0 (socket says 172.21.15.137:55614) 2024-09-12T12:11:22.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 -- 172.21.15.137:0/3239610897 learned_addr learned my addr 172.21.15.137:0/3239610897 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb320bf640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb2c102d40 0x7fdb2c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 -- 172.21.15.137:0/3239610897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb2c103660 msgr2=0x7fdb2c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb2c103660 0x7fdb2c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 -- 172.21.15.137:0/3239610897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb2c102d40 msgr2=0x7fdb2c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb2c102d40 0x7fdb2c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 -- 172.21.15.137:0/3239610897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c19d3c0 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb328c0640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c10c170 0x7fdb2c19cce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400cce0 tx=0x7fdb2400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb1f7fe640 1 -- 172.21.15.137:0/3239610897 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2400f450 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb1f7fe640 1 -- 172.21.15.137:0/3239610897 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2400fad0 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2c19d6b0 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.994+0000 7fdb1f7fe640 1 -- 172.21.15.137:0/3239610897 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24018c10 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.995+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2c1a4f90 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.995+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2c103120 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.997+0000 7fdb1f7fe640 1 -- 172.21.15.137:0/3239610897 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdb2400f5f0 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.997+0000 7fdb1f7fe640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdb180a2b80 0x7fdb180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.997+0000 7fdb320bf640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdb180a2b80 0x7fdb180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.997+0000 7fdb1f7fe640 1 -- 172.21.15.137:0/3239610897 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fdb240cf5e0 con 0x7fdb2c10c170 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:21.998+0000 7fdb320bf640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdb180a2b80 0x7fdb180a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb200058c0 tx=0x7fdb2000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:22.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.001+0000 7fdb1f7fe640 1 -- 172.21.15.137:0/3239610897 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb24014030 con 0x7fdb2c10c170 2024-09-12T12:11:22.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.206+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fdb2c02d7c0 con 0x7fdb2c10c170 2024-09-12T12:11:22.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.458+0000 7fdb1f7fe640 1 -- 172.21.15.137:0/3239610897 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fdb2409c690 con 0x7fdb2c10c170 2024-09-12T12:11:22.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.460+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdb180a2b80 msgr2=0x7fdb180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:22.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.460+0000 7fdb34323640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdb180a2b80 0x7fdb180a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb200058c0 tx=0x7fdb2000a4c0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.460+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c10c170 msgr2=0x7fdb2c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:22.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.460+0000 7fdb34323640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c10c170 0x7fdb2c19cce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400cce0 tx=0x7fdb2400a620 comp rx=0 tx=0).stop 2024-09-12T12:11:22.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 shutdown_connections 2024-09-12T12:11:22.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdb180a2b80 0x7fdb180a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb2c10c170 0x7fdb2c19cce0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb2c103660 0x7fdb2c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 --2- 172.21.15.137:0/3239610897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb2c102d40 0x7fdb2c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 >> 172.21.15.137:0/3239610897 conn(0x7fdb2c0fc7a0 msgr2=0x7fdb2c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 shutdown_connections 2024-09-12T12:11:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:22.461+0000 7fdb34323640 1 -- 172.21.15.137:0/3239610897 wait complete. 2024-09-12T12:11:22.700 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: 2024-09-12T12:11:22.308+0000 7fc9754f6640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T12:11:22.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] up:active 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: Cluster is now healthy 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3239610897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T12:11:22.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3239610897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T12:11:22.702 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] up:active 2024-09-12T12:11:22.702 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:22 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:11:22.757 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] up:active 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: Cluster is now healthy 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3239610897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3239610897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] up:active 2024-09-12T12:11:22.758 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:22 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] up:active 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: Cluster is now healthy 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3239610897' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3239610897' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] up:active 2024-09-12T12:11:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:22 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:11:23.118 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T12:11:23.364 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:23.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:23 smithi137 ceph-mon[30950]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.6 KiB/s wr, 7 op/s 2024-09-12T12:11:23.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:23 smithi189 ceph-mon[35475]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.6 KiB/s wr, 7 op/s 2024-09-12T12:11:23.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 -- 172.21.15.137:0/1886974957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 msgr2=0x7f9084100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1886974957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084100ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9074009930 tx=0x7f907402a420 comp rx=0 tx=0).stop 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 -- 172.21.15.137:0/1886974957 shutdown_connections 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1886974957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9084101420 0x7f908410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1886974957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084100ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1886974957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9084106a80 0x7f9084106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 -- 172.21.15.137:0/1886974957 >> 172.21.15.137:0/1886974957 conn(0x7f90840fc7a0 msgr2=0x7f90840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 -- 172.21.15.137:0/1886974957 shutdown_connections 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.885+0000 7f908c5d7640 1 -- 172.21.15.137:0/1886974957 wait complete. 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 Processor -- start 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 -- start start 2024-09-12T12:11:23.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9084101420 0x7f9084199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9084106a80 0x7f90841a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9084110ea0 con 0x7f9084100a80 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9084110d20 con 0x7f9084106a80 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908c5d7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9084111020 con 0x7f9084101420 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55624/0 (socket says 172.21.15.137:55624) 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 -- 172.21.15.137:0/1818345980 learned_addr learned my addr 172.21.15.137:0/1818345980 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 -- 172.21.15.137:0/1818345980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9084101420 msgr2=0x7f9084199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9084101420 0x7f9084199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908ab74640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9084106a80 0x7f90841a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:23.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 -- 172.21.15.137:0/1818345980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9084106a80 msgr2=0x7f90841a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9084106a80 0x7f90841a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.886+0000 7f908a373640 1 -- 172.21.15.137:0/1818345980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90841a48d0 con 0x7f9084100a80 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.887+0000 7f908a373640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084199310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f908000b4a0 tx=0x7f908000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.887+0000 7f907b7fe640 1 -- 172.21.15.137:0/1818345980 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080014070 con 0x7f9084100a80 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.887+0000 7f907b7fe640 1 -- 172.21.15.137:0/1818345980 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9080010af0 con 0x7f9084100a80 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.887+0000 7f907b7fe640 1 -- 172.21.15.137:0/1818345980 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080017d10 con 0x7f9084100a80 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.887+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9084197100 con 0x7f9084100a80 2024-09-12T12:11:23.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.887+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9084197690 con 0x7f9084100a80 2024-09-12T12:11:23.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.888+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9084100ee0 con 0x7f9084100a80 2024-09-12T12:11:23.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.889+0000 7f907b7fe640 1 -- 172.21.15.137:0/1818345980 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9080017070 con 0x7f9084100a80 2024-09-12T12:11:23.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.890+0000 7f907b7fe640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f90700a2c00 0x7f90700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:23.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.890+0000 7f9089b72640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f90700a2c00 0x7f90700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:23.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.890+0000 7f907b7fe640 1 -- 172.21.15.137:0/1818345980 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f90800ce290 con 0x7f9084100a80 2024-09-12T12:11:23.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.890+0000 7f9089b72640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f90700a2c00 0x7f90700a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f907402a940 tx=0x7f9074000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:23.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:23.894+0000 7f907b7fe640 1 -- 172.21.15.137:0/1818345980 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f908009b340 con 0x7f9084100a80 2024-09-12T12:11:23.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:23 smithi171 ceph-mon[35098]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.6 KiB/s wr, 7 op/s 2024-09-12T12:11:24.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.093+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f908406adc0 con 0x7f9084100a80 2024-09-12T12:11:24.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.554+0000 7f907b7fe640 1 -- 172.21.15.137:0/1818345980 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7f908009ac40 con 0x7f9084100a80 2024-09-12T12:11:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f90700a2c00 msgr2=0x7f90700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f90700a2c00 0x7f90700a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f907402a940 tx=0x7f9074000bc0 comp rx=0 tx=0).stop 2024-09-12T12:11:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 msgr2=0x7f9084199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084199310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f908000b4a0 tx=0x7f908000b960 comp rx=0 tx=0).stop 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 shutdown_connections 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f90700a2c00 0x7f90700a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9084106a80 0x7f90841a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9084101420 0x7f9084199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 --2- 172.21.15.137:0/1818345980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084100a80 0x7f9084199310 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 >> 172.21.15.137:0/1818345980 conn(0x7f90840fc7a0 msgr2=0x7f90840fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 shutdown_connections 2024-09-12T12:11:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:24.557+0000 7f908c5d7640 1 -- 172.21.15.137:0/1818345980 wait complete. 2024-09-12T12:11:24.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:24 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1818345980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T12:11:24.851 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:24 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1818345980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T12:11:24.883 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:24 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1818345980' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T12:11:25.244 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-12T12:11:25.245 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T12:11:25.499 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:25.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:25 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1818345980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T12:11:25.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:25 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:11:25.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:25 smithi189 ceph-mon[35475]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-12T12:11:25.869 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:25 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1818345980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T12:11:25.869 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:25 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:11:25.869 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:25 smithi171 ceph-mon[35098]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-12T12:11:25.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:25 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1818345980' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T12:11:25.905 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:25 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:11:25.905 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:25 smithi137 ceph-mon[30950]: pgmap v287: 97 pgs: 97 active+clean; 581 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-12T12:11:25.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 -- 172.21.15.137:0/1297202344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8106a90 msgr2=0x7f72c8106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:25.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1297202344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8106a90 0x7f72c8106e70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009930 tx=0x7f72c402a420 comp rx=0 tx=0).stop 2024-09-12T12:11:25.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 -- 172.21.15.137:0/1297202344 shutdown_connections 2024-09-12T12:11:25.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1297202344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8101430 0x7f72c810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:25.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1297202344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72c8100a90 0x7f72c8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:25.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1297202344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8106a90 0x7f72c8106e70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:25.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 -- 172.21.15.137:0/1297202344 >> 172.21.15.137:0/1297202344 conn(0x7f72c80fc7d0 msgr2=0x7f72c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:25.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 -- 172.21.15.137:0/1297202344 shutdown_connections 2024-09-12T12:11:25.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 -- 172.21.15.137:0/1297202344 wait complete. 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 Processor -- start 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.986+0000 7f72d00ab640 1 -- start start 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72d00ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8100a90 0x7f72c8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72d00ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8101430 0x7f72c81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72d00ab640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72c8106a90 0x7f72c819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72d00ab640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8111fb0 con 0x7f72c8100a90 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72d00ab640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8111e30 con 0x7f72c8106a90 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72d00ab640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8112130 con 0x7f72c8101430 2024-09-12T12:11:25.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8100a90 0x7f72c8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8100a90 0x7f72c8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55644/0 (socket says 172.21.15.137:55644) 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 -- 172.21.15.137:0/1672060378 learned_addr learned my addr 172.21.15.137:0/1672060378 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72ce648640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72c8106a90 0x7f72c819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cd646640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8101430 0x7f72c81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 -- 172.21.15.137:0/1672060378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8101430 msgr2=0x7f72c81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8101430 0x7f72c81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 -- 172.21.15.137:0/1672060378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72c8106a90 msgr2=0x7f72c819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72c8106a90 0x7f72c819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 -- 172.21.15.137:0/1672060378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c819d440 con 0x7f72c8100a90 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72cde47640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8100a90 0x7f72c8198490 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009230 tx=0x7f72c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72baffd640 1 -- 172.21.15.137:0/1672060378 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c403f030 con 0x7f72c8100a90 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.987+0000 7f72baffd640 1 -- 172.21.15.137:0/1672060378 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c4003ad0 con 0x7f72c8100a90 2024-09-12T12:11:25.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.988+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c819d6d0 con 0x7f72c8100a90 2024-09-12T12:11:25.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.988+0000 7f72baffd640 1 -- 172.21.15.137:0/1672060378 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4035070 con 0x7f72c8100a90 2024-09-12T12:11:25.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.988+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c81a5010 con 0x7f72c8100a90 2024-09-12T12:11:25.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.989+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c8100ef0 con 0x7f72c8100a90 2024-09-12T12:11:25.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.990+0000 7f72baffd640 1 -- 172.21.15.137:0/1672060378 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f72c4042030 con 0x7f72c8100a90 2024-09-12T12:11:25.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.990+0000 7f72baffd640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72b40a2b80 0x7f72b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:25.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.990+0000 7f72cd646640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72b40a2b80 0x7f72b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:25.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.990+0000 7f72baffd640 1 -- 172.21.15.137:0/1672060378 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f72c40ef400 con 0x7f72c8100a90 2024-09-12T12:11:25.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.991+0000 7f72cd646640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72b40a2b80 0x7f72b40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005fd0 tx=0x7f72bc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:25.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:25.994+0000 7f72baffd640 1 -- 172.21.15.137:0/1672060378 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72c40bc4b0 con 0x7f72c8100a90 2024-09-12T12:11:26.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.183+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f72c8062630 con 0x7f72b40a2b80 2024-09-12T12:11:26.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.206+0000 7f72baffd640 1 -- 172.21.15.137:0/1672060378 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f72c8062630 con 0x7f72b40a2b80 2024-09-12T12:11:26.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72b40a2b80 msgr2=0x7f72b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:26.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72b40a2b80 0x7f72b40a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72bc005fd0 tx=0x7f72bc005850 comp rx=0 tx=0).stop 2024-09-12T12:11:26.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8100a90 msgr2=0x7f72c8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:26.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8100a90 0x7f72c8198490 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009230 tx=0x7f72c4004270 comp rx=0 tx=0).stop 2024-09-12T12:11:26.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 shutdown_connections 2024-09-12T12:11:26.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72b40a2b80 0x7f72b40a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:26.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72c8106a90 0x7f72c819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:26.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72c8101430 0x7f72c81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:26.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 --2- 172.21.15.137:0/1672060378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72c8100a90 0x7f72c8198490 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:26.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 >> 172.21.15.137:0/1672060378 conn(0x7f72c80fc7d0 msgr2=0x7f72c80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:26.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 shutdown_connections 2024-09-12T12:11:26.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:26.208+0000 7f72d00ab640 1 -- 172.21.15.137:0/1672060378 wait complete. 2024-09-12T12:11:26.534 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:11:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:11:26.189+0000 7f62fa7e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:11:26.534 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:11:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:11:26.189+0000 7f62fa7e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:11:26.534 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:11:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:11:26.189+0000 7f62fa7e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:11:26.534 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:11:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:11:26.189+0000 7f62fa7e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:11:26.535 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:11:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:11:26.189+0000 7f62fa7e8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:11:26.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:26 smithi137 ceph-mon[30950]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:26.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:26 smithi189 ceph-mon[35475]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:26.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:26 smithi171 ceph-mon[35098]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:26.963 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T12:11:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:27.605 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:27 smithi137 ceph-mon[30950]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-12T12:11:27.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.690+0000 7f0255390640 1 -- 172.21.15.137:0/2597040261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250102d30 msgr2=0x7f0250103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:27.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.690+0000 7f0255390640 1 --2- 172.21.15.137:0/2597040261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250102d30 0x7f0250103110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0240009930 tx=0x7f024002a440 comp rx=0 tx=0).stop 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 -- 172.21.15.137:0/2597040261 shutdown_connections 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 --2- 172.21.15.137:0/2597040261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02501082a0 0x7f025010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 --2- 172.21.15.137:0/2597040261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02501036e0 0x7f0250107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 --2- 172.21.15.137:0/2597040261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0250102d30 0x7f0250103110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 -- 172.21.15.137:0/2597040261 >> 172.21.15.137:0/2597040261 conn(0x7f02500fc7d0 msgr2=0x7f02500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 -- 172.21.15.137:0/2597040261 shutdown_connections 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 -- 172.21.15.137:0/2597040261 wait complete. 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 Processor -- start 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.691+0000 7f0255390640 1 -- start start 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f0255390640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0250102d30 0x7f0250100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f0255390640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02501036e0 0x7f0250100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f0255390640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501082a0 0x7f0250101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:27.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f0255390640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f02501120e0 con 0x7f02501082a0 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f0255390640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111f60 con 0x7f02501036e0 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f0255390640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0250112260 con 0x7f0250102d30 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501082a0 0x7f0250101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501082a0 0x7f0250101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55682/0 (socket says 172.21.15.137:55682) 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 -- 172.21.15.137:0/3982980114 learned_addr learned my addr 172.21.15.137:0/3982980114 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f02467fc640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02501036e0 0x7f0250100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024effd640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0250102d30 0x7f0250100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 -- 172.21.15.137:0/3982980114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0250102d30 msgr2=0x7f0250100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0250102d30 0x7f0250100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 -- 172.21.15.137:0/3982980114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02501036e0 msgr2=0x7f0250100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02501036e0 0x7f0250100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 -- 172.21.15.137:0/3982980114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02501a6b10 con 0x7f02501082a0 2024-09-12T12:11:27.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024effd640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0250102d30 0x7f0250100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.692+0000 7f024f7fe640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501082a0 0x7f0250101410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f024800cad0 tx=0x7f024800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.693+0000 7f024cff9640 1 -- 172.21.15.137:0/3982980114 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800ad60 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.693+0000 7f024cff9640 1 -- 172.21.15.137:0/3982980114 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0248005020 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.693+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02501a6e00 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.693+0000 7f024cff9640 1 -- 172.21.15.137:0/3982980114 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024800f6b0 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.693+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a72c0 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.694+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025006a850 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.695+0000 7f024cff9640 1 -- 172.21.15.137:0/3982980114 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0248005560 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.695+0000 7f024cff9640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f023c0a2c10 0x7f023c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.695+0000 7f024effd640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f023c0a2c10 0x7f023c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.695+0000 7f024cff9640 1 -- 172.21.15.137:0/3982980114 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f02480cf0a0 con 0x7f02501082a0 2024-09-12T12:11:27.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.696+0000 7f024effd640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f023c0a2c10 0x7f023c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0240000c00 tx=0x7f02400058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:27.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.699+0000 7f024cff9640 1 -- 172.21.15.137:0/3982980114 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0248014030 con 0x7f02501082a0 2024-09-12T12:11:27.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:27 smithi189 ceph-mon[35475]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-12T12:11:27.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.891+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0250062630 con 0x7f023c0a2c10 2024-09-12T12:11:27.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.914+0000 7f024cff9640 1 -- 172.21.15.137:0/3982980114 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0250062630 con 0x7f023c0a2c10 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f023c0a2c10 msgr2=0x7f023c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f023c0a2c10 0x7f023c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0240000c00 tx=0x7f02400058b0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501082a0 msgr2=0x7f0250101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501082a0 0x7f0250101410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f024800cad0 tx=0x7f024800ced0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 shutdown_connections 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f023c0a2c10 0x7f023c0a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02501082a0 0x7f0250101410 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02501036e0 0x7f0250100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 --2- 172.21.15.137:0/3982980114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0250102d30 0x7f0250100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 >> 172.21.15.137:0/3982980114 conn(0x7f02500fc7d0 msgr2=0x7f0250105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 shutdown_connections 2024-09-12T12:11:27.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:27.916+0000 7f0255390640 1 -- 172.21.15.137:0/3982980114 wait complete. 2024-09-12T12:11:27.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:27 smithi171 ceph-mon[35098]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-12T12:11:28.574 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T12:11:28.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:28 smithi137 ceph-mon[30950]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T12:11:28.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:28 smithi137 ceph-mon[30950]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:28.818 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:28.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:28 smithi189 ceph-mon[35475]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T12:11:28.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:28 smithi189 ceph-mon[35475]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:28.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:28 smithi171 ceph-mon[35098]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T12:11:28.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:28 smithi171 ceph-mon[35098]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:29.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 -- 172.21.15.137:0/4187991597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102ef0 msgr2=0x7f045410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:29.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 --2- 172.21.15.137:0/4187991597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102ef0 0x7f045410da40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f044c009ca0 tx=0x7f044c02a950 comp rx=0 tx=0).stop 2024-09-12T12:11:29.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 -- 172.21.15.137:0/4187991597 shutdown_connections 2024-09-12T12:11:29.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 --2- 172.21.15.137:0/4187991597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102ef0 0x7f045410da40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 --2- 172.21.15.137:0/4187991597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0454102570 0x7f04541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 --2- 172.21.15.137:0/4187991597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454100e20 0x7f0454101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 -- 172.21.15.137:0/4187991597 >> 172.21.15.137:0/4187991597 conn(0x7f04540fc7d0 msgr2=0x7f04540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 -- 172.21.15.137:0/4187991597 shutdown_connections 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.303+0000 7f0458ddc640 1 -- 172.21.15.137:0/4187991597 wait complete. 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 Processor -- start 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 -- start start 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0454100e20 0x7f04541994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102570 0x7f04541999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454102ef0 0x7f04541a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0454111020 con 0x7f0454102570 2024-09-12T12:11:29.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0454110ea0 con 0x7f0454102ef0 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0458ddc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04541111a0 con 0x7f0454100e20 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102570 0x7f04541999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102570 0x7f04541999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36968/0 (socket says 172.21.15.137:36968) 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 -- 172.21.15.137:0/3390080204 learned_addr learned my addr 172.21.15.137:0/3390080204 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f0452d9d640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454102ef0 0x7f04541a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 -- 172.21.15.137:0/3390080204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0454100e20 msgr2=0x7f04541994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f045259c640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0454100e20 0x7f04541994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0454100e20 0x7f04541994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 -- 172.21.15.137:0/3390080204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454102ef0 msgr2=0x7f04541a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454102ef0 0x7f04541a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.304+0000 7f044bfff640 1 -- 172.21.15.137:0/3390080204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04541a4a70 con 0x7f0454102570 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.305+0000 7f044bfff640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102570 0x7f04541999f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f043c00b4a0 tx=0x7f043c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:29.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.305+0000 7f044b7fe640 1 -- 172.21.15.137:0/3390080204 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c014070 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.305+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04541a4c40 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.305+0000 7f044b7fe640 1 -- 172.21.15.137:0/3390080204 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043c010af0 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.305+0000 7f044b7fe640 1 -- 172.21.15.137:0/3390080204 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c017d10 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.305+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04541a5150 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.306+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04541021b0 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.307+0000 7f044b7fe640 1 -- 172.21.15.137:0/3390080204 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f043c017070 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.308+0000 7f044b7fe640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f04400a2c10 0x7f04400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.308+0000 7f045259c640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f04400a2c10 0x7f04400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.308+0000 7f044b7fe640 1 -- 172.21.15.137:0/3390080204 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f043c0ce740 con 0x7f0454102570 2024-09-12T12:11:29.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.308+0000 7f045259c640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f04400a2c10 0x7f04400a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f044400a810 tx=0x7f0444008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:29.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.311+0000 7f044b7fe640 1 -- 172.21.15.137:0/3390080204 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f043c09b800 con 0x7f0454102570 2024-09-12T12:11:29.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.504+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0454109100 con 0x7f04400a2c10 2024-09-12T12:11:29.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.509+0000 7f044b7fe640 1 -- 172.21.15.137:0/3390080204 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f044400a400 con 0x7f04400a2c10 2024-09-12T12:11:29.510 INFO:teuthology.orchestra.run.smithi137.stdout:/volumes/qa/sv_0/7a71eae9-a34f-4c2c-829e-2887237ac94e 2024-09-12T12:11:29.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f04400a2c10 msgr2=0x7f04400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f04400a2c10 0x7f04400a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f044400a810 tx=0x7f0444008040 comp rx=0 tx=0).stop 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102570 msgr2=0x7f04541999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102570 0x7f04541999f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f043c00b4a0 tx=0x7f043c00b960 comp rx=0 tx=0).stop 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 shutdown_connections 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f04400a2c10 0x7f04400a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0454102ef0 0x7f04541a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0454102570 0x7f04541999f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 --2- 172.21.15.137:0/3390080204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0454100e20 0x7f04541994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:29.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 >> 172.21.15.137:0/3390080204 conn(0x7f04540fc7d0 msgr2=0x7f04540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:29.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 shutdown_connections 2024-09-12T12:11:29.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:29.511+0000 7f0458ddc640 1 -- 172.21.15.137:0/3390080204 wait complete. 2024-09-12T12:11:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:29 smithi137 ceph-mon[30950]: pgmap v289: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-12T12:11:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:29 smithi189 ceph-mon[35475]: pgmap v289: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-12T12:11:29.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:29 smithi171 ceph-mon[35098]: pgmap v289: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-09-12T12:11:30.331 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T12:11:30.576 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:30.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:30 smithi189 ceph-mon[35475]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:30.851 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:30 smithi137 ceph-mon[30950]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:30.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:30 smithi171 ceph-mon[35098]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 -- 172.21.15.137:0/1563521922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8101df0 msgr2=0x7fdef81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/1563521922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8101df0 0x7fdef81021d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009900 tx=0x7fdee802a410 comp rx=0 tx=0).stop 2024-09-12T12:11:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 -- 172.21.15.137:0/1563521922 shutdown_connections 2024-09-12T12:11:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/1563521922 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdef8106f20 0x7fdef810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/1563521922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef81027a0 0x7fdef81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/1563521922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8101df0 0x7fdef81021d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 -- 172.21.15.137:0/1563521922 >> 172.21.15.137:0/1563521922 conn(0x7fdef80fd8b0 msgr2=0x7fdef80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 -- 172.21.15.137:0/1563521922 shutdown_connections 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.031+0000 7fdefcf8d640 1 -- 172.21.15.137:0/1563521922 wait complete. 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 Processor -- start 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 -- start start 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef8101df0 0x7fdef8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef81027a0 0x7fdef81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdef8106f20 0x7fdef819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdef8111fc0 con 0x7fdef81027a0 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdef8111e40 con 0x7fdef8106f20 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.032+0000 7fdefcf8d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdef8112140 con 0x7fdef8101df0 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef81027a0 0x7fdef81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:31.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef81027a0 0x7fdef81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36974/0 (socket says 172.21.15.137:36974) 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 -- 172.21.15.137:0/2931297248 learned_addr learned my addr 172.21.15.137:0/2931297248 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdef659c640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef8101df0 0x7fdef8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdef6d9d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdef8106f20 0x7fdef819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 -- 172.21.15.137:0/2931297248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef8101df0 msgr2=0x7fdef8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef8101df0 0x7fdef8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 -- 172.21.15.137:0/2931297248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdef8106f20 msgr2=0x7fdef819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdef8106f20 0x7fdef819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 -- 172.21.15.137:0/2931297248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef819d440 con 0x7fdef81027a0 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdef6d9d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdef8106f20 0x7fdef819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeeffff640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef81027a0 0x7fdef81989d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdee000d770 tx=0x7fdee000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeef7fe640 1 -- 172.21.15.137:0/2931297248 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0017070 con 0x7fdef81027a0 2024-09-12T12:11:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeef7fe640 1 -- 172.21.15.137:0/2931297248 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee0005500 con 0x7fdef81027a0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef819d730 con 0x7fdef81027a0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdeef7fe640 1 -- 172.21.15.137:0/2931297248 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004e00 con 0x7fdef81027a0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.033+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef81a5010 con 0x7fdef81027a0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.034+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef81021d0 con 0x7fdef81027a0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.035+0000 7fdeef7fe640 1 -- 172.21.15.137:0/2931297248 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdee0004fa0 con 0x7fdef81027a0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.036+0000 7fdeef7fe640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdee40a2c10 0x7fdee40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.036+0000 7fdef659c640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdee40a2c10 0x7fdee40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.036+0000 7fdeef7fe640 1 -- 172.21.15.137:0/2931297248 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fdee00ceb50 con 0x7fdef81027a0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.037+0000 7fdef659c640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdee40a2c10 0x7fdee40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009230 tx=0x7fdee80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.040+0000 7fdeef7fe640 1 -- 172.21.15.137:0/2931297248 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdee009ba00 con 0x7fdef81027a0 2024-09-12T12:11:31.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.232+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdef8062630 con 0x7fdee40a2c10 2024-09-12T12:11:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.257+0000 7fdeef7fe640 1 -- 172.21.15.137:0/2931297248 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdef8062630 con 0x7fdee40a2c10 2024-09-12T12:11:31.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdee40a2c10 msgr2=0x7fdee40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:31.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdee40a2c10 0x7fdee40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009230 tx=0x7fdee80058d0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef81027a0 msgr2=0x7fdef81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef81027a0 0x7fdef81989d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdee000d770 tx=0x7fdee000dc30 comp rx=0 tx=0).stop 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 shutdown_connections 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdee40a2c10 0x7fdee40a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdef8106f20 0x7fdef819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef81027a0 0x7fdef81989d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 --2- 172.21.15.137:0/2931297248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef8101df0 0x7fdef8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 >> 172.21.15.137:0/2931297248 conn(0x7fdef80fd8b0 msgr2=0x7fdef80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 shutdown_connections 2024-09-12T12:11:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:31.259+0000 7fdefcf8d640 1 -- 172.21.15.137:0/2931297248 wait complete. 2024-09-12T12:11:31.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:31 smithi189 ceph-mon[35475]: pgmap v290: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T12:11:31.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:31 smithi189 ceph-mon[35475]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:31.855 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:31 smithi137 ceph-mon[30950]: pgmap v290: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T12:11:31.855 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:31 smithi137 ceph-mon[30950]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:31.926 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T12:11:31.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:31 smithi171 ceph-mon[35098]: pgmap v290: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-09-12T12:11:31.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:31 smithi171 ceph-mon[35098]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:32.172 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:32.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 -- 172.21.15.137:0/194558140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101420 msgr2=0x7f9b0010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:32.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 --2- 172.21.15.137:0/194558140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101420 0x7f9b0010d950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9af8009cd0 tx=0x7f9af802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 -- 172.21.15.137:0/194558140 shutdown_connections 2024-09-12T12:11:32.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 --2- 172.21.15.137:0/194558140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00101420 0x7f9b0010d950 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 --2- 172.21.15.137:0/194558140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b00100a80 0x7f9b00100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 --2- 172.21.15.137:0/194558140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b00106a80 0x7f9b00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 -- 172.21.15.137:0/194558140 >> 172.21.15.137:0/194558140 conn(0x7f9b000fc7a0 msgr2=0x7f9b000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:32.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 -- 172.21.15.137:0/194558140 shutdown_connections 2024-09-12T12:11:32.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.718+0000 7f9b086e6640 1 -- 172.21.15.137:0/194558140 wait complete. 2024-09-12T12:11:32.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 Processor -- start 2024-09-12T12:11:32.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 -- start start 2024-09-12T12:11:32.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00100a80 0x7f9b00198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:32.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b00101420 0x7f9b001989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:32.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b00106a80 0x7f9b0019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00110ea0 con 0x7f9b00100a80 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00110d20 con 0x7f9b00106a80 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b086e6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00111020 con 0x7f9b00101420 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b06482640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00100a80 0x7f9b00198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b06482640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00100a80 0x7f9b00198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36984/0 (socket says 172.21.15.137:36984) 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b06482640 1 -- 172.21.15.137:0/2705654347 learned_addr learned my addr 172.21.15.137:0/2705654347 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b06c83640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b00106a80 0x7f9b0019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.719+0000 7f9b05c81640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b00101420 0x7f9b001989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b06482640 1 -- 172.21.15.137:0/2705654347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b00101420 msgr2=0x7f9b001989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:32.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b06482640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b00101420 0x7f9b001989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b06482640 1 -- 172.21.15.137:0/2705654347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b00106a80 msgr2=0x7f9b0019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b06482640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b00106a80 0x7f9b0019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b06482640 1 -- 172.21.15.137:0/2705654347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0019d420 con 0x7f9b00100a80 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b05c81640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b00101420 0x7f9b001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b06c83640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b00106a80 0x7f9b0019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b06482640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00100a80 0x7f9b00198470 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b4a0 tx=0x7f9afc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9af77fe640 1 -- 172.21.15.137:0/2705654347 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc014070 con 0x7f9b00100a80 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9af77fe640 1 -- 172.21.15.137:0/2705654347 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc010af0 con 0x7f9b00100a80 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0019d710 con 0x7f9b00100a80 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9af77fe640 1 -- 172.21.15.137:0/2705654347 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc017d10 con 0x7f9b00100a80 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.720+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b001a4ff0 con 0x7f9b00100a80 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.721+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b00100ee0 con 0x7f9b00100a80 2024-09-12T12:11:32.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.722+0000 7f9af77fe640 1 -- 172.21.15.137:0/2705654347 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9afc017070 con 0x7f9b00100a80 2024-09-12T12:11:32.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.722+0000 7f9af77fe640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9aec0a2c10 0x7f9aec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:32.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.723+0000 7f9b05c81640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9aec0a2c10 0x7f9aec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:32.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.723+0000 7f9af77fe640 1 -- 172.21.15.137:0/2705654347 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f9afc0ce490 con 0x7f9b00100a80 2024-09-12T12:11:32.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.723+0000 7f9b05c81640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9aec0a2c10 0x7f9aec0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b00105120 tx=0x7f9af0005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:32.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.727+0000 7f9af77fe640 1 -- 172.21.15.137:0/2705654347 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9afc09b450 con 0x7f9b00100a80 2024-09-12T12:11:32.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.923+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f9b00062630 con 0x7f9aec0a2c10 2024-09-12T12:11:32.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.927+0000 7f9af77fe640 1 -- 172.21.15.137:0/2705654347 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9b00062630 con 0x7f9aec0a2c10 2024-09-12T12:11:32.928 INFO:teuthology.orchestra.run.smithi137.stdout:/volumes/qa/sv_1/3a65211d-c5a8-489e-9665-4910f9a11529 2024-09-12T12:11:32.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9aec0a2c10 msgr2=0x7f9aec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:32.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9aec0a2c10 0x7f9aec0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b00105120 tx=0x7f9af0005e50 comp rx=0 tx=0).stop 2024-09-12T12:11:32.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00100a80 msgr2=0x7f9b00198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00100a80 0x7f9b00198470 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00b4a0 tx=0x7f9afc00b960 comp rx=0 tx=0).stop 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 shutdown_connections 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9aec0a2c10 0x7f9aec0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b00106a80 0x7f9b0019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b00101420 0x7f9b001989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 --2- 172.21.15.137:0/2705654347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9b00100a80 0x7f9b00198470 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 >> 172.21.15.137:0/2705654347 conn(0x7f9b000fc7a0 msgr2=0x7f9b0006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 shutdown_connections 2024-09-12T12:11:32.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:32.929+0000 7f9b086e6640 1 -- 172.21.15.137:0/2705654347 wait complete. 2024-09-12T12:11:33.566 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:11:33.810 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:34.201 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:34 smithi137 ceph-mon[30950]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:34.201 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:34 smithi137 ceph-mon[30950]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-12T12:11:34.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 -- 172.21.15.137:0/2024237228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 msgr2=0x7fe2b010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:34.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 --2- 172.21.15.137:0/2024237228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b010d950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009cd0 tx=0x7fe2a802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 -- 172.21.15.137:0/2024237228 shutdown_connections 2024-09-12T12:11:34.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 --2- 172.21.15.137:0/2024237228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b010d950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 --2- 172.21.15.137:0/2024237228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b0100a80 0x7fe2b0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 --2- 172.21.15.137:0/2024237228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2b0106a80 0x7fe2b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 -- 172.21.15.137:0/2024237228 >> 172.21.15.137:0/2024237228 conn(0x7fe2b00fc7a0 msgr2=0x7fe2b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 -- 172.21.15.137:0/2024237228 shutdown_connections 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.279+0000 7fe2b7264640 1 -- 172.21.15.137:0/2024237228 wait complete. 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 Processor -- start 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 -- start start 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2b0100a80 0x7fe2b01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b0106a80 0x7fe2b01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0110e80 con 0x7fe2b0101420 2024-09-12T12:11:34.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0110d00 con 0x7fe2b0100a80 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b7264640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0111000 con 0x7fe2b0106a80 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2affff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b01997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37000/0 (socket says 172.21.15.137:37000) 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2affff640 1 -- 172.21.15.137:0/3281443290 learned_addr learned my addr 172.21.15.137:0/3281443290 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b5000640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2b0100a80 0x7fe2b01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.280+0000 7fe2b5801640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b0106a80 0x7fe2b01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2affff640 1 -- 172.21.15.137:0/3281443290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b0106a80 msgr2=0x7fe2b01a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2affff640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b0106a80 0x7fe2b01a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2affff640 1 -- 172.21.15.137:0/3281443290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2b0100a80 msgr2=0x7fe2b01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2affff640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2b0100a80 0x7fe2b01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2affff640 1 -- 172.21.15.137:0/3281443290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b01a4870 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2affff640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b01997f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400d950 tx=0x7fe2a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2adffb640 1 -- 172.21.15.137:0/3281443290 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4017070 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2adffb640 1 -- 172.21.15.137:0/3281443290 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a4009c30 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b01a4a40 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2adffb640 1 -- 172.21.15.137:0/3281443290 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400f6b0 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.281+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b01a4f50 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.282+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b0100ee0 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.283+0000 7fe2adffb640 1 -- 172.21.15.137:0/3281443290 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe2a40109d0 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.284+0000 7fe2adffb640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2a00a2c10 0x7fe2a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.284+0000 7fe2b5000640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2a00a2c10 0x7fe2a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.284+0000 7fe2adffb640 1 -- 172.21.15.137:0/3281443290 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fe2a40cec00 con 0x7fe2b0101420 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.284+0000 7fe2b5000640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2a00a2c10 0x7fe2a00a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00bb60 tx=0x7fe29c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:34.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.288+0000 7fe2adffb640 1 -- 172.21.15.137:0/3281443290 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe2a40d4020 con 0x7fe2b0101420 2024-09-12T12:11:34.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:34 smithi171 ceph-mon[35098]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:34.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:34 smithi171 ceph-mon[35098]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-12T12:11:34.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.488+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2b0108ff0 con 0x7fe2b0101420 2024-09-12T12:11:34.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.489+0000 7fe2adffb640 1 -- 172.21.15.137:0/3281443290 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7fe2a4009de0 con 0x7fe2b0101420 2024-09-12T12:11:34.490 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:34.491 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"btime":"2024-09-12T12:11:24:550801+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:11:24.095061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:11:34.491 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-09-12T12:11:34.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2a00a2c10 msgr2=0x7fe2a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:34.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2a00a2c10 0x7fe2a00a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00bb60 tx=0x7fe29c00b390 comp rx=0 tx=0).stop 2024-09-12T12:11:34.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 msgr2=0x7fe2b01997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:34.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b01997f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400d950 tx=0x7fe2a400de10 comp rx=0 tx=0).stop 2024-09-12T12:11:34.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 shutdown_connections 2024-09-12T12:11:34.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe2a00a2c10 0x7fe2a00a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b0106a80 0x7fe2b01a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2b0101420 0x7fe2b01997f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 --2- 172.21.15.137:0/3281443290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2b0100a80 0x7fe2b01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:34.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 >> 172.21.15.137:0/3281443290 conn(0x7fe2b00fc7a0 msgr2=0x7fe2b00fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:34.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 shutdown_connections 2024-09-12T12:11:34.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:34.491+0000 7fe2b7264640 1 -- 172.21.15.137:0/3281443290 wait complete. 2024-09-12T12:11:34.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:34 smithi189 ceph-mon[35475]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:11:34.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:34 smithi189 ceph-mon[35475]: pgmap v291: 97 pgs: 97 active+clean; 602 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-09-12T12:11:35.169 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:11:35.420 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:35.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:35 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3281443290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:35.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:35 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3281443290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:35.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:35 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3281443290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:11:35.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 -- 172.21.15.137:0/1195404824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690101420 msgr2=0x7f369010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 --2- 172.21.15.137:0/1195404824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690101420 0x7f369010d950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3688009cd0 tx=0x7f368802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 -- 172.21.15.137:0/1195404824 shutdown_connections 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 --2- 172.21.15.137:0/1195404824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690101420 0x7f369010d950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 --2- 172.21.15.137:0/1195404824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3690100a80 0x7f3690100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 --2- 172.21.15.137:0/1195404824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3690106a80 0x7f3690106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 -- 172.21.15.137:0/1195404824 >> 172.21.15.137:0/1195404824 conn(0x7f36900fc7a0 msgr2=0x7f36900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 -- 172.21.15.137:0/1195404824 shutdown_connections 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.910+0000 7f3697b7b640 1 -- 172.21.15.137:0/1195404824 wait complete. 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 Processor -- start 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 -- start start 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690100a80 0x7f3690198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:35.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3690101420 0x7f3690198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3690106a80 0x7f369019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3690110e80 con 0x7f3690100a80 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3690110d00 con 0x7f3690101420 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3697b7b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3690111000 con 0x7f3690106a80 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690100a80 0x7f3690198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690100a80 0x7f3690198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37018/0 (socket says 172.21.15.137:37018) 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 -- 172.21.15.137:0/3513205446 learned_addr learned my addr 172.21.15.137:0/3513205446 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3696118640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3690106a80 0x7f369019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 -- 172.21.15.137:0/3513205446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3690106a80 msgr2=0x7f369019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3690106a80 0x7f369019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:35.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 -- 172.21.15.137:0/3513205446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3690101420 msgr2=0x7f3690198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3690101420 0x7f3690198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 -- 172.21.15.137:0/3513205446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369019d3e0 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.911+0000 7f3695917640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690100a80 0x7f3690198430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f368c00d950 tx=0x7f368c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.912+0000 7f3686ffd640 1 -- 172.21.15.137:0/3513205446 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c017030 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.912+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369019d6d0 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.912+0000 7f3686ffd640 1 -- 172.21.15.137:0/3513205446 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f368c004020 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.912+0000 7f3686ffd640 1 -- 172.21.15.137:0/3513205446 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c0047a0 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.912+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36901a4fb0 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.913+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3690100ee0 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.914+0000 7f3686ffd640 1 -- 172.21.15.137:0/3513205446 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f368c0041d0 con 0x7f3690100a80 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.915+0000 7f3686ffd640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f367c0a2c10 0x7f367c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.915+0000 7f3695116640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f367c0a2c10 0x7f367c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:35.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.915+0000 7f3686ffd640 1 -- 172.21.15.137:0/3513205446 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f368c0cf1d0 con 0x7f3690100a80 2024-09-12T12:11:35.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.915+0000 7f3695116640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f367c0a2c10 0x7f367c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f3680005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:35.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:35.918+0000 7f3686ffd640 1 -- 172.21.15.137:0/3513205446 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f368c09c080 con 0x7f3690100a80 2024-09-12T12:11:36.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.087+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3690199790 con 0x7f3690100a80 2024-09-12T12:11:36.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.088+0000 7f3686ffd640 1 -- 172.21.15.137:0/3513205446 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22348 (secure 0 0 0) 0x7f368c09b980 con 0x7f3690100a80 2024-09-12T12:11:36.089 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:36.090 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:11:13.196473+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:11:36.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f367c0a2c10 msgr2=0x7f367c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:36.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f367c0a2c10 0x7f367c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f3680005850 comp rx=0 tx=0).stop 2024-09-12T12:11:36.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690100a80 msgr2=0x7f3690198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:36.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690100a80 0x7f3690198430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f368c00d950 tx=0x7f368c00de10 comp rx=0 tx=0).stop 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 shutdown_connections 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f367c0a2c10 0x7f367c0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3690106a80 0x7f369019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3690101420 0x7f3690198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 --2- 172.21.15.137:0/3513205446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3690100a80 0x7f3690198430 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 >> 172.21.15.137:0/3513205446 conn(0x7f36900fc7a0 msgr2=0x7f369006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 shutdown_connections 2024-09-12T12:11:36.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:36.091+0000 7f3697b7b640 1 -- 172.21.15.137:0/3513205446 wait complete. 2024-09-12T12:11:36.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:36 smithi171 ceph-mon[35098]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-09-12T12:11:36.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:36 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3513205446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:36.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:36 smithi137 ceph-mon[30950]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-09-12T12:11:36.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:36 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3513205446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:36.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:36 smithi189 ceph-mon[35475]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-09-12T12:11:36.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:36 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3513205446' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:37.088 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T12:11:37.088 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:11:37.338 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:37.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 -- 172.21.15.137:0/2732892518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114102790 msgr2=0x7ff1141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 --2- 172.21.15.137:0/2732892518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114102790 0x7ff1141067e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff104009900 tx=0x7ff10402a410 comp rx=0 tx=0).stop 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 -- 172.21.15.137:0/2732892518 shutdown_connections 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 --2- 172.21.15.137:0/2732892518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff114106f10 0x7ff11410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 --2- 172.21.15.137:0/2732892518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114102790 0x7ff1141067e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 --2- 172.21.15.137:0/2732892518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff114101de0 0x7ff1141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 -- 172.21.15.137:0/2732892518 >> 172.21.15.137:0/2732892518 conn(0x7ff1140fd8c0 msgr2=0x7ff1140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 -- 172.21.15.137:0/2732892518 shutdown_connections 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.864+0000 7ff118850640 1 -- 172.21.15.137:0/2732892518 wait complete. 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 Processor -- start 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 -- start start 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114101de0 0x7ff1141983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:37.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff114102790 0x7ff1141988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff114106f10 0x7ff11419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff114111f70 con 0x7ff114101de0 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff114111df0 con 0x7ff114102790 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff118850640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141120f0 con 0x7ff114106f10 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff11259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114101de0 0x7ff1141983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff11259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114101de0 0x7ff1141983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37040/0 (socket says 172.21.15.137:37040) 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.865+0000 7ff11259c640 1 -- 172.21.15.137:0/1214110708 learned_addr learned my addr 172.21.15.137:0/1214110708 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff111d9b640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff114102790 0x7ff1141988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff112d9d640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff114106f10 0x7ff11419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff11259c640 1 -- 172.21.15.137:0/1214110708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff114106f10 msgr2=0x7ff11419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff11259c640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff114106f10 0x7ff11419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff11259c640 1 -- 172.21.15.137:0/1214110708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff114102790 msgr2=0x7ff1141988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:37.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff11259c640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff114102790 0x7ff1141988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff11259c640 1 -- 172.21.15.137:0/1214110708 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11419d360 con 0x7ff114101de0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff112d9d640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff114106f10 0x7ff11419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff11259c640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114101de0 0x7ff1141983b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d950 tx=0x7ff10800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff1037fe640 1 -- 172.21.15.137:0/1214110708 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108017070 con 0x7ff114101de0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11419d650 con 0x7ff114101de0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff1037fe640 1 -- 172.21.15.137:0/1214110708 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff108009c30 con 0x7ff114101de0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff1037fe640 1 -- 172.21.15.137:0/1214110708 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10800f6b0 con 0x7ff114101de0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.866+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141a4f30 con 0x7ff114101de0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.867+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1141021c0 con 0x7ff114101de0 2024-09-12T12:11:37.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.868+0000 7ff1037fe640 1 -- 172.21.15.137:0/1214110708 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff1080109d0 con 0x7ff114101de0 2024-09-12T12:11:37.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.869+0000 7ff1037fe640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0fc0a2c10 0x7ff0fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:37.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.869+0000 7ff111d9b640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0fc0a2c10 0x7ff0fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:37.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.869+0000 7ff1037fe640 1 -- 172.21.15.137:0/1214110708 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7ff1080ce9b0 con 0x7ff114101de0 2024-09-12T12:11:37.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.869+0000 7ff111d9b640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0fc0a2c10 0x7ff0fc0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff1141088e0 tx=0x7ff104005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:37.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:37.873+0000 7ff1037fe640 1 -- 172.21.15.137:0/1214110708 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff10809b960 con 0x7ff114101de0 2024-09-12T12:11:38.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.036+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff11410a1c0 con 0x7ff0fc0a2c10 2024-09-12T12:11:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.043+0000 7ff1037fe640 1 -- 172.21.15.137:0/1214110708 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7ff1040315b0 con 0x7ff0fc0a2c10 2024-09-12T12:11:38.049 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:38.050 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:11:38.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0fc0a2c10 msgr2=0x7ff0fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:38.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0fc0a2c10 0x7ff0fc0a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff1141088e0 tx=0x7ff104005a20 comp rx=0 tx=0).stop 2024-09-12T12:11:38.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114101de0 msgr2=0x7ff1141983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:38.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114101de0 0x7ff1141983b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d950 tx=0x7ff10800de10 comp rx=0 tx=0).stop 2024-09-12T12:11:38.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 shutdown_connections 2024-09-12T12:11:38.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0fc0a2c10 0x7ff0fc0a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:38.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff114106f10 0x7ff11419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:38.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.046+0000 7ff118850640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff114102790 0x7ff1141988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:38.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.047+0000 7ff118850640 1 --2- 172.21.15.137:0/1214110708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff114101de0 0x7ff1141983b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:38.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.047+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 >> 172.21.15.137:0/1214110708 conn(0x7ff1140fd8c0 msgr2=0x7ff1140fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:38.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.047+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 shutdown_connections 2024-09-12T12:11:38.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:38.047+0000 7ff118850640 1 -- 172.21.15.137:0/1214110708 wait complete. 2024-09-12T12:11:38.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:38 smithi137 ceph-mon[30950]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-12T12:11:38.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:38 smithi171 ceph-mon[35098]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-12T12:11:38.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:38 smithi189 ceph-mon[35475]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-09-12T12:11:38.851 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-12T12:11:37.115378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641407,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":234,"num_read_kb":221,"num_write":333,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355948,"kb_used_data":10860,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585876,"statfs":{"total":1151940427776,"available":1151575937024,"internally_reserved":0,"allocated":11120640,"data_stored":7043469,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46529,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":85,"num_read_kb":90,"num_write":43,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002817"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491849+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:11:13.491849+0000","last_peered":"2024-09-12T12:11:13.491849+0000","last_clean":"2024-09-12T12:11:13.491849+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:11:13.491849+0000","last_undegraded":"2024-09-12T12:11:13.491849+0000","last_fullsized":"2024-09-12T12:11:13.491849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500296+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:11:13.500296+0000","last_peered":"2024-09-12T12:11:13.500296+0000","last_clean":"2024-09-12T12:11:13.500296+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:11:13.500296+0000","last_undegraded":"2024-09-12T12:11:13.500296+0000","last_fullsized":"2024-09-12T12:11:13.500296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500541+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:11:13.500541+0000","last_peered":"2024-09-12T12:11:13.500541+0000","last_clean":"2024-09-12T12:11:13.500541+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:11:13.500541+0000","last_undegraded":"2024-09-12T12:11:13.500541+0000","last_fullsized":"2024-09-12T12:11:13.500541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492536+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:11:13.492536+0000","last_peered":"2024-09-12T12:11:13.492536+0000","last_clean":"2024-09-12T12:11:13.492536+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:11:13.492536+0000","last_undegraded":"2024-09-12T12:11:13.492536+0000","last_fullsized":"2024-09-12T12:11:13.492536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501335+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:11:13.501335+0000","last_peered":"2024-09-12T12:11:13.501335+0000","last_clean":"2024-09-12T12:11:13.501335+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:11:13.501335+0000","last_undegraded":"2024-09-12T12:11:13.501335+0000","last_fullsized":"2024-09-12T12:11:13.501335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500893+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:11:13.500893+0000","last_peered":"2024-09-12T12:11:13.500893+0000","last_clean":"2024-09-12T12:11:13.500893+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:11:13.500893+0000","last_undegraded":"2024-09-12T12:11:13.500893+0000","last_fullsized":"2024-09-12T12:11:13.500893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503506+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:11:13.503506+0000","last_peered":"2024-09-12T12:11:13.503506+0000","last_clean":"2024-09-12T12:11:13.503506+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:11:13.503506+0000","last_undegraded":"2024-09-12T12:11:13.503506+0000","last_fullsized":"2024-09-12T12:11:13.503506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498001+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:11:13.498001+0000","last_peered":"2024-09-12T12:11:13.498001+0000","last_clean":"2024-09-12T12:11:13.498001+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:11:13.498001+0000","last_undegraded":"2024-09-12T12:11:13.498001+0000","last_fullsized":"2024-09-12T12:11:13.498001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503514+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:11:13.503514+0000","last_peered":"2024-09-12T12:11:13.503514+0000","last_clean":"2024-09-12T12:11:13.503514+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:11:13.503514+0000","last_undegraded":"2024-09-12T12:11:13.503514+0000","last_fullsized":"2024-09-12T12:11:13.503514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501033+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:11:13.501033+0000","last_peered":"2024-09-12T12:11:13.501033+0000","last_clean":"2024-09-12T12:11:13.501033+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:11:13.501033+0000","last_undegraded":"2024-09-12T12:11:13.501033+0000","last_fullsized":"2024-09-12T12:11:13.501033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504240+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:11:13.504240+0000","last_peered":"2024-09-12T12:11:13.504240+0000","last_clean":"2024-09-12T12:11:13.504240+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:11:13.504240+0000","last_undegraded":"2024-09-12T12:11:13.504240+0000","last_fullsized":"2024-09-12T12:11:13.504240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492580+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:11:13.492580+0000","last_peered":"2024-09-12T12:11:13.492580+0000","last_clean":"2024-09-12T12:11:13.492580+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:11:13.492580+0000","last_undegraded":"2024-09-12T12:11:13.492580+0000","last_fullsized":"2024-09-12T12:11:13.492580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491145+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:11:13.491145+0000","last_peered":"2024-09-12T12:11:13.491145+0000","last_clean":"2024-09-12T12:11:13.491145+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:11:13.491145+0000","last_undegraded":"2024-09-12T12:11:13.491145+0000","last_fullsized":"2024-09-12T12:11:13.491145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500084+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:11:13.500084+0000","last_peered":"2024-09-12T12:11:13.500084+0000","last_clean":"2024-09-12T12:11:13.500084+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:11:13.500084+0000","last_undegraded":"2024-09-12T12:11:13.500084+0000","last_fullsized":"2024-09-12T12:11:13.500084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.499972+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:11:13.499972+0000","last_peered":"2024-09-12T12:11:13.499972+0000","last_clean":"2024-09-12T12:11:13.499972+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:11:13.499972+0000","last_undegraded":"2024-09-12T12:11:13.499972+0000","last_fullsized":"2024-09-12T12:11:13.499972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502792+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:11:13.502792+0000","last_peered":"2024-09-12T12:11:13.502792+0000","last_clean":"2024-09-12T12:11:13.502792+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:11:13.502792+0000","last_undegraded":"2024-09-12T12:11:13.502792+0000","last_fullsized":"2024-09-12T12:11:13.502792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491907+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:11:13.491907+0000","last_peered":"2024-09-12T12:11:13.491907+0000","last_clean":"2024-09-12T12:11:13.491907+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:11:13.491907+0000","last_undegraded":"2024-09-12T12:11:13.491907+0000","last_fullsized":"2024-09-12T12:11:13.491907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502053+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:11:13.502053+0000","last_peered":"2024-09-12T12:11:13.502053+0000","last_clean":"2024-09-12T12:11:13.502053+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:11:13.502053+0000","last_undegraded":"2024-09-12T12:11:13.502053+0000","last_fullsized":"2024-09-12T12:11:13.502053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498689+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:11:13.498689+0000","last_peered":"2024-09-12T12:11:13.498689+0000","last_clean":"2024-09-12T12:11:13.498689+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:11:13.498689+0000","last_undegraded":"2024-09-12T12:11:13.498689+0000","last_fullsized":"2024-09-12T12:11:13.498689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491690+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:11:13.491690+0000","last_peered":"2024-09-12T12:11:13.491690+0000","last_clean":"2024-09-12T12:11:13.491690+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:11:13.491690+0000","last_undegraded":"2024-09-12T12:11:13.491690+0000","last_fullsized":"2024-09-12T12:11:13.491690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491185+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:11:13.491185+0000","last_peered":"2024-09-12T12:11:13.491185+0000","last_clean":"2024-09-12T12:11:13.491185+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:11:13.491185+0000","last_undegraded":"2024-09-12T12:11:13.491185+0000","last_fullsized":"2024-09-12T12:11:13.491185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503390+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:11:13.503390+0000","last_peered":"2024-09-12T12:11:13.503390+0000","last_clean":"2024-09-12T12:11:13.503390+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:11:13.503390+0000","last_undegraded":"2024-09-12T12:11:13.503390+0000","last_fullsized":"2024-09-12T12:11:13.503390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497730+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:11:13.497730+0000","last_peered":"2024-09-12T12:11:13.497730+0000","last_clean":"2024-09-12T12:11:13.497730+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:11:13.497730+0000","last_undegraded":"2024-09-12T12:11:13.497730+0000","last_fullsized":"2024-09-12T12:11:13.497730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500719+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:11:13.500719+0000","last_peered":"2024-09-12T12:11:13.500719+0000","last_clean":"2024-09-12T12:11:13.500719+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:11:13.500719+0000","last_undegraded":"2024-09-12T12:11:13.500719+0000","last_fullsized":"2024-09-12T12:11:13.500719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498394+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:11:13.498394+0000","last_peered":"2024-09-12T12:11:13.498394+0000","last_clean":"2024-09-12T12:11:13.498394+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:11:13.498394+0000","last_undegraded":"2024-09-12T12:11:13.498394+0000","last_fullsized":"2024-09-12T12:11:13.498394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498501+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:11:13.498501+0000","last_peered":"2024-09-12T12:11:13.498501+0000","last_clean":"2024-09-12T12:11:13.498501+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:11:13.498501+0000","last_undegraded":"2024-09-12T12:11:13.498501+0000","last_fullsized":"2024-09-12T12:11:13.498501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491855+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:11:13.491855+0000","last_peered":"2024-09-12T12:11:13.491855+0000","last_clean":"2024-09-12T12:11:13.491855+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:11:13.491855+0000","last_undegraded":"2024-09-12T12:11:13.491855+0000","last_fullsized":"2024-09-12T12:11:13.491855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491115+0000","last_change":"2024-09-12T12:11:12.500411+0000","last_active":"2024-09-12T12:11:13.491115+0000","last_peered":"2024-09-12T12:11:13.491115+0000","last_clean":"2024-09-12T12:11:13.491115+0000","last_became_active":"2024-09-12T12:11:12.500126+0000","last_became_peered":"2024-09-12T12:11:12.500126+0000","last_unstale":"2024-09-12T12:11:13.491115+0000","last_undegraded":"2024-09-12T12:11:13.491115+0000","last_fullsized":"2024-09-12T12:11:13.491115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:23:49.072306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500514+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:11:13.500514+0000","last_peered":"2024-09-12T12:11:13.500514+0000","last_clean":"2024-09-12T12:11:13.500514+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:11:13.500514+0000","last_undegraded":"2024-09-12T12:11:13.500514+0000","last_fullsized":"2024-09-12T12:11:13.500514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498074+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:11:13.498074+0000","last_peered":"2024-09-12T12:11:13.498074+0000","last_clean":"2024-09-12T12:11:13.498074+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:11:13.498074+0000","last_undegraded":"2024-09-12T12:11:13.498074+0000","last_fullsized":"2024-09-12T12:11:13.498074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498204+0000","last_change":"2024-09-12T12:11:12.510232+0000","last_active":"2024-09-12T12:11:13.498204+0000","last_peered":"2024-09-12T12:11:13.498204+0000","last_clean":"2024-09-12T12:11:13.498204+0000","last_became_active":"2024-09-12T12:11:12.509856+0000","last_became_peered":"2024-09-12T12:11:12.509856+0000","last_unstale":"2024-09-12T12:11:13.498204+0000","last_undegraded":"2024-09-12T12:11:13.498204+0000","last_fullsized":"2024-09-12T12:11:13.498204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:56.293960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491659+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:11:13.491659+0000","last_peered":"2024-09-12T12:11:13.491659+0000","last_clean":"2024-09-12T12:11:13.491659+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:11:13.491659+0000","last_undegraded":"2024-09-12T12:11:13.491659+0000","last_fullsized":"2024-09-12T12:11:13.491659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501135+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:11:13.501135+0000","last_peered":"2024-09-12T12:11:13.501135+0000","last_clean":"2024-09-12T12:11:13.501135+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:11:13.501135+0000","last_undegraded":"2024-09-12T12:11:13.501135+0000","last_fullsized":"2024-09-12T12:11:13.501135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:34.485355+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:11:34.485355+0000","last_peered":"2024-09-12T12:11:34.485355+0000","last_clean":"2024-09-12T12:11:34.485355+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:11:34.485355+0000","last_undegraded":"2024-09-12T12:11:34.485355+0000","last_fullsized":"2024-09-12T12:11:34.485355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500766+0000","last_change":"2024-09-12T12:11:12.508286+0000","last_active":"2024-09-12T12:11:13.500766+0000","last_peered":"2024-09-12T12:11:13.500766+0000","last_clean":"2024-09-12T12:11:13.500766+0000","last_became_active":"2024-09-12T12:11:12.507860+0000","last_became_peered":"2024-09-12T12:11:12.507860+0000","last_unstale":"2024-09-12T12:11:13.500766+0000","last_undegraded":"2024-09-12T12:11:13.500766+0000","last_fullsized":"2024-09-12T12:11:13.500766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:15:49.330419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556617+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:11:21.556617+0000","last_peered":"2024-09-12T12:11:21.556617+0000","last_clean":"2024-09-12T12:11:21.556617+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:11:21.556617+0000","last_undegraded":"2024-09-12T12:11:21.556617+0000","last_fullsized":"2024-09-12T12:11:21.556617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556515+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:11:21.556515+0000","last_peered":"2024-09-12T12:11:21.556515+0000","last_clean":"2024-09-12T12:11:21.556515+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:11:21.556515+0000","last_undegraded":"2024-09-12T12:11:21.556515+0000","last_fullsized":"2024-09-12T12:11:21.556515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497911+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:11:13.497911+0000","last_peered":"2024-09-12T12:11:13.497911+0000","last_clean":"2024-09-12T12:11:13.497911+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:11:13.497911+0000","last_undegraded":"2024-09-12T12:11:13.497911+0000","last_fullsized":"2024-09-12T12:11:13.497911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501882+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:11:13.501882+0000","last_peered":"2024-09-12T12:11:13.501882+0000","last_clean":"2024-09-12T12:11:13.501882+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:11:13.501882+0000","last_undegraded":"2024-09-12T12:11:13.501882+0000","last_fullsized":"2024-09-12T12:11:13.501882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501931+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:11:13.501931+0000","last_peered":"2024-09-12T12:11:13.501931+0000","last_clean":"2024-09-12T12:11:13.501931+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:11:13.501931+0000","last_undegraded":"2024-09-12T12:11:13.501931+0000","last_fullsized":"2024-09-12T12:11:13.501931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":34,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:31.480565+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:11:31.480565+0000","last_peered":"2024-09-12T12:11:31.480565+0000","last_clean":"2024-09-12T12:11:31.480565+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:11:31.480565+0000","last_undegraded":"2024-09-12T12:11:31.480565+0000","last_fullsized":"2024-09-12T12:11:31.480565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503842+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:11:13.503842+0000","last_peered":"2024-09-12T12:11:13.503842+0000","last_clean":"2024-09-12T12:11:13.503842+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:11:13.503842+0000","last_undegraded":"2024-09-12T12:11:13.503842+0000","last_fullsized":"2024-09-12T12:11:13.503842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.543897+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:11:20.543897+0000","last_peered":"2024-09-12T12:11:20.543897+0000","last_clean":"2024-09-12T12:11:20.543897+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:11:20.543897+0000","last_undegraded":"2024-09-12T12:11:20.543897+0000","last_fullsized":"2024-09-12T12:11:20.543897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500509+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:11:13.500509+0000","last_peered":"2024-09-12T12:11:13.500509+0000","last_clean":"2024-09-12T12:11:13.500509+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:11:13.500509+0000","last_undegraded":"2024-09-12T12:11:13.500509+0000","last_fullsized":"2024-09-12T12:11:13.500509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545077+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:11:20.545077+0000","last_peered":"2024-09-12T12:11:20.545077+0000","last_clean":"2024-09-12T12:11:20.545077+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:11:20.545077+0000","last_undegraded":"2024-09-12T12:11:20.545077+0000","last_fullsized":"2024-09-12T12:11:20.545077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503678+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:11:13.503678+0000","last_peered":"2024-09-12T12:11:13.503678+0000","last_clean":"2024-09-12T12:11:13.503678+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:11:13.503678+0000","last_undegraded":"2024-09-12T12:11:13.503678+0000","last_fullsized":"2024-09-12T12:11:13.503678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'22","reported_seq":70,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:32.484711+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:11:32.484711+0000","last_peered":"2024-09-12T12:11:32.484711+0000","last_clean":"2024-09-12T12:11:32.484711+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:11:32.484711+0000","last_undegraded":"2024-09-12T12:11:32.484711+0000","last_fullsized":"2024-09-12T12:11:32.484711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":59,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503335+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:11:13.503335+0000","last_peered":"2024-09-12T12:11:13.503335+0000","last_clean":"2024-09-12T12:11:13.503335+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:11:13.503335+0000","last_undegraded":"2024-09-12T12:11:13.503335+0000","last_fullsized":"2024-09-12T12:11:13.503335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557533+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:11:21.557533+0000","last_peered":"2024-09-12T12:11:21.557533+0000","last_clean":"2024-09-12T12:11:21.557533+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:11:21.557533+0000","last_undegraded":"2024-09-12T12:11:21.557533+0000","last_fullsized":"2024-09-12T12:11:21.557533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490815+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:11:13.490815+0000","last_peered":"2024-09-12T12:11:13.490815+0000","last_clean":"2024-09-12T12:11:13.490815+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:11:13.490815+0000","last_undegraded":"2024-09-12T12:11:13.490815+0000","last_fullsized":"2024-09-12T12:11:13.490815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.546044+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:11:20.546044+0000","last_peered":"2024-09-12T12:11:20.546044+0000","last_clean":"2024-09-12T12:11:20.546044+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:11:20.546044+0000","last_undegraded":"2024-09-12T12:11:20.546044+0000","last_fullsized":"2024-09-12T12:11:20.546044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503539+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:11:13.503539+0000","last_peered":"2024-09-12T12:11:13.503539+0000","last_clean":"2024-09-12T12:11:13.503539+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:11:13.503539+0000","last_undegraded":"2024-09-12T12:11:13.503539+0000","last_fullsized":"2024-09-12T12:11:13.503539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:33.484365+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:11:33.484365+0000","last_peered":"2024-09-12T12:11:33.484365+0000","last_clean":"2024-09-12T12:11:33.484365+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:11:33.484365+0000","last_undegraded":"2024-09-12T12:11:33.484365+0000","last_fullsized":"2024-09-12T12:11:33.484365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503667+0000","last_change":"2024-09-12T12:11:12.504047+0000","last_active":"2024-09-12T12:11:13.503667+0000","last_peered":"2024-09-12T12:11:13.503667+0000","last_clean":"2024-09-12T12:11:13.503667+0000","last_became_active":"2024-09-12T12:11:12.503648+0000","last_became_peered":"2024-09-12T12:11:12.503648+0000","last_unstale":"2024-09-12T12:11:13.503667+0000","last_undegraded":"2024-09-12T12:11:13.503667+0000","last_fullsized":"2024-09-12T12:11:13.503667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:43.443802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.558326+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:11:21.558326+0000","last_peered":"2024-09-12T12:11:21.558326+0000","last_clean":"2024-09-12T12:11:21.558326+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:11:21.558326+0000","last_undegraded":"2024-09-12T12:11:21.558326+0000","last_fullsized":"2024-09-12T12:11:21.558326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501026+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:11:13.501026+0000","last_peered":"2024-09-12T12:11:13.501026+0000","last_clean":"2024-09-12T12:11:13.501026+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:11:13.501026+0000","last_undegraded":"2024-09-12T12:11:13.501026+0000","last_fullsized":"2024-09-12T12:11:13.501026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:22.466026+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:11:22.466026+0000","last_peered":"2024-09-12T12:11:22.466026+0000","last_clean":"2024-09-12T12:11:22.466026+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:11:22.466026+0000","last_undegraded":"2024-09-12T12:11:22.466026+0000","last_fullsized":"2024-09-12T12:11:22.466026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491726+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:11:13.491726+0000","last_peered":"2024-09-12T12:11:13.491726+0000","last_clean":"2024-09-12T12:11:13.491726+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:11:13.491726+0000","last_undegraded":"2024-09-12T12:11:13.491726+0000","last_fullsized":"2024-09-12T12:11:13.491726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497437+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:11:13.497437+0000","last_peered":"2024-09-12T12:11:13.497437+0000","last_clean":"2024-09-12T12:11:13.497437+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:11:13.497437+0000","last_undegraded":"2024-09-12T12:11:13.497437+0000","last_fullsized":"2024-09-12T12:11:13.497437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504243+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:11:13.504243+0000","last_peered":"2024-09-12T12:11:13.504243+0000","last_clean":"2024-09-12T12:11:13.504243+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:11:13.504243+0000","last_undegraded":"2024-09-12T12:11:13.504243+0000","last_fullsized":"2024-09-12T12:11:13.504243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.507452+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:11:13.507452+0000","last_peered":"2024-09-12T12:11:13.507452+0000","last_clean":"2024-09-12T12:11:13.507452+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:11:13.507452+0000","last_undegraded":"2024-09-12T12:11:13.507452+0000","last_fullsized":"2024-09-12T12:11:13.507452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503836+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:11:13.503836+0000","last_peered":"2024-09-12T12:11:13.503836+0000","last_clean":"2024-09-12T12:11:13.503836+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:11:13.503836+0000","last_undegraded":"2024-09-12T12:11:13.503836+0000","last_fullsized":"2024-09-12T12:11:13.503836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557550+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:11:21.557550+0000","last_peered":"2024-09-12T12:11:21.557550+0000","last_clean":"2024-09-12T12:11:21.557550+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:11:21.557550+0000","last_undegraded":"2024-09-12T12:11:21.557550+0000","last_fullsized":"2024-09-12T12:11:21.557550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.505016+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:11:13.505016+0000","last_peered":"2024-09-12T12:11:13.505016+0000","last_clean":"2024-09-12T12:11:13.505016+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:11:13.505016+0000","last_undegraded":"2024-09-12T12:11:13.505016+0000","last_fullsized":"2024-09-12T12:11:13.505016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503794+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:11:13.503794+0000","last_peered":"2024-09-12T12:11:13.503794+0000","last_clean":"2024-09-12T12:11:13.503794+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:11:13.503794+0000","last_undegraded":"2024-09-12T12:11:13.503794+0000","last_fullsized":"2024-09-12T12:11:13.503794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:22.465059+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:11:22.465059+0000","last_peered":"2024-09-12T12:11:22.465059+0000","last_clean":"2024-09-12T12:11:22.465059+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:11:22.465059+0000","last_undegraded":"2024-09-12T12:11:22.465059+0000","last_fullsized":"2024-09-12T12:11:22.465059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498114+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:11:13.498114+0000","last_peered":"2024-09-12T12:11:13.498114+0000","last_clean":"2024-09-12T12:11:13.498114+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:11:13.498114+0000","last_undegraded":"2024-09-12T12:11:13.498114+0000","last_fullsized":"2024-09-12T12:11:13.498114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556747+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:11:21.556747+0000","last_peered":"2024-09-12T12:11:21.556747+0000","last_clean":"2024-09-12T12:11:21.556747+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:11:21.556747+0000","last_undegraded":"2024-09-12T12:11:21.556747+0000","last_fullsized":"2024-09-12T12:11:21.556747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490784+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:11:13.490784+0000","last_peered":"2024-09-12T12:11:13.490784+0000","last_clean":"2024-09-12T12:11:13.490784+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:11:13.490784+0000","last_undegraded":"2024-09-12T12:11:13.490784+0000","last_fullsized":"2024-09-12T12:11:13.490784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:33.483943+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:11:33.483943+0000","last_peered":"2024-09-12T12:11:33.483943+0000","last_clean":"2024-09-12T12:11:33.483943+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:11:33.483943+0000","last_undegraded":"2024-09-12T12:11:33.483943+0000","last_fullsized":"2024-09-12T12:11:33.483943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500783+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:11:13.500783+0000","last_peered":"2024-09-12T12:11:13.500783+0000","last_clean":"2024-09-12T12:11:13.500783+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:11:13.500783+0000","last_undegraded":"2024-09-12T12:11:13.500783+0000","last_fullsized":"2024-09-12T12:11:13.500783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504416+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:11:13.504416+0000","last_peered":"2024-09-12T12:11:13.504416+0000","last_clean":"2024-09-12T12:11:13.504416+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:11:13.504416+0000","last_undegraded":"2024-09-12T12:11:13.504416+0000","last_fullsized":"2024-09-12T12:11:13.504416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500974+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:11:13.500974+0000","last_peered":"2024-09-12T12:11:13.500974+0000","last_clean":"2024-09-12T12:11:13.500974+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:11:13.500974+0000","last_undegraded":"2024-09-12T12:11:13.500974+0000","last_fullsized":"2024-09-12T12:11:13.500974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501897+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:11:13.501897+0000","last_peered":"2024-09-12T12:11:13.501897+0000","last_clean":"2024-09-12T12:11:13.501897+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:11:13.501897+0000","last_undegraded":"2024-09-12T12:11:13.501897+0000","last_fullsized":"2024-09-12T12:11:13.501897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500355+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:11:13.500355+0000","last_peered":"2024-09-12T12:11:13.500355+0000","last_clean":"2024-09-12T12:11:13.500355+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:11:13.500355+0000","last_undegraded":"2024-09-12T12:11:13.500355+0000","last_fullsized":"2024-09-12T12:11:13.500355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.544273+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:11:20.544273+0000","last_peered":"2024-09-12T12:11:20.544273+0000","last_clean":"2024-09-12T12:11:20.544273+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:11:20.544273+0000","last_undegraded":"2024-09-12T12:11:20.544273+0000","last_fullsized":"2024-09-12T12:11:20.544273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491176+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:11:13.491176+0000","last_peered":"2024-09-12T12:11:13.491176+0000","last_clean":"2024-09-12T12:11:13.491176+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:11:13.491176+0000","last_undegraded":"2024-09-12T12:11:13.491176+0000","last_fullsized":"2024-09-12T12:11:13.491176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:32.498628+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:11:32.498628+0000","last_peered":"2024-09-12T12:11:32.498628+0000","last_clean":"2024-09-12T12:11:32.498628+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:11:32.498628+0000","last_undegraded":"2024-09-12T12:11:32.498628+0000","last_fullsized":"2024-09-12T12:11:32.498628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498306+0000","last_change":"2024-09-12T12:11:12.506733+0000","last_active":"2024-09-12T12:11:13.498306+0000","last_peered":"2024-09-12T12:11:13.498306+0000","last_clean":"2024-09-12T12:11:13.498306+0000","last_became_active":"2024-09-12T12:11:12.506447+0000","last_became_peered":"2024-09-12T12:11:12.506447+0000","last_unstale":"2024-09-12T12:11:13.498306+0000","last_undegraded":"2024-09-12T12:11:13.498306+0000","last_fullsized":"2024-09-12T12:11:13.498306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:36.338767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.544999+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:11:20.544999+0000","last_peered":"2024-09-12T12:11:20.544999+0000","last_clean":"2024-09-12T12:11:20.544999+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:11:20.544999+0000","last_undegraded":"2024-09-12T12:11:20.544999+0000","last_fullsized":"2024-09-12T12:11:20.544999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500465+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:11:13.500465+0000","last_peered":"2024-09-12T12:11:13.500465+0000","last_clean":"2024-09-12T12:11:13.500465+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:11:13.500465+0000","last_undegraded":"2024-09-12T12:11:13.500465+0000","last_fullsized":"2024-09-12T12:11:13.500465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545274+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:11:20.545274+0000","last_peered":"2024-09-12T12:11:20.545274+0000","last_clean":"2024-09-12T12:11:20.545274+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:11:20.545274+0000","last_undegraded":"2024-09-12T12:11:20.545274+0000","last_fullsized":"2024-09-12T12:11:20.545274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502756+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:11:13.502756+0000","last_peered":"2024-09-12T12:11:13.502756+0000","last_clean":"2024-09-12T12:11:13.502756+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:11:13.502756+0000","last_undegraded":"2024-09-12T12:11:13.502756+0000","last_fullsized":"2024-09-12T12:11:13.502756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545237+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:11:20.545237+0000","last_peered":"2024-09-12T12:11:20.545237+0000","last_clean":"2024-09-12T12:11:20.545237+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:11:20.545237+0000","last_undegraded":"2024-09-12T12:11:20.545237+0000","last_fullsized":"2024-09-12T12:11:20.545237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491769+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:11:13.491769+0000","last_peered":"2024-09-12T12:11:13.491769+0000","last_clean":"2024-09-12T12:11:13.491769+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:11:13.491769+0000","last_undegraded":"2024-09-12T12:11:13.491769+0000","last_fullsized":"2024-09-12T12:11:13.491769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498164+0000","last_change":"2024-09-12T12:11:09.477481+0000","last_active":"2024-09-12T12:11:13.498164+0000","last_peered":"2024-09-12T12:11:13.498164+0000","last_clean":"2024-09-12T12:11:13.498164+0000","last_became_active":"2024-09-12T12:11:09.477126+0000","last_became_peered":"2024-09-12T12:11:09.477126+0000","last_unstale":"2024-09-12T12:11:13.498164+0000","last_undegraded":"2024-09-12T12:11:13.498164+0000","last_fullsized":"2024-09-12T12:11:13.498164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:57.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497665+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:11:13.497665+0000","last_peered":"2024-09-12T12:11:13.497665+0000","last_clean":"2024-09-12T12:11:13.497665+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:11:13.497665+0000","last_undegraded":"2024-09-12T12:11:13.497665+0000","last_fullsized":"2024-09-12T12:11:13.497665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545706+0000","last_change":"2024-09-12T12:11:09.484682+0000","last_active":"2024-09-12T12:11:20.545706+0000","last_peered":"2024-09-12T12:11:20.545706+0000","last_clean":"2024-09-12T12:11:20.545706+0000","last_became_active":"2024-09-12T12:11:09.484294+0000","last_became_peered":"2024-09-12T12:11:09.484294+0000","last_unstale":"2024-09-12T12:11:20.545706+0000","last_undegraded":"2024-09-12T12:11:20.545706+0000","last_fullsized":"2024-09-12T12:11:20.545706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:01:39.731705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504162+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:11:13.504162+0000","last_peered":"2024-09-12T12:11:13.504162+0000","last_clean":"2024-09-12T12:11:13.504162+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:11:13.504162+0000","last_undegraded":"2024-09-12T12:11:13.504162+0000","last_fullsized":"2024-09-12T12:11:13.504162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501487+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:11:13.501487+0000","last_peered":"2024-09-12T12:11:13.501487+0000","last_clean":"2024-09-12T12:11:13.501487+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:11:13.501487+0000","last_undegraded":"2024-09-12T12:11:13.501487+0000","last_fullsized":"2024-09-12T12:11:13.501487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.507254+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:11:13.507254+0000","last_peered":"2024-09-12T12:11:13.507254+0000","last_clean":"2024-09-12T12:11:13.507254+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:11:13.507254+0000","last_undegraded":"2024-09-12T12:11:13.507254+0000","last_fullsized":"2024-09-12T12:11:13.507254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503536+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:11:13.503536+0000","last_peered":"2024-09-12T12:11:13.503536+0000","last_clean":"2024-09-12T12:11:13.503536+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:11:13.503536+0000","last_undegraded":"2024-09-12T12:11:13.503536+0000","last_fullsized":"2024-09-12T12:11:13.503536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557696+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:11:21.557696+0000","last_peered":"2024-09-12T12:11:21.557696+0000","last_clean":"2024-09-12T12:11:21.557696+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:11:21.557696+0000","last_undegraded":"2024-09-12T12:11:21.557696+0000","last_fullsized":"2024-09-12T12:11:21.557696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500538+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:11:13.500538+0000","last_peered":"2024-09-12T12:11:13.500538+0000","last_clean":"2024-09-12T12:11:13.500538+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:11:13.500538+0000","last_undegraded":"2024-09-12T12:11:13.500538+0000","last_fullsized":"2024-09-12T12:11:13.500538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.499817+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:11:13.499817+0000","last_peered":"2024-09-12T12:11:13.499817+0000","last_clean":"2024-09-12T12:11:13.499817+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:11:13.499817+0000","last_undegraded":"2024-09-12T12:11:13.499817+0000","last_fullsized":"2024-09-12T12:11:13.499817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498213+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:11:13.498213+0000","last_peered":"2024-09-12T12:11:13.498213+0000","last_clean":"2024-09-12T12:11:13.498213+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:11:13.498213+0000","last_undegraded":"2024-09-12T12:11:13.498213+0000","last_fullsized":"2024-09-12T12:11:13.498213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557140+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:11:21.557140+0000","last_peered":"2024-09-12T12:11:21.557140+0000","last_clean":"2024-09-12T12:11:21.557140+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:11:21.557140+0000","last_undegraded":"2024-09-12T12:11:21.557140+0000","last_fullsized":"2024-09-12T12:11:21.557140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50805,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":98,"num_read_kb":102,"num_write":150,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":152415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129,"ondisk_log_size":129,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":427186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1065862,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":427342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1019274,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":426811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1018565,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":427122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":474756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":473406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":48152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:11:38.856 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:11:39.099 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:39.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:39 smithi171 ceph-mon[35098]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:11:39.520 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:39 smithi137 ceph-mon[30950]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:11:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:39 smithi189 ceph-mon[35475]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:11:39.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.598+0000 7f42f34d3640 1 -- 172.21.15.137:0/129386065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec100e20 msgr2=0x7f42ec101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:39.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.598+0000 7f42f34d3640 1 --2- 172.21.15.137:0/129386065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec100e20 0x7f42ec101200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009900 tx=0x7f42e002a480 comp rx=0 tx=0).stop 2024-09-12T12:11:39.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- 172.21.15.137:0/129386065 shutdown_connections 2024-09-12T12:11:39.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 --2- 172.21.15.137:0/129386065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ec102ef0 0x7f42ec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 --2- 172.21.15.137:0/129386065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42ec102570 0x7f42ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 --2- 172.21.15.137:0/129386065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec100e20 0x7f42ec101200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- 172.21.15.137:0/129386065 >> 172.21.15.137:0/129386065 conn(0x7f42ec0fc7d0 msgr2=0x7f42ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:39.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- 172.21.15.137:0/129386065 shutdown_connections 2024-09-12T12:11:39.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- 172.21.15.137:0/129386065 wait complete. 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 Processor -- start 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- start start 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42ec100e20 0x7f42ec103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec102570 0x7f42ec1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ec102ef0 0x7f42ec104710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec110fc0 con 0x7f42ec102570 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec110e40 con 0x7f42ec102ef0 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.599+0000 7f42f34d3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec111140 con 0x7f42ec100e20 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec102570 0x7f42ec1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec102570 0x7f42ec1041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35738/0 (socket says 172.21.15.137:35738) 2024-09-12T12:11:39.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 -- 172.21.15.137:0/3240559967 learned_addr learned my addr 172.21.15.137:0/3240559967 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f1a70640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ec102ef0 0x7f42ec104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f126f640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42ec100e20 0x7f42ec103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 -- 172.21.15.137:0/3240559967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42ec100e20 msgr2=0x7f42ec103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42ec100e20 0x7f42ec103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 -- 172.21.15.137:0/3240559967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ec102ef0 msgr2=0x7f42ec104710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ec102ef0 0x7f42ec104710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 -- 172.21.15.137:0/3240559967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec1a6a20 con 0x7f42ec102570 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f1a70640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ec102ef0 0x7f42ec104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f126f640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42ec100e20 0x7f42ec103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f0a6e640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec102570 0x7f42ec1041d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42e800c970 tx=0x7f42e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42de7fc640 1 -- 172.21.15.137:0/3240559967 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8007bb0 con 0x7f42ec102570 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ec1a6d10 con 0x7f42ec102570 2024-09-12T12:11:39.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42de7fc640 1 -- 172.21.15.137:0/3240559967 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42e8007d50 con 0x7f42ec102570 2024-09-12T12:11:39.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42de7fc640 1 -- 172.21.15.137:0/3240559967 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e800f450 con 0x7f42ec102570 2024-09-12T12:11:39.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.600+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ec1a71f0 con 0x7f42ec102570 2024-09-12T12:11:39.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.601+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ec101200 con 0x7f42ec102570 2024-09-12T12:11:39.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.602+0000 7f42de7fc640 1 -- 172.21.15.137:0/3240559967 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f42e8005020 con 0x7f42ec102570 2024-09-12T12:11:39.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.603+0000 7f42de7fc640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f42d80a2c10 0x7f42d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:39.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.603+0000 7f42f126f640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f42d80a2c10 0x7f42d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:39.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.603+0000 7f42de7fc640 1 -- 172.21.15.137:0/3240559967 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f42e80ceec0 con 0x7f42ec102570 2024-09-12T12:11:39.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.603+0000 7f42f126f640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f42d80a2c10 0x7f42d80a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42e0000c00 tx=0x7f42e00041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:39.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.607+0000 7f42de7fc640 1 -- 172.21.15.137:0/3240559967 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42e8014030 con 0x7f42ec102570 2024-09-12T12:11:39.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.772+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42ec0fdfa0 con 0x7f42d80a2c10 2024-09-12T12:11:39.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.778+0000 7f42de7fc640 1 -- 172.21.15.137:0/3240559967 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258863 (secure 0 0 0) 0x7f42ec0fdfa0 con 0x7f42d80a2c10 2024-09-12T12:11:39.781 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:11:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f42d80a2c10 msgr2=0x7f42d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f42d80a2c10 0x7f42d80a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42e0000c00 tx=0x7f42e00041e0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec102570 msgr2=0x7f42ec1041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec102570 0x7f42ec1041d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f42e800c970 tx=0x7f42e800ce30 comp rx=0 tx=0).stop 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 shutdown_connections 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f42d80a2c10 0x7f42d80a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42ec102ef0 0x7f42ec104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42ec102570 0x7f42ec1041d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 --2- 172.21.15.137:0/3240559967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42ec100e20 0x7f42ec103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.782+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 >> 172.21.15.137:0/3240559967 conn(0x7f42ec0fc7d0 msgr2=0x7f42ec105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.783+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 shutdown_connections 2024-09-12T12:11:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:39.783+0000 7f42f34d3640 1 -- 172.21.15.137:0/3240559967 wait complete. 2024-09-12T12:11:40.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:40 smithi171 ceph-mon[35098]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-12T12:11:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:40 smithi137 ceph-mon[30950]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-12T12:11:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:40 smithi189 ceph-mon[35475]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-09-12T12:11:40.631 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-12T12:11:39.115791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649559,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":250,"num_read_kb":239,"num_write":336,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355972,"kb_used_data":10884,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585852,"statfs":{"total":1151940427776,"available":1151575912448,"internally_reserved":0,"allocated":11145216,"data_stored":7067925,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53969,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":95,"num_write":30,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002710"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491849+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:11:13.491849+0000","last_peered":"2024-09-12T12:11:13.491849+0000","last_clean":"2024-09-12T12:11:13.491849+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:11:13.491849+0000","last_undegraded":"2024-09-12T12:11:13.491849+0000","last_fullsized":"2024-09-12T12:11:13.491849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500296+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:11:13.500296+0000","last_peered":"2024-09-12T12:11:13.500296+0000","last_clean":"2024-09-12T12:11:13.500296+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:11:13.500296+0000","last_undegraded":"2024-09-12T12:11:13.500296+0000","last_fullsized":"2024-09-12T12:11:13.500296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500541+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:11:13.500541+0000","last_peered":"2024-09-12T12:11:13.500541+0000","last_clean":"2024-09-12T12:11:13.500541+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:11:13.500541+0000","last_undegraded":"2024-09-12T12:11:13.500541+0000","last_fullsized":"2024-09-12T12:11:13.500541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492536+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:11:13.492536+0000","last_peered":"2024-09-12T12:11:13.492536+0000","last_clean":"2024-09-12T12:11:13.492536+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:11:13.492536+0000","last_undegraded":"2024-09-12T12:11:13.492536+0000","last_fullsized":"2024-09-12T12:11:13.492536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501335+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:11:13.501335+0000","last_peered":"2024-09-12T12:11:13.501335+0000","last_clean":"2024-09-12T12:11:13.501335+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:11:13.501335+0000","last_undegraded":"2024-09-12T12:11:13.501335+0000","last_fullsized":"2024-09-12T12:11:13.501335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500893+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:11:13.500893+0000","last_peered":"2024-09-12T12:11:13.500893+0000","last_clean":"2024-09-12T12:11:13.500893+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:11:13.500893+0000","last_undegraded":"2024-09-12T12:11:13.500893+0000","last_fullsized":"2024-09-12T12:11:13.500893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503506+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:11:13.503506+0000","last_peered":"2024-09-12T12:11:13.503506+0000","last_clean":"2024-09-12T12:11:13.503506+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:11:13.503506+0000","last_undegraded":"2024-09-12T12:11:13.503506+0000","last_fullsized":"2024-09-12T12:11:13.503506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498001+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:11:13.498001+0000","last_peered":"2024-09-12T12:11:13.498001+0000","last_clean":"2024-09-12T12:11:13.498001+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:11:13.498001+0000","last_undegraded":"2024-09-12T12:11:13.498001+0000","last_fullsized":"2024-09-12T12:11:13.498001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503514+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:11:13.503514+0000","last_peered":"2024-09-12T12:11:13.503514+0000","last_clean":"2024-09-12T12:11:13.503514+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:11:13.503514+0000","last_undegraded":"2024-09-12T12:11:13.503514+0000","last_fullsized":"2024-09-12T12:11:13.503514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501033+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:11:13.501033+0000","last_peered":"2024-09-12T12:11:13.501033+0000","last_clean":"2024-09-12T12:11:13.501033+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:11:13.501033+0000","last_undegraded":"2024-09-12T12:11:13.501033+0000","last_fullsized":"2024-09-12T12:11:13.501033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504240+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:11:13.504240+0000","last_peered":"2024-09-12T12:11:13.504240+0000","last_clean":"2024-09-12T12:11:13.504240+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:11:13.504240+0000","last_undegraded":"2024-09-12T12:11:13.504240+0000","last_fullsized":"2024-09-12T12:11:13.504240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492580+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:11:13.492580+0000","last_peered":"2024-09-12T12:11:13.492580+0000","last_clean":"2024-09-12T12:11:13.492580+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:11:13.492580+0000","last_undegraded":"2024-09-12T12:11:13.492580+0000","last_fullsized":"2024-09-12T12:11:13.492580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491145+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:11:13.491145+0000","last_peered":"2024-09-12T12:11:13.491145+0000","last_clean":"2024-09-12T12:11:13.491145+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:11:13.491145+0000","last_undegraded":"2024-09-12T12:11:13.491145+0000","last_fullsized":"2024-09-12T12:11:13.491145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500084+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:11:13.500084+0000","last_peered":"2024-09-12T12:11:13.500084+0000","last_clean":"2024-09-12T12:11:13.500084+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:11:13.500084+0000","last_undegraded":"2024-09-12T12:11:13.500084+0000","last_fullsized":"2024-09-12T12:11:13.500084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.499972+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:11:13.499972+0000","last_peered":"2024-09-12T12:11:13.499972+0000","last_clean":"2024-09-12T12:11:13.499972+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:11:13.499972+0000","last_undegraded":"2024-09-12T12:11:13.499972+0000","last_fullsized":"2024-09-12T12:11:13.499972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502792+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:11:13.502792+0000","last_peered":"2024-09-12T12:11:13.502792+0000","last_clean":"2024-09-12T12:11:13.502792+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:11:13.502792+0000","last_undegraded":"2024-09-12T12:11:13.502792+0000","last_fullsized":"2024-09-12T12:11:13.502792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491907+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:11:13.491907+0000","last_peered":"2024-09-12T12:11:13.491907+0000","last_clean":"2024-09-12T12:11:13.491907+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:11:13.491907+0000","last_undegraded":"2024-09-12T12:11:13.491907+0000","last_fullsized":"2024-09-12T12:11:13.491907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502053+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:11:13.502053+0000","last_peered":"2024-09-12T12:11:13.502053+0000","last_clean":"2024-09-12T12:11:13.502053+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:11:13.502053+0000","last_undegraded":"2024-09-12T12:11:13.502053+0000","last_fullsized":"2024-09-12T12:11:13.502053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498689+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:11:13.498689+0000","last_peered":"2024-09-12T12:11:13.498689+0000","last_clean":"2024-09-12T12:11:13.498689+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:11:13.498689+0000","last_undegraded":"2024-09-12T12:11:13.498689+0000","last_fullsized":"2024-09-12T12:11:13.498689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491690+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:11:13.491690+0000","last_peered":"2024-09-12T12:11:13.491690+0000","last_clean":"2024-09-12T12:11:13.491690+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:11:13.491690+0000","last_undegraded":"2024-09-12T12:11:13.491690+0000","last_fullsized":"2024-09-12T12:11:13.491690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491185+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:11:13.491185+0000","last_peered":"2024-09-12T12:11:13.491185+0000","last_clean":"2024-09-12T12:11:13.491185+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:11:13.491185+0000","last_undegraded":"2024-09-12T12:11:13.491185+0000","last_fullsized":"2024-09-12T12:11:13.491185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503390+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:11:13.503390+0000","last_peered":"2024-09-12T12:11:13.503390+0000","last_clean":"2024-09-12T12:11:13.503390+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:11:13.503390+0000","last_undegraded":"2024-09-12T12:11:13.503390+0000","last_fullsized":"2024-09-12T12:11:13.503390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497730+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:11:13.497730+0000","last_peered":"2024-09-12T12:11:13.497730+0000","last_clean":"2024-09-12T12:11:13.497730+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:11:13.497730+0000","last_undegraded":"2024-09-12T12:11:13.497730+0000","last_fullsized":"2024-09-12T12:11:13.497730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500719+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:11:13.500719+0000","last_peered":"2024-09-12T12:11:13.500719+0000","last_clean":"2024-09-12T12:11:13.500719+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:11:13.500719+0000","last_undegraded":"2024-09-12T12:11:13.500719+0000","last_fullsized":"2024-09-12T12:11:13.500719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498394+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:11:13.498394+0000","last_peered":"2024-09-12T12:11:13.498394+0000","last_clean":"2024-09-12T12:11:13.498394+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:11:13.498394+0000","last_undegraded":"2024-09-12T12:11:13.498394+0000","last_fullsized":"2024-09-12T12:11:13.498394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498501+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:11:13.498501+0000","last_peered":"2024-09-12T12:11:13.498501+0000","last_clean":"2024-09-12T12:11:13.498501+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:11:13.498501+0000","last_undegraded":"2024-09-12T12:11:13.498501+0000","last_fullsized":"2024-09-12T12:11:13.498501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491855+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:11:13.491855+0000","last_peered":"2024-09-12T12:11:13.491855+0000","last_clean":"2024-09-12T12:11:13.491855+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:11:13.491855+0000","last_undegraded":"2024-09-12T12:11:13.491855+0000","last_fullsized":"2024-09-12T12:11:13.491855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491115+0000","last_change":"2024-09-12T12:11:12.500411+0000","last_active":"2024-09-12T12:11:13.491115+0000","last_peered":"2024-09-12T12:11:13.491115+0000","last_clean":"2024-09-12T12:11:13.491115+0000","last_became_active":"2024-09-12T12:11:12.500126+0000","last_became_peered":"2024-09-12T12:11:12.500126+0000","last_unstale":"2024-09-12T12:11:13.491115+0000","last_undegraded":"2024-09-12T12:11:13.491115+0000","last_fullsized":"2024-09-12T12:11:13.491115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:23:49.072306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500514+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:11:13.500514+0000","last_peered":"2024-09-12T12:11:13.500514+0000","last_clean":"2024-09-12T12:11:13.500514+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:11:13.500514+0000","last_undegraded":"2024-09-12T12:11:13.500514+0000","last_fullsized":"2024-09-12T12:11:13.500514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498074+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:11:13.498074+0000","last_peered":"2024-09-12T12:11:13.498074+0000","last_clean":"2024-09-12T12:11:13.498074+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:11:13.498074+0000","last_undegraded":"2024-09-12T12:11:13.498074+0000","last_fullsized":"2024-09-12T12:11:13.498074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498204+0000","last_change":"2024-09-12T12:11:12.510232+0000","last_active":"2024-09-12T12:11:13.498204+0000","last_peered":"2024-09-12T12:11:13.498204+0000","last_clean":"2024-09-12T12:11:13.498204+0000","last_became_active":"2024-09-12T12:11:12.509856+0000","last_became_peered":"2024-09-12T12:11:12.509856+0000","last_unstale":"2024-09-12T12:11:13.498204+0000","last_undegraded":"2024-09-12T12:11:13.498204+0000","last_fullsized":"2024-09-12T12:11:13.498204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:56.293960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491659+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:11:13.491659+0000","last_peered":"2024-09-12T12:11:13.491659+0000","last_clean":"2024-09-12T12:11:13.491659+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:11:13.491659+0000","last_undegraded":"2024-09-12T12:11:13.491659+0000","last_fullsized":"2024-09-12T12:11:13.491659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501135+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:11:13.501135+0000","last_peered":"2024-09-12T12:11:13.501135+0000","last_clean":"2024-09-12T12:11:13.501135+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:11:13.501135+0000","last_undegraded":"2024-09-12T12:11:13.501135+0000","last_fullsized":"2024-09-12T12:11:13.501135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":41,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:34.485355+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:11:34.485355+0000","last_peered":"2024-09-12T12:11:34.485355+0000","last_clean":"2024-09-12T12:11:34.485355+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:11:34.485355+0000","last_undegraded":"2024-09-12T12:11:34.485355+0000","last_fullsized":"2024-09-12T12:11:34.485355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500766+0000","last_change":"2024-09-12T12:11:12.508286+0000","last_active":"2024-09-12T12:11:13.500766+0000","last_peered":"2024-09-12T12:11:13.500766+0000","last_clean":"2024-09-12T12:11:13.500766+0000","last_became_active":"2024-09-12T12:11:12.507860+0000","last_became_peered":"2024-09-12T12:11:12.507860+0000","last_unstale":"2024-09-12T12:11:13.500766+0000","last_undegraded":"2024-09-12T12:11:13.500766+0000","last_fullsized":"2024-09-12T12:11:13.500766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:15:49.330419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556617+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:11:21.556617+0000","last_peered":"2024-09-12T12:11:21.556617+0000","last_clean":"2024-09-12T12:11:21.556617+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:11:21.556617+0000","last_undegraded":"2024-09-12T12:11:21.556617+0000","last_fullsized":"2024-09-12T12:11:21.556617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556515+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:11:21.556515+0000","last_peered":"2024-09-12T12:11:21.556515+0000","last_clean":"2024-09-12T12:11:21.556515+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:11:21.556515+0000","last_undegraded":"2024-09-12T12:11:21.556515+0000","last_fullsized":"2024-09-12T12:11:21.556515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497911+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:11:13.497911+0000","last_peered":"2024-09-12T12:11:13.497911+0000","last_clean":"2024-09-12T12:11:13.497911+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:11:13.497911+0000","last_undegraded":"2024-09-12T12:11:13.497911+0000","last_fullsized":"2024-09-12T12:11:13.497911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501882+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:11:13.501882+0000","last_peered":"2024-09-12T12:11:13.501882+0000","last_clean":"2024-09-12T12:11:13.501882+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:11:13.501882+0000","last_undegraded":"2024-09-12T12:11:13.501882+0000","last_fullsized":"2024-09-12T12:11:13.501882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501931+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:11:13.501931+0000","last_peered":"2024-09-12T12:11:13.501931+0000","last_clean":"2024-09-12T12:11:13.501931+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:11:13.501931+0000","last_undegraded":"2024-09-12T12:11:13.501931+0000","last_fullsized":"2024-09-12T12:11:13.501931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:36.488005+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:11:36.488005+0000","last_peered":"2024-09-12T12:11:36.488005+0000","last_clean":"2024-09-12T12:11:36.488005+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:11:36.488005+0000","last_undegraded":"2024-09-12T12:11:36.488005+0000","last_fullsized":"2024-09-12T12:11:36.488005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503842+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:11:13.503842+0000","last_peered":"2024-09-12T12:11:13.503842+0000","last_clean":"2024-09-12T12:11:13.503842+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:11:13.503842+0000","last_undegraded":"2024-09-12T12:11:13.503842+0000","last_fullsized":"2024-09-12T12:11:13.503842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.543897+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:11:20.543897+0000","last_peered":"2024-09-12T12:11:20.543897+0000","last_clean":"2024-09-12T12:11:20.543897+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:11:20.543897+0000","last_undegraded":"2024-09-12T12:11:20.543897+0000","last_fullsized":"2024-09-12T12:11:20.543897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500509+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:11:13.500509+0000","last_peered":"2024-09-12T12:11:13.500509+0000","last_clean":"2024-09-12T12:11:13.500509+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:11:13.500509+0000","last_undegraded":"2024-09-12T12:11:13.500509+0000","last_fullsized":"2024-09-12T12:11:13.500509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545077+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:11:20.545077+0000","last_peered":"2024-09-12T12:11:20.545077+0000","last_clean":"2024-09-12T12:11:20.545077+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:11:20.545077+0000","last_undegraded":"2024-09-12T12:11:20.545077+0000","last_fullsized":"2024-09-12T12:11:20.545077+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503678+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:11:13.503678+0000","last_peered":"2024-09-12T12:11:13.503678+0000","last_clean":"2024-09-12T12:11:13.503678+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:11:13.503678+0000","last_undegraded":"2024-09-12T12:11:13.503678+0000","last_fullsized":"2024-09-12T12:11:13.503678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'25","reported_seq":84,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:37.492233+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:11:37.492233+0000","last_peered":"2024-09-12T12:11:37.492233+0000","last_clean":"2024-09-12T12:11:37.492233+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:11:37.492233+0000","last_undegraded":"2024-09-12T12:11:37.492233+0000","last_fullsized":"2024-09-12T12:11:37.492233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54954,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":72,"num_write":26,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503335+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:11:13.503335+0000","last_peered":"2024-09-12T12:11:13.503335+0000","last_clean":"2024-09-12T12:11:13.503335+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:11:13.503335+0000","last_undegraded":"2024-09-12T12:11:13.503335+0000","last_fullsized":"2024-09-12T12:11:13.503335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557533+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:11:21.557533+0000","last_peered":"2024-09-12T12:11:21.557533+0000","last_clean":"2024-09-12T12:11:21.557533+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:11:21.557533+0000","last_undegraded":"2024-09-12T12:11:21.557533+0000","last_fullsized":"2024-09-12T12:11:21.557533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490815+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:11:13.490815+0000","last_peered":"2024-09-12T12:11:13.490815+0000","last_clean":"2024-09-12T12:11:13.490815+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:11:13.490815+0000","last_undegraded":"2024-09-12T12:11:13.490815+0000","last_fullsized":"2024-09-12T12:11:13.490815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.546044+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:11:20.546044+0000","last_peered":"2024-09-12T12:11:20.546044+0000","last_clean":"2024-09-12T12:11:20.546044+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:11:20.546044+0000","last_undegraded":"2024-09-12T12:11:20.546044+0000","last_fullsized":"2024-09-12T12:11:20.546044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503539+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:11:13.503539+0000","last_peered":"2024-09-12T12:11:13.503539+0000","last_clean":"2024-09-12T12:11:13.503539+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:11:13.503539+0000","last_undegraded":"2024-09-12T12:11:13.503539+0000","last_fullsized":"2024-09-12T12:11:13.503539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:33.484365+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:11:33.484365+0000","last_peered":"2024-09-12T12:11:33.484365+0000","last_clean":"2024-09-12T12:11:33.484365+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:11:33.484365+0000","last_undegraded":"2024-09-12T12:11:33.484365+0000","last_fullsized":"2024-09-12T12:11:33.484365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503667+0000","last_change":"2024-09-12T12:11:12.504047+0000","last_active":"2024-09-12T12:11:13.503667+0000","last_peered":"2024-09-12T12:11:13.503667+0000","last_clean":"2024-09-12T12:11:13.503667+0000","last_became_active":"2024-09-12T12:11:12.503648+0000","last_became_peered":"2024-09-12T12:11:12.503648+0000","last_unstale":"2024-09-12T12:11:13.503667+0000","last_undegraded":"2024-09-12T12:11:13.503667+0000","last_fullsized":"2024-09-12T12:11:13.503667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:43.443802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.558326+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:11:21.558326+0000","last_peered":"2024-09-12T12:11:21.558326+0000","last_clean":"2024-09-12T12:11:21.558326+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:11:21.558326+0000","last_undegraded":"2024-09-12T12:11:21.558326+0000","last_fullsized":"2024-09-12T12:11:21.558326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501026+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:11:13.501026+0000","last_peered":"2024-09-12T12:11:13.501026+0000","last_clean":"2024-09-12T12:11:13.501026+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:11:13.501026+0000","last_undegraded":"2024-09-12T12:11:13.501026+0000","last_fullsized":"2024-09-12T12:11:13.501026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:22.466026+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:11:22.466026+0000","last_peered":"2024-09-12T12:11:22.466026+0000","last_clean":"2024-09-12T12:11:22.466026+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:11:22.466026+0000","last_undegraded":"2024-09-12T12:11:22.466026+0000","last_fullsized":"2024-09-12T12:11:22.466026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491726+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:11:13.491726+0000","last_peered":"2024-09-12T12:11:13.491726+0000","last_clean":"2024-09-12T12:11:13.491726+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:11:13.491726+0000","last_undegraded":"2024-09-12T12:11:13.491726+0000","last_fullsized":"2024-09-12T12:11:13.491726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497437+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:11:13.497437+0000","last_peered":"2024-09-12T12:11:13.497437+0000","last_clean":"2024-09-12T12:11:13.497437+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:11:13.497437+0000","last_undegraded":"2024-09-12T12:11:13.497437+0000","last_fullsized":"2024-09-12T12:11:13.497437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504243+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:11:13.504243+0000","last_peered":"2024-09-12T12:11:13.504243+0000","last_clean":"2024-09-12T12:11:13.504243+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:11:13.504243+0000","last_undegraded":"2024-09-12T12:11:13.504243+0000","last_fullsized":"2024-09-12T12:11:13.504243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.507452+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:11:13.507452+0000","last_peered":"2024-09-12T12:11:13.507452+0000","last_clean":"2024-09-12T12:11:13.507452+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:11:13.507452+0000","last_undegraded":"2024-09-12T12:11:13.507452+0000","last_fullsized":"2024-09-12T12:11:13.507452+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503836+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:11:13.503836+0000","last_peered":"2024-09-12T12:11:13.503836+0000","last_clean":"2024-09-12T12:11:13.503836+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:11:13.503836+0000","last_undegraded":"2024-09-12T12:11:13.503836+0000","last_fullsized":"2024-09-12T12:11:13.503836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557550+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:11:21.557550+0000","last_peered":"2024-09-12T12:11:21.557550+0000","last_clean":"2024-09-12T12:11:21.557550+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:11:21.557550+0000","last_undegraded":"2024-09-12T12:11:21.557550+0000","last_fullsized":"2024-09-12T12:11:21.557550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.505016+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:11:13.505016+0000","last_peered":"2024-09-12T12:11:13.505016+0000","last_clean":"2024-09-12T12:11:13.505016+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:11:13.505016+0000","last_undegraded":"2024-09-12T12:11:13.505016+0000","last_fullsized":"2024-09-12T12:11:13.505016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503794+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:11:13.503794+0000","last_peered":"2024-09-12T12:11:13.503794+0000","last_clean":"2024-09-12T12:11:13.503794+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:11:13.503794+0000","last_undegraded":"2024-09-12T12:11:13.503794+0000","last_fullsized":"2024-09-12T12:11:13.503794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:22.465059+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:11:22.465059+0000","last_peered":"2024-09-12T12:11:22.465059+0000","last_clean":"2024-09-12T12:11:22.465059+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:11:22.465059+0000","last_undegraded":"2024-09-12T12:11:22.465059+0000","last_fullsized":"2024-09-12T12:11:22.465059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498114+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:11:13.498114+0000","last_peered":"2024-09-12T12:11:13.498114+0000","last_clean":"2024-09-12T12:11:13.498114+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:11:13.498114+0000","last_undegraded":"2024-09-12T12:11:13.498114+0000","last_fullsized":"2024-09-12T12:11:13.498114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556747+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:11:21.556747+0000","last_peered":"2024-09-12T12:11:21.556747+0000","last_clean":"2024-09-12T12:11:21.556747+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:11:21.556747+0000","last_undegraded":"2024-09-12T12:11:21.556747+0000","last_fullsized":"2024-09-12T12:11:21.556747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490784+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:11:13.490784+0000","last_peered":"2024-09-12T12:11:13.490784+0000","last_clean":"2024-09-12T12:11:13.490784+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:11:13.490784+0000","last_undegraded":"2024-09-12T12:11:13.490784+0000","last_fullsized":"2024-09-12T12:11:13.490784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:33.483943+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:11:33.483943+0000","last_peered":"2024-09-12T12:11:33.483943+0000","last_clean":"2024-09-12T12:11:33.483943+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:11:33.483943+0000","last_undegraded":"2024-09-12T12:11:33.483943+0000","last_fullsized":"2024-09-12T12:11:33.483943+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500783+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:11:13.500783+0000","last_peered":"2024-09-12T12:11:13.500783+0000","last_clean":"2024-09-12T12:11:13.500783+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:11:13.500783+0000","last_undegraded":"2024-09-12T12:11:13.500783+0000","last_fullsized":"2024-09-12T12:11:13.500783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504416+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:11:13.504416+0000","last_peered":"2024-09-12T12:11:13.504416+0000","last_clean":"2024-09-12T12:11:13.504416+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:11:13.504416+0000","last_undegraded":"2024-09-12T12:11:13.504416+0000","last_fullsized":"2024-09-12T12:11:13.504416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500974+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:11:13.500974+0000","last_peered":"2024-09-12T12:11:13.500974+0000","last_clean":"2024-09-12T12:11:13.500974+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:11:13.500974+0000","last_undegraded":"2024-09-12T12:11:13.500974+0000","last_fullsized":"2024-09-12T12:11:13.500974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501897+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:11:13.501897+0000","last_peered":"2024-09-12T12:11:13.501897+0000","last_clean":"2024-09-12T12:11:13.501897+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:11:13.501897+0000","last_undegraded":"2024-09-12T12:11:13.501897+0000","last_fullsized":"2024-09-12T12:11:13.501897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500355+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:11:13.500355+0000","last_peered":"2024-09-12T12:11:13.500355+0000","last_clean":"2024-09-12T12:11:13.500355+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:11:13.500355+0000","last_undegraded":"2024-09-12T12:11:13.500355+0000","last_fullsized":"2024-09-12T12:11:13.500355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.544273+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:11:20.544273+0000","last_peered":"2024-09-12T12:11:20.544273+0000","last_clean":"2024-09-12T12:11:20.544273+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:11:20.544273+0000","last_undegraded":"2024-09-12T12:11:20.544273+0000","last_fullsized":"2024-09-12T12:11:20.544273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491176+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:11:13.491176+0000","last_peered":"2024-09-12T12:11:13.491176+0000","last_clean":"2024-09-12T12:11:13.491176+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:11:13.491176+0000","last_undegraded":"2024-09-12T12:11:13.491176+0000","last_fullsized":"2024-09-12T12:11:13.491176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'8","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:32.498628+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:11:32.498628+0000","last_peered":"2024-09-12T12:11:32.498628+0000","last_clean":"2024-09-12T12:11:32.498628+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:11:32.498628+0000","last_undegraded":"2024-09-12T12:11:32.498628+0000","last_fullsized":"2024-09-12T12:11:32.498628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498306+0000","last_change":"2024-09-12T12:11:12.506733+0000","last_active":"2024-09-12T12:11:13.498306+0000","last_peered":"2024-09-12T12:11:13.498306+0000","last_clean":"2024-09-12T12:11:13.498306+0000","last_became_active":"2024-09-12T12:11:12.506447+0000","last_became_peered":"2024-09-12T12:11:12.506447+0000","last_unstale":"2024-09-12T12:11:13.498306+0000","last_undegraded":"2024-09-12T12:11:13.498306+0000","last_fullsized":"2024-09-12T12:11:13.498306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:36.338767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.544999+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:11:20.544999+0000","last_peered":"2024-09-12T12:11:20.544999+0000","last_clean":"2024-09-12T12:11:20.544999+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:11:20.544999+0000","last_undegraded":"2024-09-12T12:11:20.544999+0000","last_fullsized":"2024-09-12T12:11:20.544999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500465+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:11:13.500465+0000","last_peered":"2024-09-12T12:11:13.500465+0000","last_clean":"2024-09-12T12:11:13.500465+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:11:13.500465+0000","last_undegraded":"2024-09-12T12:11:13.500465+0000","last_fullsized":"2024-09-12T12:11:13.500465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545274+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:11:20.545274+0000","last_peered":"2024-09-12T12:11:20.545274+0000","last_clean":"2024-09-12T12:11:20.545274+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:11:20.545274+0000","last_undegraded":"2024-09-12T12:11:20.545274+0000","last_fullsized":"2024-09-12T12:11:20.545274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502756+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:11:13.502756+0000","last_peered":"2024-09-12T12:11:13.502756+0000","last_clean":"2024-09-12T12:11:13.502756+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:11:13.502756+0000","last_undegraded":"2024-09-12T12:11:13.502756+0000","last_fullsized":"2024-09-12T12:11:13.502756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545237+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:11:20.545237+0000","last_peered":"2024-09-12T12:11:20.545237+0000","last_clean":"2024-09-12T12:11:20.545237+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:11:20.545237+0000","last_undegraded":"2024-09-12T12:11:20.545237+0000","last_fullsized":"2024-09-12T12:11:20.545237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491769+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:11:13.491769+0000","last_peered":"2024-09-12T12:11:13.491769+0000","last_clean":"2024-09-12T12:11:13.491769+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:11:13.491769+0000","last_undegraded":"2024-09-12T12:11:13.491769+0000","last_fullsized":"2024-09-12T12:11:13.491769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498164+0000","last_change":"2024-09-12T12:11:09.477481+0000","last_active":"2024-09-12T12:11:13.498164+0000","last_peered":"2024-09-12T12:11:13.498164+0000","last_clean":"2024-09-12T12:11:13.498164+0000","last_became_active":"2024-09-12T12:11:09.477126+0000","last_became_peered":"2024-09-12T12:11:09.477126+0000","last_unstale":"2024-09-12T12:11:13.498164+0000","last_undegraded":"2024-09-12T12:11:13.498164+0000","last_fullsized":"2024-09-12T12:11:13.498164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:57.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497665+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:11:13.497665+0000","last_peered":"2024-09-12T12:11:13.497665+0000","last_clean":"2024-09-12T12:11:13.497665+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:11:13.497665+0000","last_undegraded":"2024-09-12T12:11:13.497665+0000","last_fullsized":"2024-09-12T12:11:13.497665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545706+0000","last_change":"2024-09-12T12:11:09.484682+0000","last_active":"2024-09-12T12:11:20.545706+0000","last_peered":"2024-09-12T12:11:20.545706+0000","last_clean":"2024-09-12T12:11:20.545706+0000","last_became_active":"2024-09-12T12:11:09.484294+0000","last_became_peered":"2024-09-12T12:11:09.484294+0000","last_unstale":"2024-09-12T12:11:20.545706+0000","last_undegraded":"2024-09-12T12:11:20.545706+0000","last_fullsized":"2024-09-12T12:11:20.545706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:01:39.731705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504162+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:11:13.504162+0000","last_peered":"2024-09-12T12:11:13.504162+0000","last_clean":"2024-09-12T12:11:13.504162+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:11:13.504162+0000","last_undegraded":"2024-09-12T12:11:13.504162+0000","last_fullsized":"2024-09-12T12:11:13.504162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501487+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:11:13.501487+0000","last_peered":"2024-09-12T12:11:13.501487+0000","last_clean":"2024-09-12T12:11:13.501487+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:11:13.501487+0000","last_undegraded":"2024-09-12T12:11:13.501487+0000","last_fullsized":"2024-09-12T12:11:13.501487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.507254+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:11:13.507254+0000","last_peered":"2024-09-12T12:11:13.507254+0000","last_clean":"2024-09-12T12:11:13.507254+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:11:13.507254+0000","last_undegraded":"2024-09-12T12:11:13.507254+0000","last_fullsized":"2024-09-12T12:11:13.507254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503536+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:11:13.503536+0000","last_peered":"2024-09-12T12:11:13.503536+0000","last_clean":"2024-09-12T12:11:13.503536+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:11:13.503536+0000","last_undegraded":"2024-09-12T12:11:13.503536+0000","last_fullsized":"2024-09-12T12:11:13.503536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557696+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:11:21.557696+0000","last_peered":"2024-09-12T12:11:21.557696+0000","last_clean":"2024-09-12T12:11:21.557696+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:11:21.557696+0000","last_undegraded":"2024-09-12T12:11:21.557696+0000","last_fullsized":"2024-09-12T12:11:21.557696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500538+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:11:13.500538+0000","last_peered":"2024-09-12T12:11:13.500538+0000","last_clean":"2024-09-12T12:11:13.500538+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:11:13.500538+0000","last_undegraded":"2024-09-12T12:11:13.500538+0000","last_fullsized":"2024-09-12T12:11:13.500538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.499817+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:11:13.499817+0000","last_peered":"2024-09-12T12:11:13.499817+0000","last_clean":"2024-09-12T12:11:13.499817+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:11:13.499817+0000","last_undegraded":"2024-09-12T12:11:13.499817+0000","last_fullsized":"2024-09-12T12:11:13.499817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498213+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:11:13.498213+0000","last_peered":"2024-09-12T12:11:13.498213+0000","last_clean":"2024-09-12T12:11:13.498213+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:11:13.498213+0000","last_undegraded":"2024-09-12T12:11:13.498213+0000","last_fullsized":"2024-09-12T12:11:13.498213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557140+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:11:21.557140+0000","last_peered":"2024-09-12T12:11:21.557140+0000","last_clean":"2024-09-12T12:11:21.557140+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:11:21.557140+0000","last_undegraded":"2024-09-12T12:11:21.557140+0000","last_fullsized":"2024-09-12T12:11:21.557140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58957,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":114,"num_read_kb":120,"num_write":153,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":176871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":427186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1074014,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168610,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":427342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397543,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1019274,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":426811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1018565,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":427122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":482908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":481558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:11:40.636 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T12:11:40.637 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T12:11:40.637 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T12:11:40.884 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:41.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:41 smithi137 ceph-mon[30950]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:11:41.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 -- 172.21.15.137:0/3134150501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 msgr2=0x7fdc9c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:41.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/3134150501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c102000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009900 tx=0x7fdc9002a410 comp rx=0 tx=0).stop 2024-09-12T12:11:41.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 -- 172.21.15.137:0/3134150501 shutdown_connections 2024-09-12T12:11:41.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/3134150501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc9c1037c0 0x7fdc9c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/3134150501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c102e00 0x7fdc9c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/3134150501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c102000 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 -- 172.21.15.137:0/3134150501 >> 172.21.15.137:0/3134150501 conn(0x7fdc9c0fd390 msgr2=0x7fdc9c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:41.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 -- 172.21.15.137:0/3134150501 shutdown_connections 2024-09-12T12:11:41.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.354+0000 7fdca2d3c640 1 -- 172.21.15.137:0/3134150501 wait complete. 2024-09-12T12:11:41.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 Processor -- start 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 -- start start 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c102e00 0x7fdc9c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc9c1037c0 0x7fdc9c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c10d8e0 con 0x7fdc9c101c00 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c10d760 con 0x7fdc9c1037c0 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca2d3c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c10da60 con 0x7fdc9c102e00 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca0ad8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca0ad8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35752/0 (socket says 172.21.15.137:35752) 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca0ad8640 1 -- 172.21.15.137:0/410806107 learned_addr learned my addr 172.21.15.137:0/410806107 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdc9bfff640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c102e00 0x7fdc9c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:41.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.355+0000 7fdca12d9640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc9c1037c0 0x7fdc9c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca0ad8640 1 -- 172.21.15.137:0/410806107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c102e00 msgr2=0x7fdc9c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca0ad8640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c102e00 0x7fdc9c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca0ad8640 1 -- 172.21.15.137:0/410806107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc9c1037c0 msgr2=0x7fdc9c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca0ad8640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc9c1037c0 0x7fdc9c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca0ad8640 1 -- 172.21.15.137:0/410806107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c19fb90 con 0x7fdc9c101c00 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca12d9640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc9c1037c0 0x7fdc9c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca0ad8640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c197f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc90031610 tx=0x7fdc900043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdc99ffb640 1 -- 172.21.15.137:0/410806107 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9003f030 con 0x7fdc9c101c00 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdc99ffb640 1 -- 172.21.15.137:0/410806107 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc90003ce0 con 0x7fdc9c101c00 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9c19fe20 con 0x7fdc9c101c00 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdc99ffb640 1 -- 172.21.15.137:0/410806107 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90035070 con 0x7fdc9c101c00 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.356+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9c1a0290 con 0x7fdc9c101c00 2024-09-12T12:11:41.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.357+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9c109020 con 0x7fdc9c101c00 2024-09-12T12:11:41.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.358+0000 7fdc99ffb640 1 -- 172.21.15.137:0/410806107 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdc90042030 con 0x7fdc9c101c00 2024-09-12T12:11:41.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.359+0000 7fdc99ffb640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc8c0a2c40 0x7fdc8c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:41.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.359+0000 7fdc9bfff640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc8c0a2c40 0x7fdc8c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:41.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.359+0000 7fdc99ffb640 1 -- 172.21.15.137:0/410806107 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fdc900ef570 con 0x7fdc9c101c00 2024-09-12T12:11:41.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.359+0000 7fdc9bfff640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc8c0a2c40 0x7fdc8c0a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c199480 tx=0x7fdc8800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:41.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.363+0000 7fdc99ffb640 1 -- 172.21.15.137:0/410806107 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc900bc520 con 0x7fdc9c101c00 2024-09-12T12:11:41.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:41 smithi171 ceph-mon[35098]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:11:41.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:41 smithi189 ceph-mon[35475]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:11:41.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.598+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fdc9c062df0 con 0x7fdc9c101c00 2024-09-12T12:11:41.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.600+0000 7fdc99ffb640 1 -- 172.21.15.137:0/410806107 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fdc900bbe20 con 0x7fdc9c101c00 2024-09-12T12:11:41.601 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2024-09-12T12:11:41.601 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQB92uJmd3jHIxAAFQSo6986QXaIXb6/UTBLnQ== 2024-09-12T12:11:41.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc8c0a2c40 msgr2=0x7fdc8c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:41.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc8c0a2c40 0x7fdc8c0a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c199480 tx=0x7fdc8800b390 comp rx=0 tx=0).stop 2024-09-12T12:11:41.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 msgr2=0x7fdc9c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:41.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c197f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc90031610 tx=0x7fdc900043b0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 shutdown_connections 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fdc8c0a2c40 0x7fdc8c0a5100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdc9c1037c0 0x7fdc9c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c102e00 0x7fdc9c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 --2- 172.21.15.137:0/410806107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc9c101c00 0x7fdc9c197f60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 >> 172.21.15.137:0/410806107 conn(0x7fdc9c0fd390 msgr2=0x7fdc9c1061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 shutdown_connections 2024-09-12T12:11:41.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:41.603+0000 7fdca2d3c640 1 -- 172.21.15.137:0/410806107 wait complete. 2024-09-12T12:11:42.331 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:11:42.331 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T12:11:42.331 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T12:11:42.367 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T12:11:42.368 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T12:11:42.368 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T12:11:42.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:42 smithi171 ceph-mon[35098]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T12:11:42.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:42 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/410806107' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T12:11:42.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:42 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/410806107' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T12:11:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:42 smithi137 ceph-mon[30950]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T12:11:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:42 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/410806107' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T12:11:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:42 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/410806107' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T12:11:42.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:42 smithi189 ceph-mon[35475]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-09-12T12:11:42.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:42 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/410806107' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T12:11:42.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:42 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/410806107' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T12:11:42.656 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 -- 172.21.15.137:0/2652659049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102ef0 msgr2=0x7fa16410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 --2- 172.21.15.137:0/2652659049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102ef0 0x7fa16410da40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009ca0 tx=0x7fa15c02a950 comp rx=0 tx=0).stop 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 -- 172.21.15.137:0/2652659049 shutdown_connections 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 --2- 172.21.15.137:0/2652659049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102ef0 0x7fa16410da40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 --2- 172.21.15.137:0/2652659049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa164102570 0x7fa1641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 --2- 172.21.15.137:0/2652659049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa164100e20 0x7fa164101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 -- 172.21.15.137:0/2652659049 >> 172.21.15.137:0/2652659049 conn(0x7fa1640fc7d0 msgr2=0x7fa1640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 -- 172.21.15.137:0/2652659049 shutdown_connections 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.155+0000 7fa169e67640 1 -- 172.21.15.137:0/2652659049 wait complete. 2024-09-12T12:11:43.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 Processor -- start 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 -- start start 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa164100e20 0x7fa1641984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102570 0x7fa164198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa164102ef0 0x7fa16419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa164110fa0 con 0x7fa164102570 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa164110e20 con 0x7fa164102ef0 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa169e67640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa164111120 con 0x7fa164100e20 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102570 0x7fa164198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102570 0x7fa164198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35764/0 (socket says 172.21.15.137:35764) 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 -- 172.21.15.137:0/2986286401 learned_addr learned my addr 172.21.15.137:0/2986286401 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa1637fe640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa164100e20 0x7fa1641984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:43.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa163fff640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa164102ef0 0x7fa16419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 -- 172.21.15.137:0/2986286401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa164100e20 msgr2=0x7fa1641984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa164100e20 0x7fa1641984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 -- 172.21.15.137:0/2986286401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa164102ef0 msgr2=0x7fa16419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa164102ef0 0x7fa16419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa15bfff640 1 -- 172.21.15.137:0/2986286401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16419d480 con 0x7fa164102570 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa1637fe640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa164100e20 0x7fa1641984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.156+0000 7fa163fff640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa164102ef0 0x7fa16419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.157+0000 7fa15bfff640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102570 0x7fa164198a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00b4a0 tx=0x7fa14c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.157+0000 7fa1617fa640 1 -- 172.21.15.137:0/2986286401 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c014070 con 0x7fa164102570 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.157+0000 7fa1617fa640 1 -- 172.21.15.137:0/2986286401 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa14c010af0 con 0x7fa164102570 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.157+0000 7fa1617fa640 1 -- 172.21.15.137:0/2986286401 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c017d10 con 0x7fa164102570 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.157+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16419d770 con 0x7fa164102570 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.157+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1641a5050 con 0x7fa164102570 2024-09-12T12:11:43.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.158+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa164101200 con 0x7fa164102570 2024-09-12T12:11:43.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.159+0000 7fa1617fa640 1 -- 172.21.15.137:0/2986286401 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa14c017070 con 0x7fa164102570 2024-09-12T12:11:43.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.159+0000 7fa1617fa640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa1500a2c10 0x7fa1500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:43.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.160+0000 7fa1637fe640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa1500a2c10 0x7fa1500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:43.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.160+0000 7fa1617fa640 1 -- 172.21.15.137:0/2986286401 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fa14c0ce720 con 0x7fa164102570 2024-09-12T12:11:43.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.160+0000 7fa1637fe640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa1500a2c10 0x7fa1500a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa15400abe0 tx=0x7fa15400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:43.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.163+0000 7fa1617fa640 1 -- 172.21.15.137:0/2986286401 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa14c09b7e0 con 0x7fa164102570 2024-09-12T12:11:43.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.376+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa164199830 con 0x7fa164102570 2024-09-12T12:11:43.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.380+0000 7fa1617fa640 1 -- 172.21.15.137:0/2986286401 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263209 (secure 0 0 0) 0x7fa164199830 con 0x7fa164102570 2024-09-12T12:11:43.384 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa1500a2c10 msgr2=0x7fa1500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa1500a2c10 0x7fa1500a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa15400abe0 tx=0x7fa15400a5c0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102570 msgr2=0x7fa164198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102570 0x7fa164198a10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa14c00b4a0 tx=0x7fa14c00b960 comp rx=0 tx=0).stop 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 shutdown_connections 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa1500a2c10 0x7fa1500a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa164102ef0 0x7fa16419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa164102570 0x7fa164198a10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 --2- 172.21.15.137:0/2986286401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa164100e20 0x7fa1641984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:43.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 >> 172.21.15.137:0/2986286401 conn(0x7fa1640fc7d0 msgr2=0x7fa16406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:43.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 shutdown_connections 2024-09-12T12:11:43.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:43.384+0000 7fa169e67640 1 -- 172.21.15.137:0/2986286401 wait complete. 2024-09-12T12:11:44.129 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1789953576},{"type":"v1","addr":"172.21.15.137:6801","nonce":1789953576}]},"active_addr":"172.21.15.137:6801/1789953576","active_change":"2024-09-12T12:04:15.039108+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.137:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":2039555152}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":4099842792}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":123979974}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":2550482799}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":3293868226}]}]} 2024-09-12T12:11:44.135 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T12:11:44.135 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T12:11:44.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:11:44.390 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:44.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:44 smithi171 ceph-mon[35098]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-12T12:11:44.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:44 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2986286401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T12:11:44.511 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:44 smithi137 ceph-mon[30950]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-12T12:11:44.511 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:44 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2986286401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T12:11:44.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:44 smithi189 ceph-mon[35475]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2024-09-12T12:11:44.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:44 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2986286401' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T12:11:44.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 -- 172.21.15.137:0/1802906051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4106f10 msgr2=0x7fcbc410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:44.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 --2- 172.21.15.137:0/1802906051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4106f10 0x7fcbc410ea90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009ca0 tx=0x7fcbbc02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:11:44.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 -- 172.21.15.137:0/1802906051 shutdown_connections 2024-09-12T12:11:44.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 --2- 172.21.15.137:0/1802906051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4106f10 0x7fcbc410ea90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:44.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 --2- 172.21.15.137:0/1802906051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbc4102790 0x7fcbc41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 --2- 172.21.15.137:0/1802906051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc4101de0 0x7fcbc41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 -- 172.21.15.137:0/1802906051 >> 172.21.15.137:0/1802906051 conn(0x7fcbc40fd8c0 msgr2=0x7fcbc40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 -- 172.21.15.137:0/1802906051 shutdown_connections 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.850+0000 7fcbcab72640 1 -- 172.21.15.137:0/1802906051 wait complete. 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 Processor -- start 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 -- start start 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbc4101de0 0x7fcbc41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4102790 0x7fcbc4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc4106f10 0x7fcbc419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111f70 con 0x7fcbc4102790 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc4111df0 con 0x7fcbc4101de0 2024-09-12T12:11:44.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbcab72640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc41120f0 con 0x7fcbc4106f10 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4102790 0x7fcbc4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4102790 0x7fcbc4198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35784/0 (socket says 172.21.15.137:35784) 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 -- 172.21.15.137:0/566920128 learned_addr learned my addr 172.21.15.137:0/566920128 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc910f640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc4106f10 0x7fcbc419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc890e640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbc4101de0 0x7fcbc41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 -- 172.21.15.137:0/566920128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc4106f10 msgr2=0x7fcbc419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc4106f10 0x7fcbc419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 -- 172.21.15.137:0/566920128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbc4101de0 msgr2=0x7fcbc41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbc4101de0 0x7fcbc41983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.851+0000 7fcbc3fff640 1 -- 172.21.15.137:0/566920128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc419d3a0 con 0x7fcbc4102790 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.852+0000 7fcbc3fff640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4102790 0x7fcbc4198930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000d950 tx=0x7fcbb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.852+0000 7fcbc1ffb640 1 -- 172.21.15.137:0/566920128 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0017070 con 0x7fcbc4102790 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.852+0000 7fcbc1ffb640 1 -- 172.21.15.137:0/566920128 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb0009c30 con 0x7fcbc4102790 2024-09-12T12:11:44.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.852+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc419d690 con 0x7fcbc4102790 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.852+0000 7fcbc1ffb640 1 -- 172.21.15.137:0/566920128 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000f6b0 con 0x7fcbc4102790 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.852+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc41a4f70 con 0x7fcbc4102790 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.853+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc41021c0 con 0x7fcbc4102790 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.854+0000 7fcbc1ffb640 1 -- 172.21.15.137:0/566920128 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcbb00109d0 con 0x7fcbc4102790 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.855+0000 7fcbc1ffb640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.855+0000 7fcbc890e640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.855+0000 7fcbc1ffb640 1 -- 172.21.15.137:0/566920128 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fcbb00cec60 con 0x7fcbc4102790 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.855+0000 7fcbc890e640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005fd0 tx=0x7fcbb8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:44.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:44.859+0000 7fcbc1ffb640 1 -- 172.21.15.137:0/566920128 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbb0009de0 con 0x7fcbc4102790 2024-09-12T12:11:45.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.025+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbc410a120 con 0x7fcbc4102790 2024-09-12T12:11:45.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.026+0000 7fcbc1ffb640 1 -- 172.21.15.137:0/566920128 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22348 (secure 0 0 0) 0x7fcbb009bc10 con 0x7fcbc4102790 2024-09-12T12:11:45.027 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:45.027 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:11:13.196473+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcbb40a2c10 msgr2=0x7fcbb40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005fd0 tx=0x7fcbb8005850 comp rx=0 tx=0).stop 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4102790 msgr2=0x7fcbc4198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4102790 0x7fcbc4198930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000d950 tx=0x7fcbb000de10 comp rx=0 tx=0).stop 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 shutdown_connections 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcbc4106f10 0x7fcbc419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc4102790 0x7fcbc4198930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:45.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 --2- 172.21.15.137:0/566920128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbc4101de0 0x7fcbc41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:45.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 >> 172.21.15.137:0/566920128 conn(0x7fcbc40fd8c0 msgr2=0x7fcbc40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:45.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 shutdown_connections 2024-09-12T12:11:45.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:45.028+0000 7fcbcab72640 1 -- 172.21.15.137:0/566920128 wait complete. 2024-09-12T12:11:45.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:45 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/566920128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:45.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:45 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/566920128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:45.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:45 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/566920128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:45.829 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T12:11:45.829 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:11:46.075 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:46.490 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:46 smithi137 ceph-mon[30950]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T12:11:46.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:46 smithi189 ceph-mon[35475]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T12:11:46.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.568+0000 7fc187e6a640 1 -- 172.21.15.137:0/2805702069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180102570 msgr2=0x7fc1801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:46.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.568+0000 7fc187e6a640 1 --2- 172.21.15.137:0/2805702069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180102570 0x7fc1801029b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009900 tx=0x7fc17c02a480 comp rx=0 tx=0).stop 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 -- 172.21.15.137:0/2805702069 shutdown_connections 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 --2- 172.21.15.137:0/2805702069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc180102ef0 0x7fc18010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 --2- 172.21.15.137:0/2805702069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180102570 0x7fc1801029b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 --2- 172.21.15.137:0/2805702069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc180100e20 0x7fc180101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 -- 172.21.15.137:0/2805702069 >> 172.21.15.137:0/2805702069 conn(0x7fc1800fc7d0 msgr2=0x7fc1800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 -- 172.21.15.137:0/2805702069 shutdown_connections 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 -- 172.21.15.137:0/2805702069 wait complete. 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 Processor -- start 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.569+0000 7fc187e6a640 1 -- start start 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180100e20 0x7fc180198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:46.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc180102570 0x7fc1801989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc180102ef0 0x7fc18019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1801120a0 con 0x7fc180100e20 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc180111f20 con 0x7fc180102ef0 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc180112220 con 0x7fc180102570 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180100e20 0x7fc180198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180100e20 0x7fc180198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35806/0 (socket says 172.21.15.137:35806) 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 -- 172.21.15.137:0/1065015569 learned_addr learned my addr 172.21.15.137:0/1065015569 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc186407640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc180102ef0 0x7fc18019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 -- 172.21.15.137:0/1065015569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc180102570 msgr2=0x7fc1801989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc180102570 0x7fc1801989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 -- 172.21.15.137:0/1065015569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc180102ef0 msgr2=0x7fc18019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc180102ef0 0x7fc18019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 -- 172.21.15.137:0/1065015569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18019d420 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc185c06640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180100e20 0x7fc180198470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c890 tx=0x7fc17400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc172ffd640 1 -- 172.21.15.137:0/1065015569 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174004220 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc172ffd640 1 -- 172.21.15.137:0/1065015569 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1740043c0 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18019d710 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc172ffd640 1 -- 172.21.15.137:0/1065015569 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740049e0 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.570+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1801a4ff0 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.571+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc180101200 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.572+0000 7fc172ffd640 1 -- 172.21.15.137:0/1065015569 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc1740079b0 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.573+0000 7fc172ffd640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.573+0000 7fc185405640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.573+0000 7fc172ffd640 1 -- 172.21.15.137:0/1065015569 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fc1740cf130 con 0x7fc180100e20 2024-09-12T12:11:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.573+0000 7fc185405640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0096c0 tx=0x7fc17c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:46.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.577+0000 7fc172ffd640 1 -- 172.21.15.137:0/1065015569 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc174014030 con 0x7fc180100e20 2024-09-12T12:11:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:46 smithi171 ceph-mon[35098]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-09-12T12:11:46.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.746+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1801997d0 con 0x7fc180100e20 2024-09-12T12:11:46.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.748+0000 7fc172ffd640 1 -- 172.21.15.137:0/1065015569 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22348 (secure 0 0 0) 0x7fc17409c0e0 con 0x7fc180100e20 2024-09-12T12:11:46.749 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:11:46.749 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:11:13.196473+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:11:46.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc16c0a2c10 msgr2=0x7fc16c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:46.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0096c0 tx=0x7fc17c000bc0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180100e20 msgr2=0x7fc180198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:46.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180100e20 0x7fc180198470 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c890 tx=0x7fc17400cd50 comp rx=0 tx=0).stop 2024-09-12T12:11:46.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 shutdown_connections 2024-09-12T12:11:46.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fc16c0a2c10 0x7fc16c0a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc180102ef0 0x7fc18019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc180102570 0x7fc1801989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 --2- 172.21.15.137:0/1065015569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc180100e20 0x7fc180198470 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:46.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 >> 172.21.15.137:0/1065015569 conn(0x7fc1800fc7d0 msgr2=0x7fc180106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:46.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 shutdown_connections 2024-09-12T12:11:46.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:46.750+0000 7fc187e6a640 1 -- 172.21.15.137:0/1065015569 wait complete. 2024-09-12T12:11:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:47 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1065015569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:47.582 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T12:11:47.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T12:11:47.583 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T12:11:47.584 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T12:11:47.585 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T12:11:47.585 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T12:11:47.586 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T12:11:47.586 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T12:11:47.587 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T12:11:47.588 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T12:11:47.588 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T12:11:47.589 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T12:11:47.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:47 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1065015569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:47 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1065015569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:11:48.207 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:48 smithi137 ceph-mon[30950]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T12:11:48.316 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:48 smithi189 ceph-mon[35475]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T12:11:48.599 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.601 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.601 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.603 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.603 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.612 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.612 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.615 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.618 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.622 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.632 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:11:48.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:48 smithi171 ceph-mon[35098]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T12:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 -- 172.21.15.137:0/2791190965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 msgr2=0x7ff978101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 --2- 172.21.15.137:0/2791190965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 0x7ff978101200 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff974009900 tx=0x7ff97402a480 comp rx=0 tx=0).stop 2024-09-12T12:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 -- 172.21.15.137:0/2791190965 shutdown_connections 2024-09-12T12:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 --2- 172.21.15.137:0/2791190965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff978102ef0 0x7ff97810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:48.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 --2- 172.21.15.137:0/2791190965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff978102570 0x7ff9781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 --2- 172.21.15.137:0/2791190965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 0x7ff978101200 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 -- 172.21.15.137:0/2791190965 >> 172.21.15.137:0/2791190965 conn(0x7ff9780fc7d0 msgr2=0x7ff9780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 -- 172.21.15.137:0/2791190965 shutdown_connections 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.808+0000 7ff97f8c1640 1 -- 172.21.15.137:0/2791190965 wait complete. 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 Processor -- start 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 -- start start 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 0x7ff9781984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff978102570 0x7ff978198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff978102ef0 0x7ff97819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff978110f80 con 0x7ff978100e20 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff978110e00 con 0x7ff978102ef0 2024-09-12T12:11:48.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97f8c1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff978111100 con 0x7ff978102570 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97d65d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 0x7ff9781984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97d65d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 0x7ff9781984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35830/0 (socket says 172.21.15.137:35830) 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97d65d640 1 -- 172.21.15.137:0/846687140 learned_addr learned my addr 172.21.15.137:0/846687140 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97de5e640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff978102ef0 0x7ff97819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.809+0000 7ff97ce5c640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff978102570 0x7ff978198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97d65d640 1 -- 172.21.15.137:0/846687140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff978102570 msgr2=0x7ff978198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97d65d640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff978102570 0x7ff978198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97d65d640 1 -- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff978102ef0 msgr2=0x7ff97819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97d65d640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff978102ef0 0x7ff97819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97d65d640 1 -- 172.21.15.137:0/846687140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97819d4a0 con 0x7ff978100e20 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97ce5c640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff978102570 0x7ff978198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:48.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97d65d640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 0x7ff9781984f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff974006010 tx=0x7ff974003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97403f070 con 0x7ff978100e20 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97819d730 con 0x7ff978100e20 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff974031c30 con 0x7ff978100e20 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974035030 con 0x7ff978100e20 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.810+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781a5070 con 0x7ff978100e20 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.811+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff978106bf0 con 0x7ff978100e20 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.812+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff974042070 con 0x7ff978100e20 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff96e7fc640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff9640a2c90 0x7ff9640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:48.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff97ce5c640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff9640a2c90 0x7ff9640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7ff9740f0150 con 0x7ff978100e20 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff96e7fc640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] conn(0x7ff9640ac4d0 0x7ff9640ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 --> [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff974037ec0 con 0x7ff9640ac4d0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7ff9740f0630 con 0x7ff978100e20 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff97ce5c640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff9640a2c90 0x7ff9640a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9680058c0 tx=0x7ff968005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.813+0000 7ff97de5e640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] conn(0x7ff9640ac4d0 0x7ff9640ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.814+0000 7ff97de5e640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] conn(0x7ff9640ac4d0 0x7ff9640ae8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.815+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== osd.10 v2:172.21.15.171:6824/124645600 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff974037ec0 con 0x7ff9640ac4d0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.828+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 --> [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9781021b0 con 0x7ff9640ac4d0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff96e7fc640 1 -- 172.21.15.137:0/846687140 <== osd.10 v2:172.21.15.171:6824/124645600 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9781021b0 con 0x7ff9640ac4d0 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] conn(0x7ff9640ac4d0 msgr2=0x7ff9640ae8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] conn(0x7ff9640ac4d0 0x7ff9640ae8f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:48.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff9640a2c90 msgr2=0x7ff9640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff9640a2c90 0x7ff9640a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9680058c0 tx=0x7ff968005850 comp rx=0 tx=0).stop 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 msgr2=0x7ff9781984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 --2- 172.21.15.137:0/846687140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff978100e20 0x7ff9781984f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff974006010 tx=0x7ff974003c10 comp rx=0 tx=0).stop 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97de5e640 1 -- 172.21.15.137:0/846687140 reap_dead start 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 shutdown_connections 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 >> 172.21.15.137:0/846687140 conn(0x7ff9780fc7d0 msgr2=0x7ff97806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 shutdown_connections 2024-09-12T12:11:48.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:48.829+0000 7ff97f8c1640 1 -- 172.21.15.137:0/846687140 wait complete. 2024-09-12T12:11:49.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 -- 172.21.15.137:0/984156379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82441027c0 msgr2=0x7f8244106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:49.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 --2- 172.21.15.137:0/984156379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82441027c0 0x7f8244106810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8234009900 tx=0x7f823402a410 comp rx=0 tx=0).stop 2024-09-12T12:11:49.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 -- 172.21.15.137:0/984156379 shutdown_connections 2024-09-12T12:11:49.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 --2- 172.21.15.137:0/984156379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8244106f40 0x7f824410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:49.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 --2- 172.21.15.137:0/984156379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82441027c0 0x7f8244106810 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:49.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 --2- 172.21.15.137:0/984156379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8244101e10 0x7f82441021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:49.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 -- 172.21.15.137:0/984156379 >> 172.21.15.137:0/984156379 conn(0x7f82440fd8d0 msgr2=0x7f82440ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:49.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 -- 172.21.15.137:0/984156379 shutdown_connections 2024-09-12T12:11:49.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.624+0000 7f824c7d9640 1 -- 172.21.15.137:0/984156379 wait complete. 2024-09-12T12:11:49.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 Processor -- start 2024-09-12T12:11:49.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 -- start start 2024-09-12T12:11:49.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8244101e10 0x7f8244198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:49.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82441027c0 0x7f82441989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:49.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8244106f40 0x7f824419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:49.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8244111fa0 con 0x7f8244101e10 2024-09-12T12:11:49.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8244111e20 con 0x7f8244106f40 2024-09-12T12:11:49.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824c7d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112120 con 0x7f82441027c0 2024-09-12T12:11:49.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824a575640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8244101e10 0x7f8244198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:49.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824a575640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8244101e10 0x7f8244198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51168/0 (socket says 172.21.15.137:51168) 2024-09-12T12:11:49.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f824a575640 1 -- 172.21.15.137:0/142948589 learned_addr learned my addr 172.21.15.137:0/142948589 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:49.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.625+0000 7f8249d74640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82441027c0 0x7f82441989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:49.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824ad76640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8244106f40 0x7f824419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:49.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824a575640 1 -- 172.21.15.137:0/142948589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82441027c0 msgr2=0x7f82441989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:49.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824a575640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82441027c0 0x7f82441989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:49.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824a575640 1 -- 172.21.15.137:0/142948589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8244106f40 msgr2=0x7f824419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:49.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824a575640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8244106f40 0x7f824419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:49.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824a575640 1 -- 172.21.15.137:0/142948589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824419d410 con 0x7f8244101e10 2024-09-12T12:11:49.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824ad76640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8244106f40 0x7f824419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:49.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824a575640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8244101e10 0x7f8244198460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f824000d950 tx=0x7f824000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:49.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240017070 con 0x7f8244101e10 2024-09-12T12:11:49.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824419d700 con 0x7f8244101e10 2024-09-12T12:11:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8240009c30 con 0x7f8244101e10 2024-09-12T12:11:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000f6b0 con 0x7f8244101e10 2024-09-12T12:11:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.626+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a4fe0 con 0x7f8244101e10 2024-09-12T12:11:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.627+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8244062980 con 0x7f8244101e10 2024-09-12T12:11:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.628+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f82400109d0 con 0x7f8244101e10 2024-09-12T12:11:49.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.629+0000 7f823b7fe640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f82300a2c10 0x7f82300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:49.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.629+0000 7f8249d74640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f82300a2c10 0x7f82300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:49.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.629+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f82400cf130 con 0x7f8244101e10 2024-09-12T12:11:49.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.629+0000 7f823b7fe640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] conn(0x7f82300ac460 0x7f82300ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:49.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.629+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 --> [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8240016ec0 con 0x7f82300ac460 2024-09-12T12:11:49.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.629+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f82400cf4e0 con 0x7f8244101e10 2024-09-12T12:11:49.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.630+0000 7f824ad76640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] conn(0x7f82300ac460 0x7f82300ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:49.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.630+0000 7f8249d74640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f82300a2c10 0x7f82300a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8234009230 tx=0x7f82340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:49.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.630+0000 7f824ad76640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] conn(0x7f82300ac460 0x7f82300ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:49.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.630+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== osd.3 v2:172.21.15.137:6810/1732422087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8240016ec0 con 0x7f82300ac460 2024-09-12T12:11:49.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 --> [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8244062e00 con 0x7f82300ac460 2024-09-12T12:11:49.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f823b7fe640 1 -- 172.21.15.137:0/142948589 <== osd.3 v2:172.21.15.137:6810/1732422087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8244062e00 con 0x7f82300ac460 2024-09-12T12:11:49.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] conn(0x7f82300ac460 msgr2=0x7f82300ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:49.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] conn(0x7f82300ac460 0x7f82300ae880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:49.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f82300a2c10 msgr2=0x7f82300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:49.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f82300a2c10 0x7f82300a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8234009230 tx=0x7f82340058d0 comp rx=0 tx=0).stop 2024-09-12T12:11:49.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8244101e10 msgr2=0x7f8244198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:49.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 --2- 172.21.15.137:0/142948589 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8244101e10 0x7f8244198460 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f824000d950 tx=0x7f824000de10 comp rx=0 tx=0).stop 2024-09-12T12:11:49.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824ad76640 1 -- 172.21.15.137:0/142948589 reap_dead start 2024-09-12T12:11:49.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 shutdown_connections 2024-09-12T12:11:49.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.644+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 >> 172.21.15.137:0/142948589 conn(0x7f82440fd8d0 msgr2=0x7f82440fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:49.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.645+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 shutdown_connections 2024-09-12T12:11:49.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:49.645+0000 7f824c7d9640 1 -- 172.21.15.137:0/142948589 wait complete. 2024-09-12T12:11:50.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:50 smithi137 ceph-mon[30950]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T12:11:50.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:50 smithi189 ceph-mon[35475]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T12:11:50.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:50 smithi171 ceph-mon[35098]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-12T12:11:50.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 -- 172.21.15.137:0/1491455443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 msgr2=0x7f196410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:50.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 --2- 172.21.15.137:0/1491455443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 0x7f196410d950 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f195c009cd0 tx=0x7f195c02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:11:50.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 -- 172.21.15.137:0/1491455443 shutdown_connections 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 --2- 172.21.15.137:0/1491455443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 0x7f196410d950 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 --2- 172.21.15.137:0/1491455443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1964100a80 0x7f1964100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 --2- 172.21.15.137:0/1491455443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1964106a80 0x7f1964106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 -- 172.21.15.137:0/1491455443 >> 172.21.15.137:0/1491455443 conn(0x7f19640fc7a0 msgr2=0x7f19640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 -- 172.21.15.137:0/1491455443 shutdown_connections 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 -- 172.21.15.137:0/1491455443 wait complete. 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 Processor -- start 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.857+0000 7f196bae0640 1 -- start start 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1964100a80 0x7f196410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 0x7f196410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1964106a80 0x7f196410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:50.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1964111fa0 con 0x7f1964101420 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1964111e20 con 0x7f1964100a80 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1964112120 con 0x7f1964106a80 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 0x7f196410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 0x7f196410e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51180/0 (socket says 172.21.15.137:51180) 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 -- 172.21.15.137:0/2200995826 learned_addr learned my addr 172.21.15.137:0/2200995826 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196a07d640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1964106a80 0x7f196410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196987c640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1964100a80 0x7f196410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 -- 172.21.15.137:0/2200995826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1964106a80 msgr2=0x7f196410ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1964106a80 0x7f196410ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 -- 172.21.15.137:0/2200995826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1964100a80 msgr2=0x7f196410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1964100a80 0x7f196410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 -- 172.21.15.137:0/2200995826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19641a6950 con 0x7f1964101420 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196907b640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 0x7f196410e920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f195800d950 tx=0x7f195800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:50.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958017070 con 0x7f1964101420 2024-09-12T12:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19641a6c40 con 0x7f1964101420 2024-09-12T12:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1958009c30 con 0x7f1964101420 2024-09-12T12:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195800f6b0 con 0x7f1964101420 2024-09-12T12:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.858+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19641a7180 con 0x7f1964101420 2024-09-12T12:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.859+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19640628c0 con 0x7f1964101420 2024-09-12T12:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f19580109d0 con 0x7f1964101420 2024-09-12T12:11:50.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f1956ffd640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f19500a2c10 0x7f19500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f196987c640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f19500a2c10 0x7f19500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f19580cf050 con 0x7f1964101420 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f1956ffd640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] conn(0x7f19500ac460 0x7f19500ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 --> [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1958001d30 con 0x7f19500ac460 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f19580cf530 con 0x7f1964101420 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.861+0000 7f196a07d640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] conn(0x7f19500ac460 0x7f19500ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.862+0000 7f196987c640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f19500a2c10 0x7f19500a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1960005d90 tx=0x7f1960005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.862+0000 7f196a07d640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] conn(0x7f19500ac460 0x7f19500ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.862+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== osd.6 v2:172.21.15.137:6818/2955523379 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1958001d30 con 0x7f19500ac460 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 --> [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1964062cc0 con 0x7f19500ac460 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f1956ffd640 1 -- 172.21.15.137:0/2200995826 <== osd.6 v2:172.21.15.137:6818/2955523379 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1964062cc0 con 0x7f19500ac460 2024-09-12T12:11:50.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] conn(0x7f19500ac460 msgr2=0x7f19500ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] conn(0x7f19500ac460 0x7f19500ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f19500a2c10 msgr2=0x7f19500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f19500a2c10 0x7f19500a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1960005d90 tx=0x7f1960005d20 comp rx=0 tx=0).stop 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 msgr2=0x7f196410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 --2- 172.21.15.137:0/2200995826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1964101420 0x7f196410e920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f195800d950 tx=0x7f195800de10 comp rx=0 tx=0).stop 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196a07d640 1 -- 172.21.15.137:0/2200995826 reap_dead start 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 shutdown_connections 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 >> 172.21.15.137:0/2200995826 conn(0x7f19640fc7a0 msgr2=0x7f19640fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 shutdown_connections 2024-09-12T12:11:50.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:50.876+0000 7f196bae0640 1 -- 172.21.15.137:0/2200995826 wait complete. 2024-09-12T12:11:51.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 -- 172.21.15.137:0/3390953158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 msgr2=0x7fd1581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 --2- 172.21.15.137:0/3390953158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 0x7fd1581021c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd148009900 tx=0x7fd14802a3f0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 -- 172.21.15.137:0/3390953158 shutdown_connections 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 --2- 172.21.15.137:0/3390953158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158106f10 0x7fd15810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 --2- 172.21.15.137:0/3390953158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd158102790 0x7fd1581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 --2- 172.21.15.137:0/3390953158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 0x7fd1581021c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 -- 172.21.15.137:0/3390953158 >> 172.21.15.137:0/3390953158 conn(0x7fd1580fd8c0 msgr2=0x7fd1580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 -- 172.21.15.137:0/3390953158 shutdown_connections 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.298+0000 7fd15cb95640 1 -- 172.21.15.137:0/3390953158 wait complete. 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 Processor -- start 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 -- start start 2024-09-12T12:11:51.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 0x7fd1581983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158102790 0x7fd1581988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd158106f10 0x7fd15819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd158111f70 con 0x7fd158101de0 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd158111df0 con 0x7fd158106f10 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15cb95640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1581120f0 con 0x7fd158102790 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.299+0000 7fd15659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 0x7fd1581983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 0x7fd1581983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51190/0 (socket says 172.21.15.137:51190) 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 -- 172.21.15.137:0/4268434653 learned_addr learned my addr 172.21.15.137:0/4268434653 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd156d9d640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd158106f10 0x7fd15819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd155d9b640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158102790 0x7fd1581988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 -- 172.21.15.137:0/4268434653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158102790 msgr2=0x7fd1581988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd158102790 0x7fd1581988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 -- 172.21.15.137:0/4268434653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd158106f10 msgr2=0x7fd15819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd158106f10 0x7fd15819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 -- 172.21.15.137:0/4268434653 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15819d360 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd156d9d640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd158106f10 0x7fd15819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15659c640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 0x7fd1581983b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd148006010 tx=0x7fd1480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14803f030 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd148003ce0 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15819d5f0 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148035070 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.300+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1581a4f30 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.301+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd158062980 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.302+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd148042030 con 0x7fd158101de0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.303+0000 7fd1477fe640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd1400a2c10 0x7fd1400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.303+0000 7fd155d9b640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd1400a2c10 0x7fd1400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:51.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.303+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd1480eff70 con 0x7fd158101de0 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.303+0000 7fd1477fe640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] conn(0x7fd1400ac460 0x7fd1400ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.303+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 --> [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd14803be50 con 0x7fd1400ac460 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.303+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fd1480f0320 con 0x7fd158101de0 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.304+0000 7fd156d9d640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] conn(0x7fd1400ac460 0x7fd1400ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.304+0000 7fd155d9b640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd1400a2c10 0x7fd1400a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005fd0 tx=0x7fd14c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.304+0000 7fd156d9d640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] conn(0x7fd1400ac460 0x7fd1400ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.304+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== osd.0 v2:172.21.15.137:6802/655478089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd14803be50 con 0x7fd1400ac460 2024-09-12T12:11:51.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.318+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 --> [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd158062e00 con 0x7fd1400ac460 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.318+0000 7fd1477fe640 1 -- 172.21.15.137:0/4268434653 <== osd.0 v2:172.21.15.137:6802/655478089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd158062e00 con 0x7fd1400ac460 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] conn(0x7fd1400ac460 msgr2=0x7fd1400ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] conn(0x7fd1400ac460 0x7fd1400ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd1400a2c10 msgr2=0x7fd1400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd1400a2c10 0x7fd1400a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005fd0 tx=0x7fd14c005850 comp rx=0 tx=0).stop 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 msgr2=0x7fd1581983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 --2- 172.21.15.137:0/4268434653 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd158101de0 0x7fd1581983b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd148006010 tx=0x7fd1480042f0 comp rx=0 tx=0).stop 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd156d9d640 1 -- 172.21.15.137:0/4268434653 reap_dead start 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 shutdown_connections 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 >> 172.21.15.137:0/4268434653 conn(0x7fd1580fd8c0 msgr2=0x7fd1580fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 shutdown_connections 2024-09-12T12:11:51.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:51.319+0000 7fd15cb95640 1 -- 172.21.15.137:0/4268434653 wait complete. 2024-09-12T12:11:52.222 INFO:teuthology.orchestra.run.smithi137.stdout:34359738446 2024-09-12T12:11:52.223 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T12:11:52.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:52 smithi137 ceph-mon[30950]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:11:52.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:52 smithi189 ceph-mon[35475]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:11:52.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:52 smithi171 ceph-mon[35098]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:11:52.774 INFO:teuthology.orchestra.run.smithi137.stdout:107374182462 2024-09-12T12:11:52.774 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T12:11:53.489 INFO:teuthology.orchestra.run.smithi137.stdout:292057776154 2024-09-12T12:11:53.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T12:11:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1672405366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8100e20 msgr2=0x7fd5c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 --2- 172.21.15.137:0/1672405366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8100e20 0x7fd5c8101200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009900 tx=0x7fd5b802a480 comp rx=0 tx=0).stop 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1672405366 shutdown_connections 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 --2- 172.21.15.137:0/1672405366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8102ef0 0x7fd5c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 --2- 172.21.15.137:0/1672405366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c8102570 0x7fd5c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 --2- 172.21.15.137:0/1672405366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8100e20 0x7fd5c8101200 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1672405366 >> 172.21.15.137:0/1672405366 conn(0x7fd5c80fc7d0 msgr2=0x7fd5c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1672405366 shutdown_connections 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.644+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1672405366 wait complete. 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 Processor -- start 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 -- start start 2024-09-12T12:11:53.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8102570 0x7fd5c81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8110f80 con 0x7fd5c8102570 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8110e00 con 0x7fd5c8102ef0 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cfce3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8111100 con 0x7fd5c8100e20 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cd27e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8102570 0x7fd5c81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cd27e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8102570 0x7fd5c81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51208/0 (socket says 172.21.15.137:51208) 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cd27e640 1 -- 172.21.15.137:0/1031875210 learned_addr learned my addr 172.21.15.137:0/1031875210 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5cda7f640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.645+0000 7fd5ce280640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cd27e640 1 -- 172.21.15.137:0/1031875210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8100e20 msgr2=0x7fd5c8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cd27e640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8100e20 0x7fd5c8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cd27e640 1 -- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c8102ef0 msgr2=0x7fd5c819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cd27e640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5c8102ef0 0x7fd5c819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cd27e640 1 -- 172.21.15.137:0/1031875210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c819d440 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cd27e640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8102570 0x7fd5c81989d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400c890 tx=0x7fd5c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4004220 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c40043c0 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c819d730 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c40049e0 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.646+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c81a5010 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.647+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5c8106bf0 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.648+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd5c40079b0 con 0x7fd5c8102570 2024-09-12T12:11:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.649+0000 7fd5beffd640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd5b40a2c90 0x7fd5b40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:53.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.649+0000 7fd5cda7f640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd5b40a2c90 0x7fd5b40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:53.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.649+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd5c40cf2b0 con 0x7fd5c8102570 2024-09-12T12:11:53.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.650+0000 7fd5beffd640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] conn(0x7fd5b40ac4d0 0x7fd5b40ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:53.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.650+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 --> [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5b40aefa0 con 0x7fd5b40ac4d0 2024-09-12T12:11:53.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.650+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fd5c40cf660 con 0x7fd5c8102570 2024-09-12T12:11:53.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.650+0000 7fd5cda7f640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd5b40a2c90 0x7fd5b40a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8000c00 tx=0x7fd5b80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.650+0000 7fd5ce280640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] conn(0x7fd5b40ac4d0 0x7fd5b40ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.650+0000 7fd5ce280640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] conn(0x7fd5b40ac4d0 0x7fd5b40ae8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.651+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== osd.1 v2:172.21.15.171:6800/1953204063 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5b40aefa0 con 0x7fd5b40ac4d0 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.665+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 --> [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5c81021b0 con 0x7fd5b40ac4d0 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.665+0000 7fd5beffd640 1 -- 172.21.15.137:0/1031875210 <== osd.1 v2:172.21.15.171:6800/1953204063 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5c81021b0 con 0x7fd5b40ac4d0 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] conn(0x7fd5b40ac4d0 msgr2=0x7fd5b40ae8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] conn(0x7fd5b40ac4d0 0x7fd5b40ae8f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd5b40a2c90 msgr2=0x7fd5b40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd5b40a2c90 0x7fd5b40a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8000c00 tx=0x7fd5b80058d0 comp rx=0 tx=0).stop 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8102570 msgr2=0x7fd5c81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 --2- 172.21.15.137:0/1031875210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5c8102570 0x7fd5c81989d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400c890 tx=0x7fd5c400cd50 comp rx=0 tx=0).stop 2024-09-12T12:11:53.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5ce280640 1 -- 172.21.15.137:0/1031875210 reap_dead start 2024-09-12T12:11:53.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 shutdown_connections 2024-09-12T12:11:53.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 >> 172.21.15.137:0/1031875210 conn(0x7fd5c80fc7d0 msgr2=0x7fd5c806a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:53.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 shutdown_connections 2024-09-12T12:11:53.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:53.666+0000 7fd5cfce3640 1 -- 172.21.15.137:0/1031875210 wait complete. 2024-09-12T12:11:54.441 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:54 smithi137 ceph-mon[30950]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T12:11:54.502 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:54 smithi171 ceph-mon[35098]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T12:11:54.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:54 smithi189 ceph-mon[35475]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T12:11:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.829+0000 7f83dcd4b640 1 -- 172.21.15.137:0/733036702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d8101df0 msgr2=0x7f83d81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.829+0000 7f83dcd4b640 1 --2- 172.21.15.137:0/733036702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d8101df0 0x7f83d81021d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009ca0 tx=0x7f83c802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 -- 172.21.15.137:0/733036702 shutdown_connections 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 --2- 172.21.15.137:0/733036702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83d8106f20 0x7f83d810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 --2- 172.21.15.137:0/733036702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83d81027a0 0x7f83d81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 --2- 172.21.15.137:0/733036702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d8101df0 0x7f83d81021d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 -- 172.21.15.137:0/733036702 >> 172.21.15.137:0/733036702 conn(0x7f83d80fd8f0 msgr2=0x7f83d80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 -- 172.21.15.137:0/733036702 shutdown_connections 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 -- 172.21.15.137:0/733036702 wait complete. 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 Processor -- start 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 -- start start 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83d8101df0 0x7f83d8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.830+0000 7f83dcd4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d81027a0 0x7f83d8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:54.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83dcd4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83d8106f20 0x7f83d819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83dcd4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f83d8111f80 con 0x7f83d81027a0 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83dcd4b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f83d8111e00 con 0x7f83d8101df0 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83dcd4b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f83d8112100 con 0x7f83d8106f20 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d81027a0 0x7f83d8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d81027a0 0x7f83d8198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51238/0 (socket says 172.21.15.137:51238) 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 -- 172.21.15.137:0/1898985258 learned_addr learned my addr 172.21.15.137:0/1898985258 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d659c640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83d8101df0 0x7f83d8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d6d9d640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83d8106f20 0x7f83d819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 -- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83d8106f20 msgr2=0x7f83d819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83d8106f20 0x7f83d819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 -- 172.21.15.137:0/1898985258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83d8101df0 msgr2=0x7f83d8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:54.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83d8101df0 0x7f83d8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 -- 172.21.15.137:0/1898985258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d819d3d0 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d659c640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83d8101df0 0x7f83d8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d6d9d640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f83d8106f20 0x7f83d819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83d5d9b640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d81027a0 0x7f83d8198960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f83c000d950 tx=0x7f83c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0017070 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d819d6c0 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c0009c30 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c000f6b0 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.831+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d81a4fa0 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.832+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83d8062980 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.834+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f83c00109d0 con 0x7f83d81027a0 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.834+0000 7f83cf7fe640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83c40a2c10 0x7f83c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:54.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.834+0000 7f83d659c640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83c40a2c10 0x7f83c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.834+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f83c00cf0b0 con 0x7f83d81027a0 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.834+0000 7f83cf7fe640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] conn(0x7f83c40ac460 0x7f83c40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.834+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 --> [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83c40aef30 con 0x7f83c40ac460 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.834+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f83c00cf590 con 0x7f83d81027a0 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.835+0000 7f83d659c640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83c40a2c10 0x7f83c40a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006010 tx=0x7f83c80093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.835+0000 7f83d6d9d640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] conn(0x7f83c40ac460 0x7f83c40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.835+0000 7f83d6d9d640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] conn(0x7f83c40ac460 0x7f83c40ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:54.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.836+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== osd.5 v2:172.21.15.189:6808/3684989511 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83c40aef30 con 0x7f83c40ac460 2024-09-12T12:11:54.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.849+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 --> [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83d8062d80 con 0x7f83c40ac460 2024-09-12T12:11:54.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83cf7fe640 1 -- 172.21.15.137:0/1898985258 <== osd.5 v2:172.21.15.189:6808/3684989511 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f83d8062d80 con 0x7f83c40ac460 2024-09-12T12:11:54.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] conn(0x7f83c40ac460 msgr2=0x7f83c40ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:54.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] conn(0x7f83c40ac460 0x7f83c40ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83c40a2c10 msgr2=0x7f83c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:54.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f83c40a2c10 0x7f83c40a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006010 tx=0x7f83c80093f0 comp rx=0 tx=0).stop 2024-09-12T12:11:54.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d81027a0 msgr2=0x7f83d8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:54.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 --2- 172.21.15.137:0/1898985258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83d81027a0 0x7f83d8198960 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f83c000d950 tx=0x7f83c000de10 comp rx=0 tx=0).stop 2024-09-12T12:11:54.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83d6d9d640 1 -- 172.21.15.137:0/1898985258 reap_dead start 2024-09-12T12:11:54.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 shutdown_connections 2024-09-12T12:11:54.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.850+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 >> 172.21.15.137:0/1898985258 conn(0x7f83d80fd8f0 msgr2=0x7f83d80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:54.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.851+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 shutdown_connections 2024-09-12T12:11:54.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:54.851+0000 7f83dcd4b640 1 -- 172.21.15.137:0/1898985258 wait complete. 2024-09-12T12:11:55.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.030+0000 7f3f5d63c640 1 -- 172.21.15.137:0/2191691396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f58106f20 msgr2=0x7f3f5810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:55.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.030+0000 7f3f5d63c640 1 --2- 172.21.15.137:0/2191691396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f58106f20 0x7f3f5810eaa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3f48009900 tx=0x7f3f4802a410 comp rx=0 tx=0).stop 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 -- 172.21.15.137:0/2191691396 shutdown_connections 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 --2- 172.21.15.137:0/2191691396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f58106f20 0x7f3f5810eaa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 --2- 172.21.15.137:0/2191691396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f581027a0 0x7f3f581067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 --2- 172.21.15.137:0/2191691396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f58101df0 0x7f3f581021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 -- 172.21.15.137:0/2191691396 >> 172.21.15.137:0/2191691396 conn(0x7f3f580fd8b0 msgr2=0x7f3f580ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 -- 172.21.15.137:0/2191691396 shutdown_connections 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 -- 172.21.15.137:0/2191691396 wait complete. 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 Processor -- start 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.031+0000 7f3f5d63c640 1 -- start start 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f58101df0 0x7f3f58198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f581027a0 0x7f3f58198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:55.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f58106f20 0x7f3f5819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58111fa0 con 0x7f3f581027a0 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58111e20 con 0x7f3f58106f20 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58112120 con 0x7f3f58101df0 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f581027a0 0x7f3f58198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f581027a0 0x7f3f58198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51264/0 (socket says 172.21.15.137:51264) 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 -- 172.21.15.137:0/1917767806 learned_addr learned my addr 172.21.15.137:0/1917767806 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 -- 172.21.15.137:0/1917767806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f58101df0 msgr2=0x7f3f58198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3f58101df0 0x7f3f58198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 -- 172.21.15.137:0/1917767806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f58106f20 msgr2=0x7f3f5819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:55.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f58106f20 0x7f3f5819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 -- 172.21.15.137:0/1917767806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5819d3b0 con 0x7f3f581027a0 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f567fc640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f581027a0 0x7f3f58198940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ba50 tx=0x7f3f4000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40019070 con 0x7f3f581027a0 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5819d6a0 con 0x7f3f581027a0 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4000cbe0 con 0x7f3f581027a0 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400076e0 con 0x7f3f581027a0 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.032+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f581a4f80 con 0x7f3f581027a0 2024-09-12T12:11:55.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.033+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f58062980 con 0x7f3f581027a0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.034+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3f4001c070 con 0x7f3f581027a0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.035+0000 7f3f4ffff640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3f440a2c90 0x7f3f440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.035+0000 7f3f56ffd640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3f440a2c90 0x7f3f440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.035+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f3f400ce4d0 con 0x7f3f581027a0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.035+0000 7f3f4ffff640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] conn(0x7f3f440ac4e0 0x7f3f440ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.035+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 --> [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f440aefb0 con 0x7f3f440ac4e0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.035+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f3f400d5050 con 0x7f3f581027a0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.035+0000 7f3f56ffd640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3f440a2c90 0x7f3f440a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000a8f0 tx=0x7f3f50005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.036+0000 7f3f577fe640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] conn(0x7f3f440ac4e0 0x7f3f440ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.036+0000 7f3f577fe640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] conn(0x7f3f440ac4e0 0x7f3f440ae900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.037+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== osd.8 v2:172.21.15.189:6816/1497312216 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f440aefb0 con 0x7f3f440ac4e0 2024-09-12T12:11:55.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.050+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 --> [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f58062d80 con 0x7f3f440ac4e0 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f4ffff640 1 -- 172.21.15.137:0/1917767806 <== osd.8 v2:172.21.15.189:6816/1497312216 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3f58062d80 con 0x7f3f440ac4e0 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 >> [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] conn(0x7f3f440ac4e0 msgr2=0x7f3f440ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] conn(0x7f3f440ac4e0 0x7f3f440ae900 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3f440a2c90 msgr2=0x7f3f440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f3f440a2c90 0x7f3f440a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000a8f0 tx=0x7f3f50005bf0 comp rx=0 tx=0).stop 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f581027a0 msgr2=0x7f3f58198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 --2- 172.21.15.137:0/1917767806 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f581027a0 0x7f3f58198940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000ba50 tx=0x7f3f4000bf10 comp rx=0 tx=0).stop 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f577fe640 1 -- 172.21.15.137:0/1917767806 reap_dead start 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 shutdown_connections 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 >> 172.21.15.137:0/1917767806 conn(0x7f3f580fd8b0 msgr2=0x7f3f580feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 shutdown_connections 2024-09-12T12:11:55.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:55.051+0000 7f3f5d63c640 1 -- 172.21.15.137:0/1917767806 wait complete. 2024-09-12T12:11:55.878 INFO:teuthology.orchestra.run.smithi137.stdout:240518168614 2024-09-12T12:11:55.878 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T12:11:56.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 -- 172.21.15.137:0/3932367464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5410e4f0 msgr2=0x7f9c54110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 --2- 172.21.15.137:0/3932367464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5410e4f0 0x7f9c54110900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009ca0 tx=0x7f9c4c02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 -- 172.21.15.137:0/3932367464 shutdown_connections 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 --2- 172.21.15.137:0/3932367464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5410e4f0 0x7f9c54110900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 --2- 172.21.15.137:0/3932367464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c54101380 0x7f9c5410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 --2- 172.21.15.137:0/3932367464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c541009d0 0x7f9c54100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 -- 172.21.15.137:0/3932367464 >> 172.21.15.137:0/3932367464 conn(0x7f9c540fc7d0 msgr2=0x7f9c540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 -- 172.21.15.137:0/3932367464 shutdown_connections 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 -- 172.21.15.137:0/3932367464 wait complete. 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 Processor -- start 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.043+0000 7f9c5b3ab640 1 -- start start 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c5b3ab640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c541009d0 0x7f9c5419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c5b3ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c54101380 0x7f9c5419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c5b3ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c5410e4f0 0x7f9c5419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c5b3ab640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54114250 con 0x7f9c54101380 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c5b3ab640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541140d0 con 0x7f9c541009d0 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c5b3ab640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541143d0 con 0x7f9c5410e4f0 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c54101380 0x7f9c5419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c54101380 0x7f9c5419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51286/0 (socket says 172.21.15.137:51286) 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 -- 172.21.15.137:0/1197698582 learned_addr learned my addr 172.21.15.137:0/1197698582 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c59948640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c5410e4f0 0x7f9c5419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c59147640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c541009d0 0x7f9c5419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 -- 172.21.15.137:0/1197698582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c5410e4f0 msgr2=0x7f9c5419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c5410e4f0 0x7f9c5419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 -- 172.21.15.137:0/1197698582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c541009d0 msgr2=0x7f9c5419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c541009d0 0x7f9c5419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 -- 172.21.15.137:0/1197698582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5419f6a0 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c59147640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9c541009d0 0x7f9c5419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c58946640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c54101380 0x7f9c5419ac30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d770 tx=0x7f9c4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44017070 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.044+0000 7f9c5b3ab640 1 -- 172.21.15.137:0/1197698582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5419f990 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.045+0000 7f9c5b3ab640 1 -- 172.21.15.137:0/1197698582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541a7270 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.045+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c44005500 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.045+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44020060 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.046+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c34001030 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.047+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9c44009cc0 con 0x7f9c54101380 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c4a7fc640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c400a2c90 0x7f9c400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c59147640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c400a2c90 0x7f9c400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f9c440ceeb0 con 0x7f9c54101380 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c4a7fc640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] conn(0x7f9c400ac4e0 0x7f9c400ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 --> [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c44018ec0 con 0x7f9c400ac4e0 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9c440cf260 con 0x7f9c54101380 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c59147640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c400a2c90 0x7f9c400a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009c20 tx=0x7f9c50005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.048+0000 7f9c59948640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] conn(0x7f9c400ac4e0 0x7f9c400ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.049+0000 7f9c59948640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] conn(0x7f9c400ac4e0 0x7f9c400ae900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.050+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== osd.9 v2:172.21.15.137:6826/4232440350 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c44018ec0 con 0x7f9c400ac4e0 2024-09-12T12:11:56.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 --> [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c34002dd0 con 0x7f9c400ac4e0 2024-09-12T12:11:56.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c4a7fc640 1 -- 172.21.15.137:0/1197698582 <== osd.9 v2:172.21.15.137:6826/4232440350 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c34002dd0 con 0x7f9c400ac4e0 2024-09-12T12:11:56.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] conn(0x7f9c400ac4e0 msgr2=0x7f9c400ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] conn(0x7f9c400ac4e0 0x7f9c400ae900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c400a2c90 msgr2=0x7f9c400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9c400a2c90 0x7f9c400a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009c20 tx=0x7f9c50005f90 comp rx=0 tx=0).stop 2024-09-12T12:11:56.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c54101380 msgr2=0x7f9c5419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 --2- 172.21.15.137:0/1197698582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c54101380 0x7f9c5419ac30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d770 tx=0x7f9c4400dc30 comp rx=0 tx=0).stop 2024-09-12T12:11:56.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c59948640 1 -- 172.21.15.137:0/1197698582 reap_dead start 2024-09-12T12:11:56.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 shutdown_connections 2024-09-12T12:11:56.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.064+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 >> 172.21.15.137:0/1197698582 conn(0x7f9c540fc7d0 msgr2=0x7f9c540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:56.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.065+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 shutdown_connections 2024-09-12T12:11:56.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.065+0000 7f9c3ffff640 1 -- 172.21.15.137:0/1197698582 wait complete. 2024-09-12T12:11:56.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:56 smithi137 ceph-mon[30950]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:11:56.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:56 smithi189 ceph-mon[35475]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:11:56.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.644+0000 7f350e764640 1 -- 172.21.15.137:0/2753599569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102570 msgr2=0x7f35081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 --2- 172.21.15.137:0/2753599569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102570 0x7f35081029b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009900 tx=0x7f34f002a480 comp rx=0 tx=0).stop 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 -- 172.21.15.137:0/2753599569 shutdown_connections 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 --2- 172.21.15.137:0/2753599569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3508102ef0 0x7f350810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 --2- 172.21.15.137:0/2753599569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102570 0x7f35081029b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 --2- 172.21.15.137:0/2753599569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3508100e20 0x7f3508101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 -- 172.21.15.137:0/2753599569 >> 172.21.15.137:0/2753599569 conn(0x7f35080fc7d0 msgr2=0x7f35080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 -- 172.21.15.137:0/2753599569 shutdown_connections 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 -- 172.21.15.137:0/2753599569 wait complete. 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 Processor -- start 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 -- start start 2024-09-12T12:11:56.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.645+0000 7f350e764640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350e764640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3508110f80 con 0x7f3508102ef0 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350e764640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3508110e00 con 0x7f3508102570 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350e764640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3508111100 con 0x7f3508100e20 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51306/0 (socket says 172.21.15.137:51306) 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 -- 172.21.15.137:0/3190102113 learned_addr learned my addr 172.21.15.137:0/3190102113 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f3507fff640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f34fffff640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 -- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3508100e20 msgr2=0x7f35080fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3508100e20 0x7f35080fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 -- 172.21.15.137:0/3190102113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3508102570 msgr2=0x7f35080fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3508102570 0x7f35080fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 -- 172.21.15.137:0/3190102113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35081a69a0 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350cd01640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f350000cce0 tx=0x7f350000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350000f450 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35081a6c90 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f350000fad0 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500018c10 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.646+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a71d0 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.647+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35080628c0 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.649+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f350000f5f0 con 0x7f3508102ef0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.649+0000 7f3505ffb640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34f40a2c10 0x7f34f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.649+0000 7f3507fff640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34f40a2c10 0x7f34f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.649+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f35000cf8f0 con 0x7f3508102ef0 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.649+0000 7f3505ffb640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] conn(0x7f34f40ac460 0x7f34f40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.649+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 --> [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34f40aef30 con 0x7f34f40ac460 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.649+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f35000cfdd0 con 0x7f3508102ef0 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.650+0000 7f3507fff640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34f40a2c10 0x7f34f40a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34f80058c0 tx=0x7f34f800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.650+0000 7f34fffff640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] conn(0x7f34f40ac460 0x7f34f40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.650+0000 7f34fffff640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] conn(0x7f34f40ac460 0x7f34f40ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.651+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== osd.11 v2:172.21.15.189:6824/2395385355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34f40aef30 con 0x7f34f40ac460 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.664+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 --> [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3508062d40 con 0x7f34f40ac460 2024-09-12T12:11:56.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f3505ffb640 1 -- 172.21.15.137:0/3190102113 <== osd.11 v2:172.21.15.189:6824/2395385355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3508062d40 con 0x7f34f40ac460 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] conn(0x7f34f40ac460 msgr2=0x7f34f40ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] conn(0x7f34f40ac460 0x7f34f40ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34f40a2c10 msgr2=0x7f34f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34f40a2c10 0x7f34f40a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f34f80058c0 tx=0x7f34f800a2d0 comp rx=0 tx=0).stop 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102ef0 msgr2=0x7f35080fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 --2- 172.21.15.137:0/3190102113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3508102ef0 0x7f35080fd700 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f350000cce0 tx=0x7f350000a620 comp rx=0 tx=0).stop 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350cd01640 1 -- 172.21.15.137:0/3190102113 reap_dead start 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 shutdown_connections 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 >> 172.21.15.137:0/3190102113 conn(0x7f35080fc7d0 msgr2=0x7f350806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:56.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 shutdown_connections 2024-09-12T12:11:56.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:56.665+0000 7f350e764640 1 -- 172.21.15.137:0/3190102113 wait complete. 2024-09-12T12:11:56.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:56 smithi171 ceph-mon[35098]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:11:57.599 INFO:teuthology.orchestra.run.smithi137.stdout:322122547222 2024-09-12T12:11:57.600 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T12:11:58.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:11:58 smithi137 ceph-mon[30950]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:11:58.348 INFO:teuthology.orchestra.run.smithi137.stdout:55834574922 2024-09-12T12:11:58.349 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T12:11:58.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:11:58 smithi189 ceph-mon[35475]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:11:58.666 INFO:teuthology.orchestra.run.smithi137.stdout:188978561071 2024-09-12T12:11:58.666 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T12:11:58.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:11:58 smithi171 ceph-mon[35098]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:11:58.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4180880493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 msgr2=0x7fabe0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:58.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 --2- 172.21.15.137:0/4180880493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 0x7fabe0100ed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009930 tx=0x7fabc802a420 comp rx=0 tx=0).stop 2024-09-12T12:11:58.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4180880493 shutdown_connections 2024-09-12T12:11:58.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 --2- 172.21.15.137:0/4180880493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabe0101410 0x7fabe010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:58.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 --2- 172.21.15.137:0/4180880493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 0x7fabe0100ed0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:58.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 --2- 172.21.15.137:0/4180880493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe0106a70 0x7fabe0106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:58.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4180880493 >> 172.21.15.137:0/4180880493 conn(0x7fabe00fc790 msgr2=0x7fabe00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4180880493 shutdown_connections 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.838+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4180880493 wait complete. 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 Processor -- start 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 -- start start 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 0x7fabe019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe0101410 0x7fabe019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabe0106a70 0x7fabe019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0110e70 con 0x7fabe0100a70 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0110cf0 con 0x7fabe0101410 2024-09-12T12:11:58.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabe4c7a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fabe0110ff0 con 0x7fabe0106a70 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabde59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 0x7fabe019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabde59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 0x7fabe019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51310/0 (socket says 172.21.15.137:51310) 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabd7fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe0101410 0x7fabe019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabde59c640 1 -- 172.21.15.137:0/4042657751 learned_addr learned my addr 172.21.15.137:0/4042657751 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.839+0000 7fabded9d640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabe0106a70 0x7fabe019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabde59c640 1 -- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabe0106a70 msgr2=0x7fabe019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabde59c640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fabe0106a70 0x7fabe019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabde59c640 1 -- 172.21.15.137:0/4042657751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe0101410 msgr2=0x7fabe019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabde59c640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe0101410 0x7fabe019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabde59c640 1 -- 172.21.15.137:0/4042657751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe019f600 con 0x7fabe0100a70 2024-09-12T12:11:58.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabde59c640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 0x7fabe019a650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c890 tx=0x7fabd000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:58.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd0004220 con 0x7fabe0100a70 2024-09-12T12:11:58.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe019f8f0 con 0x7fabe0100a70 2024-09-12T12:11:58.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabd00043c0 con 0x7fabe0100a70 2024-09-12T12:11:58.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd00049e0 con 0x7fabe0100a70 2024-09-12T12:11:58.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.840+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe01a71d0 con 0x7fabe0100a70 2024-09-12T12:11:58.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.841+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabe0062940 con 0x7fabe0100a70 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.842+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fabd00079b0 con 0x7fabe0100a70 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.843+0000 7fabd77fe640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fabcc0a2c10 0x7fabcc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.843+0000 7fabd7fff640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fabcc0a2c10 0x7fabcc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.843+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fabd00cf150 con 0x7fabe0100a70 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.843+0000 7fabd77fe640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] conn(0x7fabcc0ac460 0x7fabcc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.843+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 --> [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabcc0aef30 con 0x7fabcc0ac460 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.843+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fabd00cf630 con 0x7fabe0100a70 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.843+0000 7fabded9d640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] conn(0x7fabcc0ac460 0x7fabcc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.844+0000 7fabd7fff640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fabcc0a2c10 0x7fabcc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009900 tx=0x7fabc80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.844+0000 7fabded9d640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] conn(0x7fabcc0ac460 0x7fabcc0ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:11:58.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.845+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== osd.2 v2:172.21.15.189:6800/2022208006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabcc0aef30 con 0x7fabcc0ac460 2024-09-12T12:11:58.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.861+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 --> [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabe0062dc0 con 0x7fabcc0ac460 2024-09-12T12:11:58.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabd77fe640 1 -- 172.21.15.137:0/4042657751 <== osd.2 v2:172.21.15.189:6800/2022208006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fabe0062dc0 con 0x7fabcc0ac460 2024-09-12T12:11:58.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] conn(0x7fabcc0ac460 msgr2=0x7fabcc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:58.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] conn(0x7fabcc0ac460 0x7fabcc0ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:11:58.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fabcc0a2c10 msgr2=0x7fabcc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:58.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fabcc0a2c10 0x7fabcc0a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009900 tx=0x7fabc80058d0 comp rx=0 tx=0).stop 2024-09-12T12:11:58.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 msgr2=0x7fabe019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:11:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 --2- 172.21.15.137:0/4042657751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabe0100a70 0x7fabe019a650 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fabd000c890 tx=0x7fabd000cd50 comp rx=0 tx=0).stop 2024-09-12T12:11:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabded9d640 1 -- 172.21.15.137:0/4042657751 reap_dead start 2024-09-12T12:11:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 shutdown_connections 2024-09-12T12:11:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 >> 172.21.15.137:0/4042657751 conn(0x7fabe00fc790 msgr2=0x7fabe006ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:11:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 shutdown_connections 2024-09-12T12:11:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:11:58.862+0000 7fabe4c7a640 1 -- 172.21.15.137:0/4042657751 wait complete. 2024-09-12T12:11:59.665 INFO:teuthology.orchestra.run.smithi137.stdout:77309411398 2024-09-12T12:11:59.666 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T12:12:00.134 INFO:teuthology.orchestra.run.smithi137.stdout:158913790004 2024-09-12T12:12:00.134 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T12:12:00.334 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:00 smithi137 ceph-mon[30950]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:00.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:00 smithi189 ceph-mon[35475]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:00.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:00 smithi171 ceph-mon[35098]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:01.060 INFO:teuthology.orchestra.run.smithi137.stdout:270582939681 2024-09-12T12:12:01.061 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T12:12:01.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 -- 172.21.15.137:0/3650743185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100480 msgr2=0x7f7830100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 --2- 172.21.15.137:0/3650743185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100480 0x7f7830100860 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7820009900 tx=0x7f782002a410 comp rx=0 tx=0).stop 2024-09-12T12:12:01.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 -- 172.21.15.137:0/3650743185 shutdown_connections 2024-09-12T12:12:01.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 --2- 172.21.15.137:0/3650743185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78301055b0 0x7f783010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 --2- 172.21.15.137:0/3650743185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7830100e30 0x7f7830104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 --2- 172.21.15.137:0/3650743185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100480 0x7f7830100860 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 -- 172.21.15.137:0/3650743185 >> 172.21.15.137:0/3650743185 conn(0x7f7830077210 msgr2=0x7f78300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 -- 172.21.15.137:0/3650743185 shutdown_connections 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.787+0000 7f7834b3d640 1 -- 172.21.15.137:0/3650743185 wait complete. 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 Processor -- start 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 -- start start 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7830100480 0x7f7830199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100e30 0x7f7830199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78301055b0 0x7f78301a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f78301106b0 con 0x7f7830100e30 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7830110530 con 0x7f7830100480 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7834b3d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7830110830 con 0x7f78301055b0 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100e30 0x7f7830199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100e30 0x7f7830199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60572/0 (socket says 172.21.15.137:60572) 2024-09-12T12:12:01.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 -- 172.21.15.137:0/518806827 learned_addr learned my addr 172.21.15.137:0/518806827 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:01.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f782e59c640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7830100480 0x7f7830199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 -- 172.21.15.137:0/518806827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78301055b0 msgr2=0x7f78301a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f782ed9d640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78301055b0 0x7f78301a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78301055b0 0x7f78301a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 -- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7830100480 msgr2=0x7f7830199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7830100480 0x7f7830199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 -- 172.21.15.137:0/518806827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78301a49c0 con 0x7f7830100e30 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f782e59c640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7830100480 0x7f7830199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.788+0000 7f7827fff640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100e30 0x7f7830199940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f781800b4a0 tx=0x7f781800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.789+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818014070 con 0x7f7830100e30 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.789+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78301a4b90 con 0x7f7830100e30 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.789+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7818010af0 con 0x7f7830100e30 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.789+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818017d10 con 0x7f7830100e30 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.789+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78301a5080 con 0x7f7830100e30 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.790+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78300fef30 con 0x7f7830100e30 2024-09-12T12:12:01.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.791+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7818017070 con 0x7f7830100e30 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.791+0000 7f78277fe640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f781c0a2c10 0x7f781c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.791+0000 7f782e59c640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f781c0a2c10 0x7f781c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.791+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f78180ce330 con 0x7f7830100e30 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.792+0000 7f78277fe640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] conn(0x7f781c0ac460 0x7f781c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.792+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 --> [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f781c0aef30 con 0x7f781c0ac460 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.792+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f78180d5050 con 0x7f7830100e30 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.792+0000 7f782e59c640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f781c0a2c10 0x7f781c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7820005ab0 tx=0x7f7820005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.792+0000 7f782ed9d640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] conn(0x7f781c0ac460 0x7f781c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.792+0000 7f782ed9d640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] conn(0x7f781c0ac460 0x7f781c0ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.793+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== osd.7 v2:172.21.15.171:6816/665345582 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f781c0aef30 con 0x7f781c0ac460 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.807+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 --> [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7830101960 con 0x7f781c0ac460 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.807+0000 7f78277fe640 1 -- 172.21.15.137:0/518806827 <== osd.7 v2:172.21.15.171:6816/665345582 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7830101960 con 0x7f781c0ac460 2024-09-12T12:12:01.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.807+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] conn(0x7f781c0ac460 msgr2=0x7f781c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.807+0000 7f7834b3d640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] conn(0x7f781c0ac460 0x7f781c0ae880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f781c0a2c10 msgr2=0x7f781c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f781c0a2c10 0x7f781c0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7820005ab0 tx=0x7f7820005a20 comp rx=0 tx=0).stop 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100e30 msgr2=0x7f7830199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 --2- 172.21.15.137:0/518806827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7830100e30 0x7f7830199940 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f781800b4a0 tx=0x7f781800b960 comp rx=0 tx=0).stop 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f782ed9d640 1 -- 172.21.15.137:0/518806827 reap_dead start 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 shutdown_connections 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 >> 172.21.15.137:0/518806827 conn(0x7f7830077210 msgr2=0x7f78300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 shutdown_connections 2024-09-12T12:12:01.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.808+0000 7f7834b3d640 1 -- 172.21.15.137:0/518806827 wait complete. 2024-09-12T12:12:01.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 -- 172.21.15.137:0/1162639113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 msgr2=0x7f034810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 --2- 172.21.15.137:0/1162639113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 0x7f034810eaa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0340009ca0 tx=0x7f034002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 -- 172.21.15.137:0/1162639113 shutdown_connections 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 --2- 172.21.15.137:0/1162639113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 0x7f034810eaa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 --2- 172.21.15.137:0/1162639113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03481027a0 0x7f03481067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 --2- 172.21.15.137:0/1162639113 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0348101df0 0x7f03481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 -- 172.21.15.137:0/1162639113 >> 172.21.15.137:0/1162639113 conn(0x7f03480fd8b0 msgr2=0x7f03480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 -- 172.21.15.137:0/1162639113 shutdown_connections 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.677+0000 7f034ff4d640 1 -- 172.21.15.137:0/1162639113 wait complete. 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 Processor -- start 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 -- start start 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0348101df0 0x7f03480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03481027a0 0x7f0348074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 0x7f0348074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0348112020 con 0x7f0348106f20 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0348111ea0 con 0x7f0348101df0 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03481121a0 con 0x7f03481027a0 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 0x7f0348074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 0x7f0348074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60562/0 (socket says 172.21.15.137:60562) 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 -- 172.21.15.137:0/308682344 learned_addr learned my addr 172.21.15.137:0/308682344 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 -- 172.21.15.137:0/308682344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03481027a0 msgr2=0x7f0348074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03481027a0 0x7f0348074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 -- 172.21.15.137:0/308682344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0348101df0 msgr2=0x7f03480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0348101df0 0x7f03480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 -- 172.21.15.137:0/308682344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03481a69e0 con 0x7f0348106f20 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034e4ea640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 0x7f0348074f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0340031a60 tx=0x7f0340031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340003c30 con 0x7f0348106f20 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0340003dd0 con 0x7f0348106f20 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03481a6c70 con 0x7f0348106f20 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.678+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400047e0 con 0x7f0348106f20 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.679+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03481a7200 con 0x7f0348106f20 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.679+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0348062900 con 0x7f0348106f20 2024-09-12T12:12:01.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.681+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f034002f790 con 0x7f0348106f20 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.681+0000 7f033effd640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f03340a2c90 0x7f03340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.681+0000 7f034dce9640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f03340a2c90 0x7f03340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.681+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f03400f05d0 con 0x7f0348106f20 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.681+0000 7f033effd640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] conn(0x7f03340ac3b0 0x7f03340ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.681+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 --> [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03340aee80 con 0x7f03340ac3b0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.681+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f03400f0ab0 con 0x7f0348106f20 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.682+0000 7f034dce9640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f03340a2c90 0x7f03340a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0344009710 tx=0x7f0344006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.682+0000 7f034d4e8640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] conn(0x7f03340ac3b0 0x7f03340ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.682+0000 7f034d4e8640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] conn(0x7f03340ac3b0 0x7f03340ae7d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.683+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== osd.4 v2:172.21.15.171:6808/1838766520 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03340aee80 con 0x7f03340ac3b0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.696+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 --> [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0348062d00 con 0x7f03340ac3b0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f033effd640 1 -- 172.21.15.137:0/308682344 <== osd.4 v2:172.21.15.171:6808/1838766520 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0348062d00 con 0x7f03340ac3b0 2024-09-12T12:12:01.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 >> [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] conn(0x7f03340ac3b0 msgr2=0x7f03340ae7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] conn(0x7f03340ac3b0 0x7f03340ae7d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f03340a2c90 msgr2=0x7f03340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f03340a2c90 0x7f03340a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0344009710 tx=0x7f0344006c90 comp rx=0 tx=0).stop 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 msgr2=0x7f0348074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 --2- 172.21.15.137:0/308682344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0348106f20 0x7f0348074f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0340031a60 tx=0x7f0340031a90 comp rx=0 tx=0).stop 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034e4ea640 1 -- 172.21.15.137:0/308682344 reap_dead start 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 shutdown_connections 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 >> 172.21.15.137:0/308682344 conn(0x7f03480fd8b0 msgr2=0x7f03480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 shutdown_connections 2024-09-12T12:12:01.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:00.697+0000 7f034ff4d640 1 -- 172.21.15.137:0/308682344 wait complete. 2024-09-12T12:12:02.189 INFO:teuthology.orchestra.run.smithi137.stdout:137438953531 2024-09-12T12:12:02.189 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T12:12:02.192 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:02.192 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:02.214 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:02.476 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:02.477 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:02.477 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:02 smithi137 ceph-mon[30950]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T12:12:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:02 smithi189 ceph-mon[35475]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T12:12:02.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:02 smithi171 ceph-mon[35098]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-12T12:12:02.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 -- 172.21.15.137:0/1415984676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e41082a0 msgr2=0x7f58e410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:02.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 --2- 172.21.15.137:0/1415984676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e41082a0 0x7f58e410ec40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009cd0 tx=0x7f58d402a8e0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 -- 172.21.15.137:0/1415984676 shutdown_connections 2024-09-12T12:12:02.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 --2- 172.21.15.137:0/1415984676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e41082a0 0x7f58e410ec40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 --2- 172.21.15.137:0/1415984676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58e41036e0 0x7f58e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 --2- 172.21.15.137:0/1415984676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e4102d30 0x7f58e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 -- 172.21.15.137:0/1415984676 >> 172.21.15.137:0/1415984676 conn(0x7f58e40fc7d0 msgr2=0x7f58e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 -- 172.21.15.137:0/1415984676 shutdown_connections 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 -- 172.21.15.137:0/1415984676 wait complete. 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 Processor -- start 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.743+0000 7f58ec13b640 1 -- start start 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4102d30 0x7f58e4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58e41036e0 0x7f58e4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41082a0 0x7f58e419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41120a0 con 0x7f58e4102d30 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4111f20 con 0x7f58e41036e0 2024-09-12T12:12:02.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4112220 con 0x7f58e41082a0 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4102d30 0x7f58e4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4102d30 0x7f58e4198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60584/0 (socket says 172.21.15.137:60584) 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 -- 172.21.15.137:0/3766117718 learned_addr learned my addr 172.21.15.137:0/3766117718 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e96d6640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58e41036e0 0x7f58e4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ea6d8640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41082a0 0x7f58e419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 -- 172.21.15.137:0/3766117718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41082a0 msgr2=0x7f58e419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41082a0 0x7f58e419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 -- 172.21.15.137:0/3766117718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58e41036e0 msgr2=0x7f58e4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58e41036e0 0x7f58e4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 -- 172.21.15.137:0/3766117718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e419d4e0 con 0x7f58e4102d30 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e9ed7640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4102d30 0x7f58e4198530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00d950 tx=0x7f58dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e2ffd640 1 -- 172.21.15.137:0/3766117718 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc017070 con 0x7f58e4102d30 2024-09-12T12:12:02.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e419d7d0 con 0x7f58e4102d30 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e2ffd640 1 -- 172.21.15.137:0/3766117718 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58dc009c30 con 0x7f58e4102d30 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58e2ffd640 1 -- 172.21.15.137:0/3766117718 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00f6b0 con 0x7f58e4102d30 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.744+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41a50b0 con 0x7f58e4102d30 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.746+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e4103110 con 0x7f58e4102d30 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.747+0000 7f58e2ffd640 1 -- 172.21.15.137:0/3766117718 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f58dc0109d0 con 0x7f58e4102d30 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.747+0000 7f58e2ffd640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.747+0000 7f58e2ffd640 1 -- 172.21.15.137:0/3766117718 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f58dc0cec90 con 0x7f58e4102d30 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.747+0000 7f58e96d6640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.748+0000 7f58e96d6640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f58e40fed80 tx=0x7f58d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:02.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.752+0000 7f58e2ffd640 1 -- 172.21.15.137:0/3766117718 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58dc009de0 con 0x7f58e4102d30 2024-09-12T12:12:02.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.931+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f58e410a2b0 con 0x7f58e4102d30 2024-09-12T12:12:02.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.931+0000 7f58e2ffd640 1 -- 172.21.15.137:0/3766117718 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58dc09bc40 con 0x7f58e4102d30 2024-09-12T12:12:02.931 INFO:teuthology.orchestra.run.smithi137.stdout:77309411398 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f58cc0a2c10 msgr2=0x7f58cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f58e40fed80 tx=0x7f58d800b390 comp rx=0 tx=0).stop 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4102d30 msgr2=0x7f58e4198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4102d30 0x7f58e4198530 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00d950 tx=0x7f58dc00de10 comp rx=0 tx=0).stop 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 shutdown_connections 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e41082a0 0x7f58e419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58e41036e0 0x7f58e4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 --2- 172.21.15.137:0/3766117718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58e4102d30 0x7f58e4198530 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 >> 172.21.15.137:0/3766117718 conn(0x7f58e40fc7d0 msgr2=0x7f58e4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:02.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 shutdown_connections 2024-09-12T12:12:02.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:02.933+0000 7f58ec13b640 1 -- 172.21.15.137:0/3766117718 wait complete. 2024-09-12T12:12:03.116 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:03.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:03 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3766117718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T12:12:03.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1894540848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 msgr2=0x7f4a881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1894540848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a881067e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009900 tx=0x7f4a7002a410 comp rx=0 tx=0).stop 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1894540848 shutdown_connections 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1894540848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a88106f10 0x7f4a8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1894540848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a881067e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1894540848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a88101de0 0x7f4a881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1894540848 >> 172.21.15.137:0/1894540848 conn(0x7f4a880fd8c0 msgr2=0x7f4a880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1894540848 shutdown_connections 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.542+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1894540848 wait complete. 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 Processor -- start 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 -- start start 2024-09-12T12:12:03.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a88101de0 0x7f4a88198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a88198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a88106f10 0x7f4a8819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88111f70 con 0x7f4a88102790 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88111df0 con 0x7f4a88106f10 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a8d61a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881120f0 con 0x7f4a88101de0 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a7e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a88198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a7e7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a88198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60604/0 (socket says 172.21.15.137:60604) 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a7e7fc640 1 -- 172.21.15.137:0/1139479827 learned_addr learned my addr 172.21.15.137:0/1139479827 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a877fe640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a88106f10 0x7f4a8819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.543+0000 7f4a86ffd640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a88101de0 0x7f4a88198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a7e7fc640 1 -- 172.21.15.137:0/1139479827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a88101de0 msgr2=0x7f4a88198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a7e7fc640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a88101de0 0x7f4a88198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a7e7fc640 1 -- 172.21.15.137:0/1139479827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a88106f10 msgr2=0x7f4a8819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a7e7fc640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a88106f10 0x7f4a8819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a7e7fc640 1 -- 172.21.15.137:0/1139479827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8819d400 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a7e7fc640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a88198990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006010 tx=0x7f4a700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a84ff9640 1 -- 172.21.15.137:0/1139479827 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7003f030 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8819d690 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a84ff9640 1 -- 172.21.15.137:0/1139479827 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a70003ce0 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a84ff9640 1 -- 172.21.15.137:0/1139479827 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70035070 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.544+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a881a4fd0 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.545+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a881021c0 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.546+0000 7f4a84ff9640 1 -- 172.21.15.137:0/1139479827 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4a70042030 con 0x7f4a88102790 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.547+0000 7f4a84ff9640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f4a740a2c10 0x7f4a740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.547+0000 7f4a86ffd640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f4a740a2c10 0x7f4a740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:03.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.547+0000 7f4a84ff9640 1 -- 172.21.15.137:0/1139479827 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f4a700ef630 con 0x7f4a88102790 2024-09-12T12:12:03.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.548+0000 7f4a86ffd640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f4a740a2c10 0x7f4a740a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800bbe0 tx=0x7f4a7800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:03.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.551+0000 7f4a84ff9640 1 -- 172.21.15.137:0/1139479827 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a700bc050 con 0x7f4a88102790 2024-09-12T12:12:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:03 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3766117718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T12:12:03.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:03 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3766117718' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T12:12:03.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.727+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4a8810a150 con 0x7f4a88102790 2024-09-12T12:12:03.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.728+0000 7f4a84ff9640 1 -- 172.21.15.137:0/1139479827 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4a700c0350 con 0x7f4a88102790 2024-09-12T12:12:03.728 INFO:teuthology.orchestra.run.smithi137.stdout:55834574923 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f4a740a2c10 msgr2=0x7f4a740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f4a740a2c10 0x7f4a740a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800bbe0 tx=0x7f4a7800b390 comp rx=0 tx=0).stop 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 msgr2=0x7f4a88198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a88198990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006010 tx=0x7f4a700042f0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 shutdown_connections 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f4a740a2c10 0x7f4a740a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a88106f10 0x7f4a8819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a88102790 0x7f4a88198990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 --2- 172.21.15.137:0/1139479827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a88101de0 0x7f4a88198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:03.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.730+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 >> 172.21.15.137:0/1139479827 conn(0x7f4a880fd8c0 msgr2=0x7f4a880febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:03.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.731+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 shutdown_connections 2024-09-12T12:12:03.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:03.731+0000 7f4a8d61a640 1 -- 172.21.15.137:0/1139479827 wait complete. 2024-09-12T12:12:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 -- 172.21.15.137:0/1014623514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238106a80 msgr2=0x7f5238106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1014623514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238106a80 0x7f5238106e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5228009930 tx=0x7f522802a420 comp rx=0 tx=0).stop 2024-09-12T12:12:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 -- 172.21.15.137:0/1014623514 shutdown_connections 2024-09-12T12:12:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1014623514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238101420 0x7f523810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1014623514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238100a80 0x7f5238100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1014623514 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238106a80 0x7f5238106e60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 -- 172.21.15.137:0/1014623514 >> 172.21.15.137:0/1014623514 conn(0x7f52380fc7a0 msgr2=0x7f52380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 -- 172.21.15.137:0/1014623514 shutdown_connections 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 -- 172.21.15.137:0/1014623514 wait complete. 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 Processor -- start 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.258+0000 7f523cc7f640 1 -- start start 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238100a80 0x7f5238102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101420 0x7f5238102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238106a80 0x7f52381030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5238110ea0 con 0x7f5238101420 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5238110d20 con 0x7f5238106a80 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111020 con 0x7f5238100a80 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101420 0x7f5238102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101420 0x7f5238102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60608/0 (socket says 172.21.15.137:60608) 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 -- 172.21.15.137:0/1799240699 learned_addr learned my addr 172.21.15.137:0/1799240699 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5236d9d640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238106a80 0x7f52381030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 -- 172.21.15.137:0/1799240699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238100a80 msgr2=0x7f5238102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523659c640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238100a80 0x7f5238102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238100a80 0x7f5238102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 -- 172.21.15.137:0/1799240699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238106a80 msgr2=0x7f52381030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238106a80 0x7f52381030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 -- 172.21.15.137:0/1799240699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52381a6980 con 0x7f5238101420 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5236d9d640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238106a80 0x7f52381030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f5235d9b640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101420 0x7f5238102b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f522000d950 tx=0x7f522000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f522f7fe640 1 -- 172.21.15.137:0/1799240699 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5220017070 con 0x7f5238101420 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f522f7fe640 1 -- 172.21.15.137:0/1799240699 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5220009c30 con 0x7f5238101420 2024-09-12T12:12:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52381a6c70 con 0x7f5238101420 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f522f7fe640 1 -- 172.21.15.137:0/1799240699 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522000f6b0 con 0x7f5238101420 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.259+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381a71b0 con 0x7f5238101420 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.261+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5238100ee0 con 0x7f5238101420 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.261+0000 7f522f7fe640 1 -- 172.21.15.137:0/1799240699 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f52200109d0 con 0x7f5238101420 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.262+0000 7f522f7fe640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.262+0000 7f523659c640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.262+0000 7f522f7fe640 1 -- 172.21.15.137:0/1799240699 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f52200ced20 con 0x7f5238101420 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.263+0000 7f523659c640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f52240a2c10 0x7f52240a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f52280096c0 tx=0x7f5228000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.266+0000 7f522f7fe640 1 -- 172.21.15.137:0/1799240699 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5220009de0 con 0x7f5238101420 2024-09-12T12:12:04.427 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:04 smithi137 ceph-mon[30950]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-12T12:12:04.427 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:04 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1139479827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T12:12:04.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.435+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f52381a7380 con 0x7f5238101420 2024-09-12T12:12:04.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.436+0000 7f522f7fe640 1 -- 172.21.15.137:0/1799240699 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f522009bcd0 con 0x7f5238101420 2024-09-12T12:12:04.436 INFO:teuthology.orchestra.run.smithi137.stdout:322122547223 2024-09-12T12:12:04.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f52240a2c10 msgr2=0x7f52240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f52240a2c10 0x7f52240a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f52280096c0 tx=0x7f5228000bc0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101420 msgr2=0x7f5238102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101420 0x7f5238102b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f522000d950 tx=0x7f522000de10 comp rx=0 tx=0).stop 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 shutdown_connections 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238106a80 0x7f52381030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101420 0x7f5238102b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 --2- 172.21.15.137:0/1799240699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238100a80 0x7f5238102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 >> 172.21.15.137:0/1799240699 conn(0x7f52380fc7a0 msgr2=0x7f523806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 shutdown_connections 2024-09-12T12:12:04.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.438+0000 7f523cc7f640 1 -- 172.21.15.137:0/1799240699 wait complete. 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 -- 172.21.15.137:0/1597014443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8106a80 msgr2=0x7fafa8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 --2- 172.21.15.137:0/1597014443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8106a80 0x7fafa8106e60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faf98009930 tx=0x7faf9802a420 comp rx=0 tx=0).stop 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 -- 172.21.15.137:0/1597014443 shutdown_connections 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 --2- 172.21.15.137:0/1597014443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa8101420 0x7fafa810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 --2- 172.21.15.137:0/1597014443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafa8100a80 0x7fafa8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 --2- 172.21.15.137:0/1597014443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8106a80 0x7fafa8106e60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 -- 172.21.15.137:0/1597014443 >> 172.21.15.137:0/1597014443 conn(0x7fafa80fc7a0 msgr2=0x7fafa80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 -- 172.21.15.137:0/1597014443 shutdown_connections 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.511+0000 7fafae0e1640 1 -- 172.21.15.137:0/1597014443 wait complete. 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 Processor -- start 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 -- start start 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa8100a80 0x7fafa8197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8101420 0x7fafa81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafa8106a80 0x7fafa819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8110e80 con 0x7fafa8101420 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8110d00 con 0x7fafa8106a80 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafae0e1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8111000 con 0x7fafa8100a80 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8101420 0x7fafa81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafa6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8101420 0x7fafa81978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60632/0 (socket says 172.21.15.137:60632) 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.512+0000 7fafa6ffd640 1 -- 172.21.15.137:0/375083828 learned_addr learned my addr 172.21.15.137:0/375083828 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa77fe640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa8100a80 0x7fafa8197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa7fff640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafa8106a80 0x7fafa819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa6ffd640 1 -- 172.21.15.137:0/375083828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa8100a80 msgr2=0x7fafa8197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa6ffd640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa8100a80 0x7fafa8197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa6ffd640 1 -- 172.21.15.137:0/375083828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafa8106a80 msgr2=0x7fafa819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa6ffd640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafa8106a80 0x7fafa819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa6ffd640 1 -- 172.21.15.137:0/375083828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa819c330 con 0x7fafa8101420 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa7fff640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafa8106a80 0x7fafa819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa6ffd640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8101420 0x7fafa81978c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d950 tx=0x7faf9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa4ff9640 1 -- 172.21.15.137:0/375083828 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c017070 con 0x7fafa8101420 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa4ff9640 1 -- 172.21.15.137:0/375083828 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf9c009c30 con 0x7fafa8101420 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa819c620 con 0x7fafa8101420 2024-09-12T12:12:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafa4ff9640 1 -- 172.21.15.137:0/375083828 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c00f6b0 con 0x7fafa8101420 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.513+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa81a3f00 con 0x7fafa8101420 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.514+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa8100ee0 con 0x7fafa8101420 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.515+0000 7fafa4ff9640 1 -- 172.21.15.137:0/375083828 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7faf9c0109d0 con 0x7fafa8101420 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.516+0000 7fafa4ff9640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf900a2c10 0x7faf900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.516+0000 7fafa77fe640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf900a2c10 0x7faf900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.516+0000 7fafa4ff9640 1 -- 172.21.15.137:0/375083828 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7faf9c0ced60 con 0x7fafa8101420 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.516+0000 7fafa77fe640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf900a2c10 0x7faf900a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf98000c00 tx=0x7faf98005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.520+0000 7fafa4ff9640 1 -- 172.21.15.137:0/375083828 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf9c009de0 con 0x7fafa8101420 2024-09-12T12:12:04.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:04 smithi189 ceph-mon[35475]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-12T12:12:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:04 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1139479827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T12:12:04.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.691+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fafa8109050 con 0x7fafa8101420 2024-09-12T12:12:04.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.691+0000 7fafa4ff9640 1 -- 172.21.15.137:0/375083828 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faf9c09bd10 con 0x7fafa8101420 2024-09-12T12:12:04.691 INFO:teuthology.orchestra.run.smithi137.stdout:270582939682 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf900a2c10 msgr2=0x7faf900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf900a2c10 0x7faf900a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faf98000c00 tx=0x7faf98005a20 comp rx=0 tx=0).stop 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8101420 msgr2=0x7fafa81978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8101420 0x7fafa81978c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d950 tx=0x7faf9c00de10 comp rx=0 tx=0).stop 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 shutdown_connections 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf900a2c10 0x7faf900a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafa8106a80 0x7fafa819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafa8101420 0x7fafa81978c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 --2- 172.21.15.137:0/375083828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafa8100a80 0x7fafa8197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:04.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 >> 172.21.15.137:0/375083828 conn(0x7fafa80fc7a0 msgr2=0x7fafa80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:04.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 shutdown_connections 2024-09-12T12:12:04.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:04.694+0000 7fafae0e1640 1 -- 172.21.15.137:0/375083828 wait complete. 2024-09-12T12:12:04.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:04 smithi171 ceph-mon[35098]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-09-12T12:12:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:04 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1139479827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T12:12:05.246 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547222 got 322122547223 for osd.11 2024-09-12T12:12:05.246 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:05.504 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:05 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1799240699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T12:12:05.504 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:05 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/375083828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T12:12:05.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:05 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1799240699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T12:12:05.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:05 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/375083828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T12:12:05.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:05 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1799240699' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T12:12:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:05 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/375083828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T12:12:05.904 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939681 got 270582939682 for osd.9 2024-09-12T12:12:05.905 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:06.324 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:06.324 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:06.324 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:06.325 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:06.368 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411398 for osd.2 2024-09-12T12:12:06.368 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:06.506 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:06 smithi137 ceph-mon[30950]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T12:12:06.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:06 smithi189 ceph-mon[35475]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T12:12:06.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:06 smithi171 ceph-mon[35098]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-12T12:12:07.040 INFO:teuthology.orchestra.run.smithi137.stdout:210453397548 2024-09-12T12:12:07.041 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T12:12:07.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 -- 172.21.15.137:0/4206953908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 msgr2=0x7fd65810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 --2- 172.21.15.137:0/4206953908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65810ec30 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd650009cd0 tx=0x7fd65002a8e0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 -- 172.21.15.137:0/4206953908 shutdown_connections 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 --2- 172.21.15.137:0/4206953908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65810ec30 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 --2- 172.21.15.137:0/4206953908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd658103740 0x7fd658107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 --2- 172.21.15.137:0/4206953908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd658102d90 0x7fd658103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 -- 172.21.15.137:0/4206953908 >> 172.21.15.137:0/4206953908 conn(0x7fd6580fc7d0 msgr2=0x7fd6580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 -- 172.21.15.137:0/4206953908 shutdown_connections 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.245+0000 7fd65cad2640 1 -- 172.21.15.137:0/4206953908 wait complete. 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 Processor -- start 2024-09-12T12:12:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 -- start start 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd658102d90 0x7fd658198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd658103740 0x7fd658198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581120c0 con 0x7fd658108290 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd658111f40 con 0x7fd658103740 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65cad2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd658112240 con 0x7fd658102d90 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60650/0 (socket says 172.21.15.137:60650) 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 -- 172.21.15.137:0/1771152003 learned_addr learned my addr 172.21.15.137:0/1771152003 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd65659c640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd658102d90 0x7fd658198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd655d9b640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd658103740 0x7fd658198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:07.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 -- 172.21.15.137:0/1771152003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd658102d90 msgr2=0x7fd658198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd658102d90 0x7fd658198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 -- 172.21.15.137:0/1771152003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd658103740 msgr2=0x7fd658198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd658103740 0x7fd658198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 -- 172.21.15.137:0/1771152003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65819d530 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.246+0000 7fd656d9d640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65819ce50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd650033040 tx=0x7fd650031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.247+0000 7fd6477fe640 1 -- 172.21.15.137:0/1771152003 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650004040 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.247+0000 7fd6477fe640 1 -- 172.21.15.137:0/1771152003 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6500046c0 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.247+0000 7fd6477fe640 1 -- 172.21.15.137:0/1771152003 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650046480 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.247+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65819d7c0 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.247+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6581a5100 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.248+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd658103170 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.249+0000 7fd6477fe640 1 -- 172.21.15.137:0/1771152003 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd6500305a0 con 0x7fd658108290 2024-09-12T12:12:07.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.249+0000 7fd6477fe640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd6400a2c90 0x7fd6400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:07.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.249+0000 7fd65659c640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd6400a2c90 0x7fd6400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:07.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.250+0000 7fd6477fe640 1 -- 172.21.15.137:0/1771152003 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd65002e030 con 0x7fd658108290 2024-09-12T12:12:07.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.250+0000 7fd65659c640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd6400a2c90 0x7fd6400a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd648005d00 tx=0x7fd64800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:07.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.253+0000 7fd6477fe640 1 -- 172.21.15.137:0/1771152003 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd65004b060 con 0x7fd658108290 2024-09-12T12:12:07.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.428+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fd65810a2a0 con 0x7fd658108290 2024-09-12T12:12:07.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.428+0000 7fd6477fe640 1 -- 172.21.15.137:0/1771152003 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd6500bd590 con 0x7fd658108290 2024-09-12T12:12:07.429 INFO:teuthology.orchestra.run.smithi137.stdout:188978561074 2024-09-12T12:12:07.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd6400a2c90 msgr2=0x7fd6400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:07.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd6400a2c90 0x7fd6400a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd648005d00 tx=0x7fd64800b410 comp rx=0 tx=0).stop 2024-09-12T12:12:07.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 msgr2=0x7fd65819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:07.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65819ce50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd650033040 tx=0x7fd650031e60 comp rx=0 tx=0).stop 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 shutdown_connections 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd6400a2c90 0x7fd6400a5150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd658108290 0x7fd65819ce50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd658103740 0x7fd658198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 --2- 172.21.15.137:0/1771152003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd658102d90 0x7fd658198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 >> 172.21.15.137:0/1771152003 conn(0x7fd6580fc7d0 msgr2=0x7fd658106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 shutdown_connections 2024-09-12T12:12:07.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:07.431+0000 7fd65cad2640 1 -- 172.21.15.137:0/1771152003 wait complete. 2024-09-12T12:12:08.263 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:08 smithi189 ceph-mon[35475]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T12:12:08.264 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:08 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:12:08.264 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:08 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1771152003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T12:12:08.321 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561071 got 188978561074 for osd.6 2024-09-12T12:12:08.322 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:08.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:08 smithi171 ceph-mon[35098]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T12:12:08.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:08 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:12:08.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:08 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1771152003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T12:12:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:08 smithi137 ceph-mon[30950]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-12T12:12:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:08 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:12:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:08 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1771152003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T12:12:08.779 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574923 for osd.1 2024-09-12T12:12:08.779 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.396+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1133687280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068103780 msgr2=0x7fd068107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.396+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1133687280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068103780 0x7fd068107bc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009930 tx=0x7fd05c02a420 comp rx=0 tx=0).stop 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1133687280 shutdown_connections 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1133687280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd068108300 0x7fd06810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1133687280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068103780 0x7fd068107bc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1133687280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd068102dd0 0x7fd0681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1133687280 >> 172.21.15.137:0/1133687280 conn(0x7fd0680fc7d0 msgr2=0x7fd0680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:09.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1133687280 shutdown_connections 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1133687280 wait complete. 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.397+0000 7fd06d7bf640 1 Processor -- start 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd06d7bf640 1 -- start start 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd06d7bf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd068102dd0 0x7fd068100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd06d7bf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd068103780 0x7fd068100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd06d7bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068108300 0x7fd0681014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd06d7bf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd068112050 con 0x7fd068108300 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd06d7bf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd068111ed0 con 0x7fd068103780 2024-09-12T12:12:09.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd06d7bf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0681121d0 con 0x7fd068102dd0 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068108300 0x7fd0681014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068108300 0x7fd0681014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47060/0 (socket says 172.21.15.137:47060) 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 -- 172.21.15.137:0/1968722018 learned_addr learned my addr 172.21.15.137:0/1968722018 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd066ffd640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd068102dd0 0x7fd068100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 -- 172.21.15.137:0/1968722018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd068102dd0 msgr2=0x7fd068100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd068102dd0 0x7fd068100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 -- 172.21.15.137:0/1968722018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd068103780 msgr2=0x7fd068100f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd068103780 0x7fd068100f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 -- 172.21.15.137:0/1968722018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0681a6a90 con 0x7fd068108300 2024-09-12T12:12:09.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.398+0000 7fd0677fe640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068108300 0x7fd0681014b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd06000eb10 tx=0x7fd06000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.399+0000 7fd04ffff640 1 -- 172.21.15.137:0/1968722018 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060004b20 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.399+0000 7fd04ffff640 1 -- 172.21.15.137:0/1968722018 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd060004cc0 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.399+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0681a6d80 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.399+0000 7fd04ffff640 1 -- 172.21.15.137:0/1968722018 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060010760 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.399+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0681a7270 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.400+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0681031b0 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.401+0000 7fd04ffff640 1 -- 172.21.15.137:0/1968722018 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd060005020 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.401+0000 7fd04ffff640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd0500a2c10 0x7fd0500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.402+0000 7fd066ffd640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd0500a2c10 0x7fd0500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.402+0000 7fd04ffff640 1 -- 172.21.15.137:0/1968722018 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fd0600cf5c0 con 0x7fd068108300 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.402+0000 7fd066ffd640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd0500a2c10 0x7fd0500a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0580095f0 tx=0x7fd058005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:09.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.406+0000 7fd04ffff640 1 -- 172.21.15.137:0/1968722018 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd06009c570 con 0x7fd068108300 2024-09-12T12:12:09.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.591+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fd06810a290 con 0x7fd068108300 2024-09-12T12:12:09.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.591+0000 7fd04ffff640 1 -- 172.21.15.137:0/1968722018 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd06009be70 con 0x7fd068108300 2024-09-12T12:12:09.592 INFO:teuthology.orchestra.run.smithi137.stdout:240518168616 2024-09-12T12:12:09.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd0500a2c10 msgr2=0x7fd0500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:09.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd0500a2c10 0x7fd0500a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0580095f0 tx=0x7fd058005f90 comp rx=0 tx=0).stop 2024-09-12T12:12:09.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068108300 msgr2=0x7fd0681014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:09.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068108300 0x7fd0681014b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd06000eb10 tx=0x7fd06000c2d0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 shutdown_connections 2024-09-12T12:12:09.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fd0500a2c10 0x7fd0500a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd068108300 0x7fd0681014b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd068103780 0x7fd068100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.593+0000 7fd06d7bf640 1 --2- 172.21.15.137:0/1968722018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd068102dd0 0x7fd068100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:09.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.594+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 >> 172.21.15.137:0/1968722018 conn(0x7fd0680fc7d0 msgr2=0x7fd068105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:09.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.594+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 shutdown_connections 2024-09-12T12:12:09.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.594+0000 7fd06d7bf640 1 -- 172.21.15.137:0/1968722018 wait complete. 2024-09-12T12:12:09.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:09 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:09.965 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:09 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:09.965 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:09 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:09.965 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:09 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:09.965 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:09 smithi137 ceph-mon[30950]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:09.965 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:09 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2374429070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T12:12:09.965 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:09 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1968722018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T12:12:10.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:09 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:09 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:09 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:09 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:09 smithi171 ceph-mon[35098]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:10.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:09 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2374429070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T12:12:10.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:09 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1968722018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T12:12:10.330 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168614 got 240518168616 for osd.8 2024-09-12T12:12:10.330 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:10.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:09 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:09 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:09 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:09 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:10.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:09 smithi189 ceph-mon[35475]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:10.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:09 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2374429070' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T12:12:10.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:09 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1968722018' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T12:12:10.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 -- 172.21.15.137:0/1276067672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f8101b40 msgr2=0x7f08f810da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 --2- 172.21.15.137:0/1276067672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f8101b40 0x7f08f810da00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009ca0 tx=0x7f08f002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 -- 172.21.15.137:0/1276067672 shutdown_connections 2024-09-12T12:12:10.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 --2- 172.21.15.137:0/1276067672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f8101b40 0x7f08f810da00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 --2- 172.21.15.137:0/1276067672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08f81011a0 0x7f08f8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 --2- 172.21.15.137:0/1276067672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08f8106c30 0x7f08f8107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 -- 172.21.15.137:0/1276067672 >> 172.21.15.137:0/1276067672 conn(0x7f08f8077210 msgr2=0x7f08f80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:10.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 -- 172.21.15.137:0/1276067672 shutdown_connections 2024-09-12T12:12:10.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 -- 172.21.15.137:0/1276067672 wait complete. 2024-09-12T12:12:10.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.525+0000 7f08fead9640 1 Processor -- start 2024-09-12T12:12:10.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fead9640 1 -- start start 2024-09-12T12:12:10.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fead9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f81011a0 0x7f08f81973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fead9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08f8101b40 0x7f08f8197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fead9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08f8106c30 0x7f08f819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fead9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8110fa0 con 0x7f08f81011a0 2024-09-12T12:12:10.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fead9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8110e20 con 0x7f08f8106c30 2024-09-12T12:12:10.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fead9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8111120 con 0x7f08f8101b40 2024-09-12T12:12:10.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f81011a0 0x7f08f81973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f81011a0 0x7f08f81973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47076/0 (socket says 172.21.15.137:47076) 2024-09-12T12:12:10.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 -- 172.21.15.137:0/1311436202 learned_addr learned my addr 172.21.15.137:0/1311436202 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:10.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08f7fff640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08f8101b40 0x7f08f8197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fd076640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08f8106c30 0x7f08f819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 -- 172.21.15.137:0/1311436202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08f8101b40 msgr2=0x7f08f8197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08f8101b40 0x7f08f8197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 -- 172.21.15.137:0/1311436202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08f8106c30 msgr2=0x7f08f819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08f8106c30 0x7f08f819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 -- 172.21.15.137:0/1311436202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f819c390 con 0x7f08f81011a0 2024-09-12T12:12:10.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fd076640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08f8106c30 0x7f08f819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:10.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08fc875640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f81011a0 0x7f08f81973e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d950 tx=0x7f08ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:10.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.526+0000 7f08f5ffb640 1 -- 172.21.15.137:0/1311436202 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec017030 con 0x7f08f81011a0 2024-09-12T12:12:10.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.527+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f819c680 con 0x7f08f81011a0 2024-09-12T12:12:10.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.527+0000 7f08f5ffb640 1 -- 172.21.15.137:0/1311436202 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08ec004020 con 0x7f08f81011a0 2024-09-12T12:12:10.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.527+0000 7f08f5ffb640 1 -- 172.21.15.137:0/1311436202 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0047a0 con 0x7f08f81011a0 2024-09-12T12:12:10.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.527+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f81a3f60 con 0x7f08f81011a0 2024-09-12T12:12:10.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.528+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f8101600 con 0x7f08f81011a0 2024-09-12T12:12:10.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.529+0000 7f08f5ffb640 1 -- 172.21.15.137:0/1311436202 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f08ec0041d0 con 0x7f08f81011a0 2024-09-12T12:12:10.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.529+0000 7f08f5ffb640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f08e80a2c10 0x7f08e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.529+0000 7f08f7fff640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f08e80a2c10 0x7f08e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.530+0000 7f08f5ffb640 1 -- 172.21.15.137:0/1311436202 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f08ec0cea30 con 0x7f08f81011a0 2024-09-12T12:12:10.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.530+0000 7f08f7fff640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f08e80a2c10 0x7f08e80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08f8106180 tx=0x7f08e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:10.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.534+0000 7f08f5ffb640 1 -- 172.21.15.137:0/1311436202 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08ec09b960 con 0x7f08f81011a0 2024-09-12T12:12:10.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.703+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f08f81a4190 con 0x7f08f81011a0 2024-09-12T12:12:10.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.703+0000 7f08f5ffb640 1 -- 172.21.15.137:0/1311436202 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f08ec09b260 con 0x7f08f81011a0 2024-09-12T12:12:10.704 INFO:teuthology.orchestra.run.smithi137.stdout:34359738450 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f08e80a2c10 msgr2=0x7f08e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f08e80a2c10 0x7f08e80a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08f8106180 tx=0x7f08e400b390 comp rx=0 tx=0).stop 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f81011a0 msgr2=0x7f08f81973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f81011a0 0x7f08f81973e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d950 tx=0x7f08ec00de10 comp rx=0 tx=0).stop 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 shutdown_connections 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f08e80a2c10 0x7f08e80a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08f8106c30 0x7f08f819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08f8101b40 0x7f08f8197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 --2- 172.21.15.137:0/1311436202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08f81011a0 0x7f08f81973e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 >> 172.21.15.137:0/1311436202 conn(0x7f08f8077210 msgr2=0x7f08f80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:10.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 shutdown_connections 2024-09-12T12:12:10.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:10.705+0000 7f08fead9640 1 -- 172.21.15.137:0/1311436202 wait complete. 2024-09-12T12:12:10.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 -- 172.21.15.137:0/3866613754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554103670 msgr2=0x7fa55410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 --2- 172.21.15.137:0/3866613754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554103670 0x7fa55410bb60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa548009900 tx=0x7fa54802a480 comp rx=0 tx=0).stop 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 -- 172.21.15.137:0/3866613754 shutdown_connections 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 --2- 172.21.15.137:0/3866613754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa55410c180 0x7fa55410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 --2- 172.21.15.137:0/3866613754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa554103670 0x7fa55410bb60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 --2- 172.21.15.137:0/3866613754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa554102d50 0x7fa554103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 -- 172.21.15.137:0/3866613754 >> 172.21.15.137:0/3866613754 conn(0x7fa5540fc7d0 msgr2=0x7fa5540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 -- 172.21.15.137:0/3866613754 shutdown_connections 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.296+0000 7fa558e4b640 1 -- 172.21.15.137:0/3866613754 wait complete. 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 Processor -- start 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 -- start start 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa554102d50 0x7fa5541984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa554103670 0x7fa554198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410c180 0x7fa55419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa554111fc0 con 0x7fa55410c180 2024-09-12T12:12:10.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa554111e40 con 0x7fa554103670 2024-09-12T12:12:10.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa558e4b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa554112140 con 0x7fa554102d50 2024-09-12T12:12:10.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410c180 0x7fa55419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.745 INFO:teuthology.orchestra.run.smithi137.stdout:107374182465 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410c180 0x7fa55419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47038/0 (socket says 172.21.15.137:47038) 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 -- 172.21.15.137:0/2374429070 learned_addr learned my addr 172.21.15.137:0/2374429070 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa55259c640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa554102d50 0x7fa5541984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa551d9b640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa554103670 0x7fa554198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 -- 172.21.15.137:0/2374429070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa554102d50 msgr2=0x7fa5541984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa554102d50 0x7fa5541984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 -- 172.21.15.137:0/2374429070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa554103670 msgr2=0x7fa554198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa554103670 0x7fa554198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 -- 172.21.15.137:0/2374429070 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55419d470 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.297+0000 7fa552d9d640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410c180 0x7fa55419cd90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b970 tx=0x7fa54c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.298+0000 7fa5437fe640 1 -- 172.21.15.137:0/2374429070 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c019070 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.298+0000 7fa5437fe640 1 -- 172.21.15.137:0/2374429070 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa54c00cb40 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.298+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55419d760 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.298+0000 7fa5437fe640 1 -- 172.21.15.137:0/2374429070 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c007650 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.298+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5541a5040 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.299+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa554103130 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.300+0000 7fa5437fe640 1 -- 172.21.15.137:0/2374429070 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa54c01c070 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.301+0000 7fa5437fe640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa53c0a2c10 0x7fa53c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.301+0000 7fa55259c640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa53c0a2c10 0x7fa53c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.301+0000 7fa5437fe640 1 -- 172.21.15.137:0/2374429070 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7fa54c0cea60 con 0x7fa55410c180 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.301+0000 7fa55259c640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa53c0a2c10 0x7fa53c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa54400bbe0 tx=0x7fa54400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:10.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.304+0000 7fa5437fe640 1 -- 172.21.15.137:0/2374429070 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa54c09b3d0 con 0x7fa55410c180 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.485+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa5541a52e0 con 0x7fa55410c180 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.486+0000 7fa5437fe640 1 -- 172.21.15.137:0/2374429070 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa54c09f6d0 con 0x7fa55410c180 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa53c0a2c10 msgr2=0x7fa53c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa53c0a2c10 0x7fa53c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa54400bbe0 tx=0x7fa54400b390 comp rx=0 tx=0).stop 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410c180 msgr2=0x7fa55419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410c180 0x7fa55419cd90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00b970 tx=0x7fa54c00be30 comp rx=0 tx=0).stop 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 shutdown_connections 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa53c0a2c10 0x7fa53c0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa55410c180 0x7fa55419cd90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa554103670 0x7fa554198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 --2- 172.21.15.137:0/2374429070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa554102d50 0x7fa5541984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 >> 172.21.15.137:0/2374429070 conn(0x7fa5540fc7d0 msgr2=0x7fa5540fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:10.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 shutdown_connections 2024-09-12T12:12:10.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:09.488+0000 7fa558e4b640 1 -- 172.21.15.137:0/2374429070 wait complete. 2024-09-12T12:12:10.944 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:11.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:11 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1311436202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T12:12:11.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:11 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1311436202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T12:12:11.402 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182465 for osd.3 2024-09-12T12:12:11.402 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:11.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:11 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1311436202' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T12:12:12.207 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:12 smithi137 ceph-mon[30950]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T12:12:12.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:12 smithi189 ceph-mon[35475]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T12:12:12.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:12 smithi171 ceph-mon[35098]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T12:12:12.554 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738450 for osd.0 2024-09-12T12:12:12.554 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:12.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 -- 172.21.15.137:0/782605139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c101de0 msgr2=0x7f254c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:12.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 --2- 172.21.15.137:0/782605139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c101de0 0x7f254c1021c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f253c009900 tx=0x7f253c02a410 comp rx=0 tx=0).stop 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 -- 172.21.15.137:0/782605139 shutdown_connections 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 --2- 172.21.15.137:0/782605139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f254c106f10 0x7f254c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 --2- 172.21.15.137:0/782605139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f254c102790 0x7f254c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 --2- 172.21.15.137:0/782605139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c101de0 0x7f254c1021c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 -- 172.21.15.137:0/782605139 >> 172.21.15.137:0/782605139 conn(0x7f254c0fd8c0 msgr2=0x7f254c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 -- 172.21.15.137:0/782605139 shutdown_connections 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.730+0000 7f2551d56640 1 -- 172.21.15.137:0/782605139 wait complete. 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 Processor -- start 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 -- start start 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f254c101de0 0x7f254c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c102790 0x7f254c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f254c106f10 0x7f254c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f254c112010 con 0x7f254c102790 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f254c111e90 con 0x7f254c101de0 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f2551d56640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f254c112190 con 0x7f254c106f10 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f254affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c102790 0x7f254c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f254affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c102790 0x7f254c198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47084/0 (socket says 172.21.15.137:47084) 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f254affd640 1 -- 172.21.15.137:0/3017506302 learned_addr learned my addr 172.21.15.137:0/3017506302 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.731+0000 7f254bfff640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f254c106f10 0x7f254c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254b7fe640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f254c101de0 0x7f254c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254affd640 1 -- 172.21.15.137:0/3017506302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f254c106f10 msgr2=0x7f254c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254affd640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f254c106f10 0x7f254c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254affd640 1 -- 172.21.15.137:0/3017506302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f254c101de0 msgr2=0x7f254c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254affd640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f254c101de0 0x7f254c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254affd640 1 -- 172.21.15.137:0/3017506302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c19d560 con 0x7f254c102790 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254bfff640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f254c106f10 0x7f254c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254b7fe640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f254c101de0 0x7f254c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f254affd640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c102790 0x7f254c198af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f254000b4a0 tx=0x7f254000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f2548ff9640 1 -- 172.21.15.137:0/3017506302 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540014070 con 0x7f254c102790 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f2548ff9640 1 -- 172.21.15.137:0/3017506302 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540010af0 con 0x7f254c102790 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c19d850 con 0x7f254c102790 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f2548ff9640 1 -- 172.21.15.137:0/3017506302 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2540017d10 con 0x7f254c102790 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.732+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c1a5130 con 0x7f254c102790 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.733+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254c102240 con 0x7f254c102790 2024-09-12T12:12:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.734+0000 7f2548ff9640 1 -- 172.21.15.137:0/3017506302 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2540017070 con 0x7f254c102790 2024-09-12T12:12:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.735+0000 7f2548ff9640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f25340a2c10 0x7f25340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.735+0000 7f254b7fe640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f25340a2c10 0x7f25340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.735+0000 7f2548ff9640 1 -- 172.21.15.137:0/3017506302 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f25400ce390 con 0x7f254c102790 2024-09-12T12:12:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.735+0000 7f254b7fe640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f25340a2c10 0x7f25340a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f253c000c00 tx=0x7f253c032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.739+0000 7f2548ff9640 1 -- 172.21.15.137:0/3017506302 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f254009b450 con 0x7f254c102790 2024-09-12T12:12:12.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.914+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f254c02d7c0 con 0x7f254c102790 2024-09-12T12:12:12.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.915+0000 7f2548ff9640 1 -- 172.21.15.137:0/3017506302 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f254009ad50 con 0x7f254c102790 2024-09-12T12:12:12.915 INFO:teuthology.orchestra.run.smithi137.stdout:292057776158 2024-09-12T12:12:12.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f25340a2c10 msgr2=0x7f25340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:12.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f25340a2c10 0x7f25340a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f253c000c00 tx=0x7f253c032d50 comp rx=0 tx=0).stop 2024-09-12T12:12:12.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c102790 msgr2=0x7f254c198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c102790 0x7f254c198af0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f254000b4a0 tx=0x7f254000b960 comp rx=0 tx=0).stop 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 shutdown_connections 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f25340a2c10 0x7f25340a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f254c106f10 0x7f254c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f254c102790 0x7f254c198af0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 --2- 172.21.15.137:0/3017506302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f254c101de0 0x7f254c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 >> 172.21.15.137:0/3017506302 conn(0x7f254c0fd8c0 msgr2=0x7f254c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 shutdown_connections 2024-09-12T12:12:12.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:12.917+0000 7f2551d56640 1 -- 172.21.15.137:0/3017506302 wait complete. 2024-09-12T12:12:13.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:13 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3017506302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T12:12:13.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:13 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3017506302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T12:12:13.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:13 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3017506302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T12:12:13.697 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776158 for osd.10 2024-09-12T12:12:13.697 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:13.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 -- 172.21.15.137:0/848688313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 msgr2=0x7f73001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:13.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 --2- 172.21.15.137:0/848688313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73001021c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009900 tx=0x7f72fc02a410 comp rx=0 tx=0).stop 2024-09-12T12:12:13.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 -- 172.21.15.137:0/848688313 shutdown_connections 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 --2- 172.21.15.137:0/848688313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7300106f10 0x7f730010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 --2- 172.21.15.137:0/848688313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7300102790 0x7f73001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 --2- 172.21.15.137:0/848688313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73001021c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 -- 172.21.15.137:0/848688313 >> 172.21.15.137:0/848688313 conn(0x7f73000fd8c0 msgr2=0x7f73000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 -- 172.21.15.137:0/848688313 shutdown_connections 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.859+0000 7f7307902640 1 -- 172.21.15.137:0/848688313 wait complete. 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 Processor -- start 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 -- start start 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7300102790 0x7f7300074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:13.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7300106f10 0x7f7300074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7300112010 con 0x7f7300101de0 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7300111e90 con 0x7f7300106f10 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f7307902640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7300112190 con 0x7f7300102790 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47094/0 (socket says 172.21.15.137:47094) 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 -- 172.21.15.137:0/3599855977 learned_addr learned my addr 172.21.15.137:0/3599855977 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 -- 172.21.15.137:0/3599855977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7300102790 msgr2=0x7f7300074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7300102790 0x7f7300074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 -- 172.21.15.137:0/3599855977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7300106f10 msgr2=0x7f7300074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7300106f10 0x7f7300074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 -- 172.21.15.137:0/3599855977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73001a6a70 con 0x7f7300101de0 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f730569e640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73000744f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005ab0 tx=0x7f72fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:13.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.860+0000 7f72f67fc640 1 -- 172.21.15.137:0/3599855977 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc03f030 con 0x7f7300101de0 2024-09-12T12:12:13.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.861+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73001a6d00 con 0x7f7300101de0 2024-09-12T12:12:13.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.861+0000 7f72f67fc640 1 -- 172.21.15.137:0/3599855977 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72fc003ce0 con 0x7f7300101de0 2024-09-12T12:12:13.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.861+0000 7f72f67fc640 1 -- 172.21.15.137:0/3599855977 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc035070 con 0x7f7300101de0 2024-09-12T12:12:13.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.861+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73001a7190 con 0x7f7300101de0 2024-09-12T12:12:13.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.862+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7300102240 con 0x7f7300101de0 2024-09-12T12:12:13.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.863+0000 7f72f67fc640 1 -- 172.21.15.137:0/3599855977 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f72fc042030 con 0x7f7300101de0 2024-09-12T12:12:13.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.863+0000 7f72f67fc640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72ec0a2c10 0x7f72ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:13.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.863+0000 7f7304e9d640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72ec0a2c10 0x7f72ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:13.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.863+0000 7f72f67fc640 1 -- 172.21.15.137:0/3599855977 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f72fc0efae0 con 0x7f7300101de0 2024-09-12T12:12:13.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.864+0000 7f7304e9d640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72ec0a2c10 0x7f72ec0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7300075f80 tx=0x7f72f0006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:13.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:13.867+0000 7f72f67fc640 1 -- 172.21.15.137:0/3599855977 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72fc0bc990 con 0x7f7300101de0 2024-09-12T12:12:14.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.039+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7300109b60 con 0x7f7300101de0 2024-09-12T12:12:14.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.039+0000 7f72f67fc640 1 -- 172.21.15.137:0/3599855977 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f72fc0bc290 con 0x7f7300101de0 2024-09-12T12:12:14.040 INFO:teuthology.orchestra.run.smithi137.stdout:158913790008 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72ec0a2c10 msgr2=0x7f72ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72ec0a2c10 0x7f72ec0a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7300075f80 tx=0x7f72f0006c90 comp rx=0 tx=0).stop 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 msgr2=0x7f73000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73000744f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f72fc005ab0 tx=0x7f72fc0042f0 comp rx=0 tx=0).stop 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 shutdown_connections 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f72ec0a2c10 0x7f72ec0a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7300106f10 0x7f7300074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7300102790 0x7f7300074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:14.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 --2- 172.21.15.137:0/3599855977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7300101de0 0x7f73000744f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:14.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 >> 172.21.15.137:0/3599855977 conn(0x7f73000fd8c0 msgr2=0x7f73000fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:14.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 shutdown_connections 2024-09-12T12:12:14.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.041+0000 7f7307902640 1 -- 172.21.15.137:0/3599855977 wait complete. 2024-09-12T12:12:14.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:14 smithi137 ceph-mon[30950]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:14.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:14 smithi189 ceph-mon[35475]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:14.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:14 smithi171 ceph-mon[35098]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:14.754 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790004 got 158913790008 for osd.5 2024-09-12T12:12:14.754 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.132+0000 7f02f0f18640 1 -- 172.21.15.137:0/3203320961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec102ef0 msgr2=0x7f02ec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.132+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3203320961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec102ef0 0x7f02ec10da40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009ca0 tx=0x7f02e402a950 comp rx=0 tx=0).stop 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.133+0000 7f02f0f18640 1 -- 172.21.15.137:0/3203320961 shutdown_connections 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.133+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3203320961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec102ef0 0x7f02ec10da40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.133+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3203320961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02ec102570 0x7f02ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.133+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3203320961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02ec100e20 0x7f02ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.133+0000 7f02f0f18640 1 -- 172.21.15.137:0/3203320961 >> 172.21.15.137:0/3203320961 conn(0x7f02ec0fc7d0 msgr2=0x7f02ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.133+0000 7f02f0f18640 1 -- 172.21.15.137:0/3203320961 shutdown_connections 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.133+0000 7f02f0f18640 1 -- 172.21.15.137:0/3203320961 wait complete. 2024-09-12T12:12:15.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 Processor -- start 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 -- start start 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec100e20 0x7f02ec1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02ec102570 0x7f02ec1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02ec102ef0 0x7f02ec1a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec110fa0 con 0x7f02ec100e20 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec110e20 con 0x7f02ec102570 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02f0f18640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec111120 con 0x7f02ec102ef0 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec100e20 0x7f02ec1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec100e20 0x7f02ec1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47112/0 (socket says 172.21.15.137:47112) 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 -- 172.21.15.137:0/3454855993 learned_addr learned my addr 172.21.15.137:0/3454855993 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 -- 172.21.15.137:0/3454855993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02ec102ef0 msgr2=0x7f02ec1a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02ec102ef0 0x7f02ec1a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 -- 172.21.15.137:0/3454855993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02ec102570 msgr2=0x7f02ec1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02e9d9b640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02ec102570 0x7f02ec1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02ec102570 0x7f02ec1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 -- 172.21.15.137:0/3454855993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec1a4860 con 0x7f02ec100e20 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02e9d9b640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02ec102570 0x7f02ec1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.134+0000 7f02ea59c640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec100e20 0x7f02ec1992a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00c920 tx=0x7f02dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.135+0000 7f02db7fe640 1 -- 172.21.15.137:0/3454855993 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc007c70 con 0x7f02ec100e20 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.135+0000 7f02db7fe640 1 -- 172.21.15.137:0/3454855993 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02dc007e10 con 0x7f02ec100e20 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.135+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec1a4a30 con 0x7f02ec100e20 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.135+0000 7f02db7fe640 1 -- 172.21.15.137:0/3454855993 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc018890 con 0x7f02ec100e20 2024-09-12T12:12:15.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.135+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec1a4f40 con 0x7f02ec100e20 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stdout:137438953533 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.136+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec101200 con 0x7f02ec100e20 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.137+0000 7f02db7fe640 1 -- 172.21.15.137:0/3454855993 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f02dc018a50 con 0x7f02ec100e20 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.137+0000 7f02db7fe640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f02d40a2c10 0x7f02d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.138+0000 7f02e9d9b640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f02d40a2c10 0x7f02d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.138+0000 7f02db7fe640 1 -- 172.21.15.137:0/3454855993 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f02dc0cf4c0 con 0x7f02ec100e20 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.138+0000 7f02e9d9b640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f02d40a2c10 0x7f02d40a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006fd0 tx=0x7f02e0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.141+0000 7f02db7fe640 1 -- 172.21.15.137:0/3454855993 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02dc014030 con 0x7f02ec100e20 2024-09-12T12:12:15.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.310+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f02ec1a51d0 con 0x7f02ec100e20 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.310+0000 7f02db7fe640 1 -- 172.21.15.137:0/3454855993 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f02dc09c500 con 0x7f02ec100e20 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f02d40a2c10 msgr2=0x7f02d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f02d40a2c10 0x7f02d40a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f02e0006fd0 tx=0x7f02e0008040 comp rx=0 tx=0).stop 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec100e20 msgr2=0x7f02ec1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec100e20 0x7f02ec1992a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00c920 tx=0x7f02dc00cde0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 shutdown_connections 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f02d40a2c10 0x7f02d40a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02ec102ef0 0x7f02ec1a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02ec102570 0x7f02ec1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 --2- 172.21.15.137:0/3454855993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02ec100e20 0x7f02ec1992a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 >> 172.21.15.137:0/3454855993 conn(0x7f02ec0fc7d0 msgr2=0x7f02ec106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 shutdown_connections 2024-09-12T12:12:15.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:14.312+0000 7f02f0f18640 1 -- 172.21.15.137:0/3454855993 wait complete. 2024-09-12T12:12:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:15 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3599855977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T12:12:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:15 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3454855993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T12:12:15.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:15 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3599855977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T12:12:15.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:15 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3454855993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T12:12:15.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:15 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3599855977' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T12:12:15.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:15 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3454855993' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T12:12:15.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953531 got 137438953533 for osd.4 2024-09-12T12:12:15.738 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:16.171 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:16.295 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T12:12:16.296 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T12:12:16.296 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-12T12:12:16.296 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-12T12:12:16.296 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-12T12:12:16.296 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-12T12:12:16.296 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T12:12:16.296 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:16 smithi137 ceph-mon[30950]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:16.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T12:12:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T12:12:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-12T12:12:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-12T12:12:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-12T12:12:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-12T12:12:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T12:12:16.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:16 smithi189 ceph-mon[35475]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-12T12:12:16.451 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:16 smithi171 ceph-mon[35098]: pgmap v312: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 -- 172.21.15.137:0/3644857494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180106f20 msgr2=0x7f818010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 --2- 172.21.15.137:0/3644857494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180106f20 0x7f818010eaa0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8178009ca0 tx=0x7f817802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 -- 172.21.15.137:0/3644857494 shutdown_connections 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 --2- 172.21.15.137:0/3644857494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8180106f20 0x7f818010eaa0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 --2- 172.21.15.137:0/3644857494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f81801027a0 0x7f81801067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 --2- 172.21.15.137:0/3644857494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8180101df0 0x7f81801021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 -- 172.21.15.137:0/3644857494 >> 172.21.15.137:0/3644857494 conn(0x7f81800fd8b0 msgr2=0x7f81800ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 -- 172.21.15.137:0/3644857494 shutdown_connections 2024-09-12T12:12:16.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.646+0000 7f8186be5640 1 -- 172.21.15.137:0/3644857494 wait complete. 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 Processor -- start 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 -- start start 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180101df0 0x7f8180198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81801027a0 0x7f8180198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8180106f20 0x7f818019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8180111fa0 con 0x7f81801027a0 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8180111e20 con 0x7f8180106f20 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f8186be5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8180112120 con 0x7f8180101df0 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f817ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81801027a0 0x7f8180198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f817ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81801027a0 0x7f8180198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47124/0 (socket says 172.21.15.137:47124) 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.647+0000 7f817ffff640 1 -- 172.21.15.137:0/185607226 learned_addr learned my addr 172.21.15.137:0/185607226 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f8184981640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180101df0 0x7f8180198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:16.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f8185182640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8180106f20 0x7f818019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817ffff640 1 -- 172.21.15.137:0/185607226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180101df0 msgr2=0x7f8180198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817ffff640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180101df0 0x7f8180198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817ffff640 1 -- 172.21.15.137:0/185607226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8180106f20 msgr2=0x7f818019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817ffff640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8180106f20 0x7f818019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817ffff640 1 -- 172.21.15.137:0/185607226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818019d3d0 con 0x7f81801027a0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f8185182640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8180106f20 0x7f818019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817ffff640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81801027a0 0x7f8180198960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f816c00c890 tx=0x7f816c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817dffb640 1 -- 172.21.15.137:0/185607226 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c004220 con 0x7f81801027a0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817dffb640 1 -- 172.21.15.137:0/185607226 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816c0043c0 con 0x7f81801027a0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818019d6c0 con 0x7f81801027a0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f817dffb640 1 -- 172.21.15.137:0/185607226 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c0049e0 con 0x7f81801027a0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.648+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81801a4fa0 con 0x7f81801027a0 2024-09-12T12:12:16.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.649+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81801021d0 con 0x7f81801027a0 2024-09-12T12:12:16.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.650+0000 7f817dffb640 1 -- 172.21.15.137:0/185607226 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f816c0079b0 con 0x7f81801027a0 2024-09-12T12:12:16.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.651+0000 7f817dffb640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f81700a2c10 0x7f81700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:16.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.651+0000 7f8184981640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f81700a2c10 0x7f81700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:16.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.651+0000 7f817dffb640 1 -- 172.21.15.137:0/185607226 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8182+0+0 (secure 0 0 0) 0x7f816c0cf2b0 con 0x7f81801027a0 2024-09-12T12:12:16.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.651+0000 7f8184981640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f81700a2c10 0x7f81700a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8174009890 tx=0x7f8174008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:16.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.655+0000 7f817dffb640 1 -- 172.21.15.137:0/185607226 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f816c014030 con 0x7f81801027a0 2024-09-12T12:12:16.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.845+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8180199780 con 0x7f81801027a0 2024-09-12T12:12:16.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.845+0000 7f817dffb640 1 -- 172.21.15.137:0/185607226 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8180199780 con 0x7f81801027a0 2024-09-12T12:12:16.846 INFO:teuthology.orchestra.run.smithi137.stdout:210453397551 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f81700a2c10 msgr2=0x7f81700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f81700a2c10 0x7f81700a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8174009890 tx=0x7f8174008040 comp rx=0 tx=0).stop 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81801027a0 msgr2=0x7f8180198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81801027a0 0x7f8180198960 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f816c00c890 tx=0x7f816c00cd50 comp rx=0 tx=0).stop 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 shutdown_connections 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f81700a2c10 0x7f81700a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8180106f20 0x7f818019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81801027a0 0x7f8180198960 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 --2- 172.21.15.137:0/185607226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8180101df0 0x7f8180198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:16.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 >> 172.21.15.137:0/185607226 conn(0x7f81800fd8b0 msgr2=0x7f81800feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:16.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 shutdown_connections 2024-09-12T12:12:16.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:16.849+0000 7f8186be5640 1 -- 172.21.15.137:0/185607226 wait complete. 2024-09-12T12:12:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T12:12:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T12:12:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-12T12:12:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-12T12:12:17.220 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-12T12:12:17.221 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-12T12:12:17.221 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T12:12:17.221 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T12:12:17.221 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:17 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/185607226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T12:12:17.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T12:12:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:17 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/185607226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T12:12:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:17 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/185607226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T12:12:17.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397548 got 210453397551 for osd.7 2024-09-12T12:12:17.584 DEBUG:teuthology.parallel:result is None 2024-09-12T12:12:17.584 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T12:12:17.585 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:12:17.856 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:18.242 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:18 smithi137 ceph-mon[30950]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T12:12:18.242 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:18 smithi137 ceph-mon[30950]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:18.242 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:18 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:18.242 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:18 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:18 smithi189 ceph-mon[35475]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T12:12:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:18 smithi189 ceph-mon[35475]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:18 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:18.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:18 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:18.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 -- 172.21.15.137:0/1922436473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102570 msgr2=0x7f45001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 --2- 172.21.15.137:0/1922436473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102570 0x7f45001029b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009900 tx=0x7f44e802a480 comp rx=0 tx=0).stop 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 -- 172.21.15.137:0/1922436473 shutdown_connections 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 --2- 172.21.15.137:0/1922436473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4500102ef0 0x7f450010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 --2- 172.21.15.137:0/1922436473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102570 0x7f45001029b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 --2- 172.21.15.137:0/1922436473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4500100e20 0x7f4500101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 -- 172.21.15.137:0/1922436473 >> 172.21.15.137:0/1922436473 conn(0x7f45000fc7d0 msgr2=0x7f45000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 -- 172.21.15.137:0/1922436473 shutdown_connections 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.325+0000 7f450483f640 1 -- 172.21.15.137:0/1922436473 wait complete. 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 Processor -- start 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 -- start start 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4500100e20 0x7f45000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:18.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4500102570 0x7f45000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102ef0 0x7f45000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4500110f80 con 0x7f4500102ef0 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4500110e00 con 0x7f4500100e20 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f450483f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4500111100 con 0x7f4500102570 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102ef0 0x7f45000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102ef0 0x7f45000fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47144/0 (socket says 172.21.15.137:47144) 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 -- 172.21.15.137:0/2821433444 learned_addr learned my addr 172.21.15.137:0/2821433444 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fe59c640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4500100e20 0x7f45000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fdd9b640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4500102570 0x7f45000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 -- 172.21.15.137:0/2821433444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4500102570 msgr2=0x7f45000fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4500102570 0x7f45000fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 -- 172.21.15.137:0/2821433444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4500100e20 msgr2=0x7f45000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:18.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4500100e20 0x7f45000fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.326+0000 7f44fed9d640 1 -- 172.21.15.137:0/2821433444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45001a6990 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.327+0000 7f44fed9d640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102ef0 0x7f45000fd700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f44f000d750 tx=0x7f44f000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.327+0000 7f44f77fe640 1 -- 172.21.15.137:0/2821433444 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0017070 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.327+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45001a6c80 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.327+0000 7f44f77fe640 1 -- 172.21.15.137:0/2821433444 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0005500 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.327+0000 7f44f77fe640 1 -- 172.21.15.137:0/2821433444 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0004d90 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.327+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45001a71e0 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.328+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4500101200 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.329+0000 7f44f77fe640 1 -- 172.21.15.137:0/2821433444 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f44f0005020 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.330+0000 7f44f77fe640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.330+0000 7f44fe59c640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.330+0000 7f44f77fe640 1 -- 172.21.15.137:0/2821433444 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f44f00cea20 con 0x7f4500102ef0 2024-09-12T12:12:18.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.330+0000 7f44fe59c640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005fd0 tx=0x7f44f8009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:18.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.334+0000 7f44f77fe640 1 -- 172.21.15.137:0/2821433444 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44f009b200 con 0x7f4500102ef0 2024-09-12T12:12:18.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:18 smithi171 ceph-mon[35098]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T12:12:18.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:18 smithi171 ceph-mon[35098]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:18.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:18 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:18.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:18 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:18.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.501+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4500062630 con 0x7f44ec0a2c10 2024-09-12T12:12:18.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.512+0000 7f44f77fe640 1 -- 172.21.15.137:0/2821433444 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258850 (secure 0 0 0) 0x7f4500062630 con 0x7f44ec0a2c10 2024-09-12T12:12:18.516 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:18.518 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:12:18.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f44ec0a2c10 msgr2=0x7f44ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:18.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005fd0 tx=0x7f44f8009760 comp rx=0 tx=0).stop 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102ef0 msgr2=0x7f45000fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102ef0 0x7f45000fd700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f44f000d750 tx=0x7f44f000dc10 comp rx=0 tx=0).stop 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 shutdown_connections 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f44ec0a2c10 0x7f44ec0a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4500102ef0 0x7f45000fd700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4500102570 0x7f45000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.515+0000 7f450483f640 1 --2- 172.21.15.137:0/2821433444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4500100e20 0x7f45000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.516+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 >> 172.21.15.137:0/2821433444 conn(0x7f45000fc7d0 msgr2=0x7f450006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.516+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 shutdown_connections 2024-09-12T12:12:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:18.516+0000 7f450483f640 1 -- 172.21.15.137:0/2821433444 wait complete. 2024-09-12T12:12:19.238 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-12T12:12:17.124629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653816,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":476,"num_read_kb":354,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355984,"kb_used_data":10896,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585840,"statfs":{"total":1151940427776,"available":1151575900160,"internally_reserved":0,"allocated":11157504,"data_stored":7080696,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":20,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001843"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491849+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:11:13.491849+0000","last_peered":"2024-09-12T12:11:13.491849+0000","last_clean":"2024-09-12T12:11:13.491849+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:11:13.491849+0000","last_undegraded":"2024-09-12T12:11:13.491849+0000","last_fullsized":"2024-09-12T12:11:13.491849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500296+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:11:13.500296+0000","last_peered":"2024-09-12T12:11:13.500296+0000","last_clean":"2024-09-12T12:11:13.500296+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:11:13.500296+0000","last_undegraded":"2024-09-12T12:11:13.500296+0000","last_fullsized":"2024-09-12T12:11:13.500296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500541+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:11:13.500541+0000","last_peered":"2024-09-12T12:11:13.500541+0000","last_clean":"2024-09-12T12:11:13.500541+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:11:13.500541+0000","last_undegraded":"2024-09-12T12:11:13.500541+0000","last_fullsized":"2024-09-12T12:11:13.500541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492536+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:11:13.492536+0000","last_peered":"2024-09-12T12:11:13.492536+0000","last_clean":"2024-09-12T12:11:13.492536+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:11:13.492536+0000","last_undegraded":"2024-09-12T12:11:13.492536+0000","last_fullsized":"2024-09-12T12:11:13.492536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501335+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:11:13.501335+0000","last_peered":"2024-09-12T12:11:13.501335+0000","last_clean":"2024-09-12T12:11:13.501335+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:11:13.501335+0000","last_undegraded":"2024-09-12T12:11:13.501335+0000","last_fullsized":"2024-09-12T12:11:13.501335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500893+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:11:13.500893+0000","last_peered":"2024-09-12T12:11:13.500893+0000","last_clean":"2024-09-12T12:11:13.500893+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:11:13.500893+0000","last_undegraded":"2024-09-12T12:11:13.500893+0000","last_fullsized":"2024-09-12T12:11:13.500893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503506+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:11:13.503506+0000","last_peered":"2024-09-12T12:11:13.503506+0000","last_clean":"2024-09-12T12:11:13.503506+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:11:13.503506+0000","last_undegraded":"2024-09-12T12:11:13.503506+0000","last_fullsized":"2024-09-12T12:11:13.503506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498001+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:11:13.498001+0000","last_peered":"2024-09-12T12:11:13.498001+0000","last_clean":"2024-09-12T12:11:13.498001+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:11:13.498001+0000","last_undegraded":"2024-09-12T12:11:13.498001+0000","last_fullsized":"2024-09-12T12:11:13.498001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503514+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:11:13.503514+0000","last_peered":"2024-09-12T12:11:13.503514+0000","last_clean":"2024-09-12T12:11:13.503514+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:11:13.503514+0000","last_undegraded":"2024-09-12T12:11:13.503514+0000","last_fullsized":"2024-09-12T12:11:13.503514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501033+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:11:13.501033+0000","last_peered":"2024-09-12T12:11:13.501033+0000","last_clean":"2024-09-12T12:11:13.501033+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:11:13.501033+0000","last_undegraded":"2024-09-12T12:11:13.501033+0000","last_fullsized":"2024-09-12T12:11:13.501033+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504240+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:11:13.504240+0000","last_peered":"2024-09-12T12:11:13.504240+0000","last_clean":"2024-09-12T12:11:13.504240+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:11:13.504240+0000","last_undegraded":"2024-09-12T12:11:13.504240+0000","last_fullsized":"2024-09-12T12:11:13.504240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492580+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:11:13.492580+0000","last_peered":"2024-09-12T12:11:13.492580+0000","last_clean":"2024-09-12T12:11:13.492580+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:11:13.492580+0000","last_undegraded":"2024-09-12T12:11:13.492580+0000","last_fullsized":"2024-09-12T12:11:13.492580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491145+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:11:13.491145+0000","last_peered":"2024-09-12T12:11:13.491145+0000","last_clean":"2024-09-12T12:11:13.491145+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:11:13.491145+0000","last_undegraded":"2024-09-12T12:11:13.491145+0000","last_fullsized":"2024-09-12T12:11:13.491145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500084+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:11:13.500084+0000","last_peered":"2024-09-12T12:11:13.500084+0000","last_clean":"2024-09-12T12:11:13.500084+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:11:13.500084+0000","last_undegraded":"2024-09-12T12:11:13.500084+0000","last_fullsized":"2024-09-12T12:11:13.500084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.499972+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:11:13.499972+0000","last_peered":"2024-09-12T12:11:13.499972+0000","last_clean":"2024-09-12T12:11:13.499972+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:11:13.499972+0000","last_undegraded":"2024-09-12T12:11:13.499972+0000","last_fullsized":"2024-09-12T12:11:13.499972+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502792+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:11:13.502792+0000","last_peered":"2024-09-12T12:11:13.502792+0000","last_clean":"2024-09-12T12:11:13.502792+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:11:13.502792+0000","last_undegraded":"2024-09-12T12:11:13.502792+0000","last_fullsized":"2024-09-12T12:11:13.502792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491907+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:11:13.491907+0000","last_peered":"2024-09-12T12:11:13.491907+0000","last_clean":"2024-09-12T12:11:13.491907+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:11:13.491907+0000","last_undegraded":"2024-09-12T12:11:13.491907+0000","last_fullsized":"2024-09-12T12:11:13.491907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502053+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:11:13.502053+0000","last_peered":"2024-09-12T12:11:13.502053+0000","last_clean":"2024-09-12T12:11:13.502053+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:11:13.502053+0000","last_undegraded":"2024-09-12T12:11:13.502053+0000","last_fullsized":"2024-09-12T12:11:13.502053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498689+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:11:13.498689+0000","last_peered":"2024-09-12T12:11:13.498689+0000","last_clean":"2024-09-12T12:11:13.498689+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:11:13.498689+0000","last_undegraded":"2024-09-12T12:11:13.498689+0000","last_fullsized":"2024-09-12T12:11:13.498689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491690+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:11:13.491690+0000","last_peered":"2024-09-12T12:11:13.491690+0000","last_clean":"2024-09-12T12:11:13.491690+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:11:13.491690+0000","last_undegraded":"2024-09-12T12:11:13.491690+0000","last_fullsized":"2024-09-12T12:11:13.491690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491185+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:11:13.491185+0000","last_peered":"2024-09-12T12:11:13.491185+0000","last_clean":"2024-09-12T12:11:13.491185+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:11:13.491185+0000","last_undegraded":"2024-09-12T12:11:13.491185+0000","last_fullsized":"2024-09-12T12:11:13.491185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503390+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:11:13.503390+0000","last_peered":"2024-09-12T12:11:13.503390+0000","last_clean":"2024-09-12T12:11:13.503390+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:11:13.503390+0000","last_undegraded":"2024-09-12T12:11:13.503390+0000","last_fullsized":"2024-09-12T12:11:13.503390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497730+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:11:13.497730+0000","last_peered":"2024-09-12T12:11:13.497730+0000","last_clean":"2024-09-12T12:11:13.497730+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:11:13.497730+0000","last_undegraded":"2024-09-12T12:11:13.497730+0000","last_fullsized":"2024-09-12T12:11:13.497730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031314+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:12:16.031314+0000","last_peered":"2024-09-12T12:12:16.031314+0000","last_clean":"2024-09-12T12:12:16.031314+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:12:16.031314+0000","last_undegraded":"2024-09-12T12:12:16.031314+0000","last_fullsized":"2024-09-12T12:12:16.031314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498394+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:11:13.498394+0000","last_peered":"2024-09-12T12:11:13.498394+0000","last_clean":"2024-09-12T12:11:13.498394+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:11:13.498394+0000","last_undegraded":"2024-09-12T12:11:13.498394+0000","last_fullsized":"2024-09-12T12:11:13.498394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498501+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:11:13.498501+0000","last_peered":"2024-09-12T12:11:13.498501+0000","last_clean":"2024-09-12T12:11:13.498501+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:11:13.498501+0000","last_undegraded":"2024-09-12T12:11:13.498501+0000","last_fullsized":"2024-09-12T12:11:13.498501+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491855+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:11:13.491855+0000","last_peered":"2024-09-12T12:11:13.491855+0000","last_clean":"2024-09-12T12:11:13.491855+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:11:13.491855+0000","last_undegraded":"2024-09-12T12:11:13.491855+0000","last_fullsized":"2024-09-12T12:11:13.491855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491115+0000","last_change":"2024-09-12T12:11:12.500411+0000","last_active":"2024-09-12T12:11:13.491115+0000","last_peered":"2024-09-12T12:11:13.491115+0000","last_clean":"2024-09-12T12:11:13.491115+0000","last_became_active":"2024-09-12T12:11:12.500126+0000","last_became_peered":"2024-09-12T12:11:12.500126+0000","last_unstale":"2024-09-12T12:11:13.491115+0000","last_undegraded":"2024-09-12T12:11:13.491115+0000","last_fullsized":"2024-09-12T12:11:13.491115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:23:49.072306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500514+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:11:13.500514+0000","last_peered":"2024-09-12T12:11:13.500514+0000","last_clean":"2024-09-12T12:11:13.500514+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:11:13.500514+0000","last_undegraded":"2024-09-12T12:11:13.500514+0000","last_fullsized":"2024-09-12T12:11:13.500514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498074+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:11:13.498074+0000","last_peered":"2024-09-12T12:11:13.498074+0000","last_clean":"2024-09-12T12:11:13.498074+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:11:13.498074+0000","last_undegraded":"2024-09-12T12:11:13.498074+0000","last_fullsized":"2024-09-12T12:11:13.498074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498204+0000","last_change":"2024-09-12T12:11:12.510232+0000","last_active":"2024-09-12T12:11:13.498204+0000","last_peered":"2024-09-12T12:11:13.498204+0000","last_clean":"2024-09-12T12:11:13.498204+0000","last_became_active":"2024-09-12T12:11:12.509856+0000","last_became_peered":"2024-09-12T12:11:12.509856+0000","last_unstale":"2024-09-12T12:11:13.498204+0000","last_undegraded":"2024-09-12T12:11:13.498204+0000","last_fullsized":"2024-09-12T12:11:13.498204+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:07:56.293960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491659+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:11:13.491659+0000","last_peered":"2024-09-12T12:11:13.491659+0000","last_clean":"2024-09-12T12:11:13.491659+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:11:13.491659+0000","last_undegraded":"2024-09-12T12:11:13.491659+0000","last_fullsized":"2024-09-12T12:11:13.491659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501135+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:11:13.501135+0000","last_peered":"2024-09-12T12:11:13.501135+0000","last_clean":"2024-09-12T12:11:13.501135+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:11:13.501135+0000","last_undegraded":"2024-09-12T12:11:13.501135+0000","last_fullsized":"2024-09-12T12:11:13.501135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032009+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:12:16.032009+0000","last_peered":"2024-09-12T12:12:16.032009+0000","last_clean":"2024-09-12T12:12:16.032009+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:12:16.032009+0000","last_undegraded":"2024-09-12T12:12:16.032009+0000","last_fullsized":"2024-09-12T12:12:16.032009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-12T12:11:16.259126+0000","last_change":"2024-09-12T12:12:16.032862+0000","last_active":"2024-09-12T12:12:16.032862+0000","last_peered":"2024-09-12T12:11:16.259126+0000","last_clean":"2024-09-12T12:11:16.259126+0000","last_became_active":"2024-09-12T12:11:12.507860+0000","last_became_peered":"2024-09-12T12:11:12.507860+0000","last_unstale":"2024-09-12T12:11:16.259126+0000","last_undegraded":"2024-09-12T12:11:16.259126+0000","last_fullsized":"2024-09-12T12:11:16.259126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556617+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:11:21.556617+0000","last_peered":"2024-09-12T12:11:21.556617+0000","last_clean":"2024-09-12T12:11:21.556617+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:11:21.556617+0000","last_undegraded":"2024-09-12T12:11:21.556617+0000","last_fullsized":"2024-09-12T12:11:21.556617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556515+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:11:21.556515+0000","last_peered":"2024-09-12T12:11:21.556515+0000","last_clean":"2024-09-12T12:11:21.556515+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:11:21.556515+0000","last_undegraded":"2024-09-12T12:11:21.556515+0000","last_fullsized":"2024-09-12T12:11:21.556515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497911+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:11:13.497911+0000","last_peered":"2024-09-12T12:11:13.497911+0000","last_clean":"2024-09-12T12:11:13.497911+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:11:13.497911+0000","last_undegraded":"2024-09-12T12:11:13.497911+0000","last_fullsized":"2024-09-12T12:11:13.497911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032731+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:12:16.032731+0000","last_peered":"2024-09-12T12:12:16.032731+0000","last_clean":"2024-09-12T12:12:16.032731+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:12:16.032731+0000","last_undegraded":"2024-09-12T12:12:16.032731+0000","last_fullsized":"2024-09-12T12:12:16.032731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501931+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:11:13.501931+0000","last_peered":"2024-09-12T12:11:13.501931+0000","last_clean":"2024-09-12T12:11:13.501931+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:11:13.501931+0000","last_undegraded":"2024-09-12T12:11:13.501931+0000","last_fullsized":"2024-09-12T12:11:13.501931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":74,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:12:11.535976+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:12:11.535976+0000","last_peered":"2024-09-12T12:12:11.535976+0000","last_clean":"2024-09-12T12:12:11.535976+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:12:11.535976+0000","last_undegraded":"2024-09-12T12:12:11.535976+0000","last_fullsized":"2024-09-12T12:12:11.535976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503842+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:11:13.503842+0000","last_peered":"2024-09-12T12:11:13.503842+0000","last_clean":"2024-09-12T12:11:13.503842+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:11:13.503842+0000","last_undegraded":"2024-09-12T12:11:13.503842+0000","last_fullsized":"2024-09-12T12:11:13.503842+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.543897+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:11:20.543897+0000","last_peered":"2024-09-12T12:11:20.543897+0000","last_clean":"2024-09-12T12:11:20.543897+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:11:20.543897+0000","last_undegraded":"2024-09-12T12:11:20.543897+0000","last_fullsized":"2024-09-12T12:11:20.543897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500509+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:11:13.500509+0000","last_peered":"2024-09-12T12:11:13.500509+0000","last_clean":"2024-09-12T12:11:13.500509+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:11:13.500509+0000","last_undegraded":"2024-09-12T12:11:13.500509+0000","last_fullsized":"2024-09-12T12:11:13.500509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032201+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:12:16.032201+0000","last_peered":"2024-09-12T12:12:16.032201+0000","last_clean":"2024-09-12T12:12:16.032201+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:12:16.032201+0000","last_undegraded":"2024-09-12T12:12:16.032201+0000","last_fullsized":"2024-09-12T12:12:16.032201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503678+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:11:13.503678+0000","last_peered":"2024-09-12T12:11:13.503678+0000","last_clean":"2024-09-12T12:11:13.503678+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:11:13.503678+0000","last_undegraded":"2024-09-12T12:11:13.503678+0000","last_fullsized":"2024-09-12T12:11:13.503678+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":156,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:12:12.537449+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:12:12.537449+0000","last_peered":"2024-09-12T12:12:12.537449+0000","last_clean":"2024-09-12T12:12:12.537449+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:12:12.537449+0000","last_undegraded":"2024-09-12T12:12:12.537449+0000","last_fullsized":"2024-09-12T12:12:12.537449+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":110,"num_read_kb":112,"num_write":27,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503335+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:11:13.503335+0000","last_peered":"2024-09-12T12:11:13.503335+0000","last_clean":"2024-09-12T12:11:13.503335+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:11:13.503335+0000","last_undegraded":"2024-09-12T12:11:13.503335+0000","last_fullsized":"2024-09-12T12:11:13.503335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557533+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:11:21.557533+0000","last_peered":"2024-09-12T12:11:21.557533+0000","last_clean":"2024-09-12T12:11:21.557533+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:11:21.557533+0000","last_undegraded":"2024-09-12T12:11:21.557533+0000","last_fullsized":"2024-09-12T12:11:21.557533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490815+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:11:13.490815+0000","last_peered":"2024-09-12T12:11:13.490815+0000","last_clean":"2024-09-12T12:11:13.490815+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:11:13.490815+0000","last_undegraded":"2024-09-12T12:11:13.490815+0000","last_fullsized":"2024-09-12T12:11:13.490815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031888+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:12:16.031888+0000","last_peered":"2024-09-12T12:12:16.031888+0000","last_clean":"2024-09-12T12:12:16.031888+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:12:16.031888+0000","last_undegraded":"2024-09-12T12:12:16.031888+0000","last_fullsized":"2024-09-12T12:12:16.031888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503539+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:11:13.503539+0000","last_peered":"2024-09-12T12:11:13.503539+0000","last_clean":"2024-09-12T12:11:13.503539+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:11:13.503539+0000","last_undegraded":"2024-09-12T12:11:13.503539+0000","last_fullsized":"2024-09-12T12:11:13.503539+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:12:13.539328+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:12:13.539328+0000","last_peered":"2024-09-12T12:12:13.539328+0000","last_clean":"2024-09-12T12:12:13.539328+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:12:13.539328+0000","last_undegraded":"2024-09-12T12:12:13.539328+0000","last_fullsized":"2024-09-12T12:12:13.539328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503667+0000","last_change":"2024-09-12T12:11:12.504047+0000","last_active":"2024-09-12T12:11:13.503667+0000","last_peered":"2024-09-12T12:11:13.503667+0000","last_clean":"2024-09-12T12:11:13.503667+0000","last_became_active":"2024-09-12T12:11:12.503648+0000","last_became_peered":"2024-09-12T12:11:12.503648+0000","last_unstale":"2024-09-12T12:11:13.503667+0000","last_undegraded":"2024-09-12T12:11:13.503667+0000","last_fullsized":"2024-09-12T12:11:13.503667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:43.443802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.558326+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:11:21.558326+0000","last_peered":"2024-09-12T12:11:21.558326+0000","last_clean":"2024-09-12T12:11:21.558326+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:11:21.558326+0000","last_undegraded":"2024-09-12T12:11:21.558326+0000","last_fullsized":"2024-09-12T12:11:21.558326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032581+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:12:16.032581+0000","last_peered":"2024-09-12T12:12:16.032581+0000","last_clean":"2024-09-12T12:12:16.032581+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:12:16.032581+0000","last_undegraded":"2024-09-12T12:12:16.032581+0000","last_fullsized":"2024-09-12T12:12:16.032581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:22.466026+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:11:22.466026+0000","last_peered":"2024-09-12T12:11:22.466026+0000","last_clean":"2024-09-12T12:11:22.466026+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:11:22.466026+0000","last_undegraded":"2024-09-12T12:11:22.466026+0000","last_fullsized":"2024-09-12T12:11:22.466026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491726+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:11:13.491726+0000","last_peered":"2024-09-12T12:11:13.491726+0000","last_clean":"2024-09-12T12:11:13.491726+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:11:13.491726+0000","last_undegraded":"2024-09-12T12:11:13.491726+0000","last_fullsized":"2024-09-12T12:11:13.491726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497437+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:11:13.497437+0000","last_peered":"2024-09-12T12:11:13.497437+0000","last_clean":"2024-09-12T12:11:13.497437+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:11:13.497437+0000","last_undegraded":"2024-09-12T12:11:13.497437+0000","last_fullsized":"2024-09-12T12:11:13.497437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504243+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:11:13.504243+0000","last_peered":"2024-09-12T12:11:13.504243+0000","last_clean":"2024-09-12T12:11:13.504243+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:11:13.504243+0000","last_undegraded":"2024-09-12T12:11:13.504243+0000","last_fullsized":"2024-09-12T12:11:13.504243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032086+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:12:16.032086+0000","last_peered":"2024-09-12T12:12:16.032086+0000","last_clean":"2024-09-12T12:12:16.032086+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:12:16.032086+0000","last_undegraded":"2024-09-12T12:12:16.032086+0000","last_fullsized":"2024-09-12T12:12:16.032086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503836+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:11:13.503836+0000","last_peered":"2024-09-12T12:11:13.503836+0000","last_clean":"2024-09-12T12:11:13.503836+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:11:13.503836+0000","last_undegraded":"2024-09-12T12:11:13.503836+0000","last_fullsized":"2024-09-12T12:11:13.503836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557550+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:11:21.557550+0000","last_peered":"2024-09-12T12:11:21.557550+0000","last_clean":"2024-09-12T12:11:21.557550+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:11:21.557550+0000","last_undegraded":"2024-09-12T12:11:21.557550+0000","last_fullsized":"2024-09-12T12:11:21.557550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.505016+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:11:13.505016+0000","last_peered":"2024-09-12T12:11:13.505016+0000","last_clean":"2024-09-12T12:11:13.505016+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:11:13.505016+0000","last_undegraded":"2024-09-12T12:11:13.505016+0000","last_fullsized":"2024-09-12T12:11:13.505016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":343,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503794+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:11:13.503794+0000","last_peered":"2024-09-12T12:11:13.503794+0000","last_clean":"2024-09-12T12:11:13.503794+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:11:13.503794+0000","last_undegraded":"2024-09-12T12:11:13.503794+0000","last_fullsized":"2024-09-12T12:11:13.503794+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:22.465059+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:11:22.465059+0000","last_peered":"2024-09-12T12:11:22.465059+0000","last_clean":"2024-09-12T12:11:22.465059+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:11:22.465059+0000","last_undegraded":"2024-09-12T12:11:22.465059+0000","last_fullsized":"2024-09-12T12:11:22.465059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498114+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:11:13.498114+0000","last_peered":"2024-09-12T12:11:13.498114+0000","last_clean":"2024-09-12T12:11:13.498114+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:11:13.498114+0000","last_undegraded":"2024-09-12T12:11:13.498114+0000","last_fullsized":"2024-09-12T12:11:13.498114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556747+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:11:21.556747+0000","last_peered":"2024-09-12T12:11:21.556747+0000","last_clean":"2024-09-12T12:11:21.556747+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:11:21.556747+0000","last_undegraded":"2024-09-12T12:11:21.556747+0000","last_fullsized":"2024-09-12T12:11:21.556747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490784+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:11:13.490784+0000","last_peered":"2024-09-12T12:11:13.490784+0000","last_clean":"2024-09-12T12:11:13.490784+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:11:13.490784+0000","last_undegraded":"2024-09-12T12:11:13.490784+0000","last_fullsized":"2024-09-12T12:11:13.490784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:12:13.538767+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:12:13.538767+0000","last_peered":"2024-09-12T12:12:13.538767+0000","last_clean":"2024-09-12T12:12:13.538767+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:12:13.538767+0000","last_undegraded":"2024-09-12T12:12:13.538767+0000","last_fullsized":"2024-09-12T12:12:13.538767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031412+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:12:16.031412+0000","last_peered":"2024-09-12T12:12:16.031412+0000","last_clean":"2024-09-12T12:12:16.031412+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:12:16.031412+0000","last_undegraded":"2024-09-12T12:12:16.031412+0000","last_fullsized":"2024-09-12T12:12:16.031412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504416+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:11:13.504416+0000","last_peered":"2024-09-12T12:11:13.504416+0000","last_clean":"2024-09-12T12:11:13.504416+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:11:13.504416+0000","last_undegraded":"2024-09-12T12:11:13.504416+0000","last_fullsized":"2024-09-12T12:11:13.504416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500974+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:11:13.500974+0000","last_peered":"2024-09-12T12:11:13.500974+0000","last_clean":"2024-09-12T12:11:13.500974+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:11:13.500974+0000","last_undegraded":"2024-09-12T12:11:13.500974+0000","last_fullsized":"2024-09-12T12:11:13.500974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501897+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:11:13.501897+0000","last_peered":"2024-09-12T12:11:13.501897+0000","last_clean":"2024-09-12T12:11:13.501897+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:11:13.501897+0000","last_undegraded":"2024-09-12T12:11:13.501897+0000","last_fullsized":"2024-09-12T12:11:13.501897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500355+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:11:13.500355+0000","last_peered":"2024-09-12T12:11:13.500355+0000","last_clean":"2024-09-12T12:11:13.500355+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:11:13.500355+0000","last_undegraded":"2024-09-12T12:11:13.500355+0000","last_fullsized":"2024-09-12T12:11:13.500355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.544273+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:11:20.544273+0000","last_peered":"2024-09-12T12:11:20.544273+0000","last_clean":"2024-09-12T12:11:20.544273+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:11:20.544273+0000","last_undegraded":"2024-09-12T12:11:20.544273+0000","last_fullsized":"2024-09-12T12:11:20.544273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491176+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:11:13.491176+0000","last_peered":"2024-09-12T12:11:13.491176+0000","last_clean":"2024-09-12T12:11:13.491176+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:11:13.491176+0000","last_undegraded":"2024-09-12T12:11:13.491176+0000","last_fullsized":"2024-09-12T12:11:13.491176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:37.499854+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:11:37.499854+0000","last_peered":"2024-09-12T12:11:37.499854+0000","last_clean":"2024-09-12T12:11:37.499854+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:11:37.499854+0000","last_undegraded":"2024-09-12T12:11:37.499854+0000","last_fullsized":"2024-09-12T12:11:37.499854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498306+0000","last_change":"2024-09-12T12:11:12.506733+0000","last_active":"2024-09-12T12:11:13.498306+0000","last_peered":"2024-09-12T12:11:13.498306+0000","last_clean":"2024-09-12T12:11:13.498306+0000","last_became_active":"2024-09-12T12:11:12.506447+0000","last_became_peered":"2024-09-12T12:11:12.506447+0000","last_unstale":"2024-09-12T12:11:13.498306+0000","last_undegraded":"2024-09-12T12:11:13.498306+0000","last_fullsized":"2024-09-12T12:11:13.498306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:36.338767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.544999+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:11:20.544999+0000","last_peered":"2024-09-12T12:11:20.544999+0000","last_clean":"2024-09-12T12:11:20.544999+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:11:20.544999+0000","last_undegraded":"2024-09-12T12:11:20.544999+0000","last_fullsized":"2024-09-12T12:11:20.544999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500465+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:11:13.500465+0000","last_peered":"2024-09-12T12:11:13.500465+0000","last_clean":"2024-09-12T12:11:13.500465+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:11:13.500465+0000","last_undegraded":"2024-09-12T12:11:13.500465+0000","last_fullsized":"2024-09-12T12:11:13.500465+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545274+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:11:20.545274+0000","last_peered":"2024-09-12T12:11:20.545274+0000","last_clean":"2024-09-12T12:11:20.545274+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:11:20.545274+0000","last_undegraded":"2024-09-12T12:11:20.545274+0000","last_fullsized":"2024-09-12T12:11:20.545274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502756+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:11:13.502756+0000","last_peered":"2024-09-12T12:11:13.502756+0000","last_clean":"2024-09-12T12:11:13.502756+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:11:13.502756+0000","last_undegraded":"2024-09-12T12:11:13.502756+0000","last_fullsized":"2024-09-12T12:11:13.502756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545237+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:11:20.545237+0000","last_peered":"2024-09-12T12:11:20.545237+0000","last_clean":"2024-09-12T12:11:20.545237+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:11:20.545237+0000","last_undegraded":"2024-09-12T12:11:20.545237+0000","last_fullsized":"2024-09-12T12:11:20.545237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491769+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:11:13.491769+0000","last_peered":"2024-09-12T12:11:13.491769+0000","last_clean":"2024-09-12T12:11:13.491769+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:11:13.491769+0000","last_undegraded":"2024-09-12T12:11:13.491769+0000","last_fullsized":"2024-09-12T12:11:13.491769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498164+0000","last_change":"2024-09-12T12:11:09.477481+0000","last_active":"2024-09-12T12:11:13.498164+0000","last_peered":"2024-09-12T12:11:13.498164+0000","last_clean":"2024-09-12T12:11:13.498164+0000","last_became_active":"2024-09-12T12:11:09.477126+0000","last_became_peered":"2024-09-12T12:11:09.477126+0000","last_unstale":"2024-09-12T12:11:13.498164+0000","last_undegraded":"2024-09-12T12:11:13.498164+0000","last_fullsized":"2024-09-12T12:11:13.498164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:57.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497665+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:11:13.497665+0000","last_peered":"2024-09-12T12:11:13.497665+0000","last_clean":"2024-09-12T12:11:13.497665+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:11:13.497665+0000","last_undegraded":"2024-09-12T12:11:13.497665+0000","last_fullsized":"2024-09-12T12:11:13.497665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545706+0000","last_change":"2024-09-12T12:11:09.484682+0000","last_active":"2024-09-12T12:11:20.545706+0000","last_peered":"2024-09-12T12:11:20.545706+0000","last_clean":"2024-09-12T12:11:20.545706+0000","last_became_active":"2024-09-12T12:11:09.484294+0000","last_became_peered":"2024-09-12T12:11:09.484294+0000","last_unstale":"2024-09-12T12:11:20.545706+0000","last_undegraded":"2024-09-12T12:11:20.545706+0000","last_fullsized":"2024-09-12T12:11:20.545706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:01:39.731705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.504162+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:11:13.504162+0000","last_peered":"2024-09-12T12:11:13.504162+0000","last_clean":"2024-09-12T12:11:13.504162+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:11:13.504162+0000","last_undegraded":"2024-09-12T12:11:13.504162+0000","last_fullsized":"2024-09-12T12:11:13.504162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031761+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:12:16.031761+0000","last_peered":"2024-09-12T12:12:16.031761+0000","last_clean":"2024-09-12T12:12:16.031761+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:12:16.031761+0000","last_undegraded":"2024-09-12T12:12:16.031761+0000","last_fullsized":"2024-09-12T12:12:16.031761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.507254+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:11:13.507254+0000","last_peered":"2024-09-12T12:11:13.507254+0000","last_clean":"2024-09-12T12:11:13.507254+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:11:13.507254+0000","last_undegraded":"2024-09-12T12:11:13.507254+0000","last_fullsized":"2024-09-12T12:11:13.507254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.503536+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:11:13.503536+0000","last_peered":"2024-09-12T12:11:13.503536+0000","last_clean":"2024-09-12T12:11:13.503536+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:11:13.503536+0000","last_undegraded":"2024-09-12T12:11:13.503536+0000","last_fullsized":"2024-09-12T12:11:13.503536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557696+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:11:21.557696+0000","last_peered":"2024-09-12T12:11:21.557696+0000","last_clean":"2024-09-12T12:11:21.557696+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:11:21.557696+0000","last_undegraded":"2024-09-12T12:11:21.557696+0000","last_fullsized":"2024-09-12T12:11:21.557696+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.500538+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:11:13.500538+0000","last_peered":"2024-09-12T12:11:13.500538+0000","last_clean":"2024-09-12T12:11:13.500538+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:11:13.500538+0000","last_undegraded":"2024-09-12T12:11:13.500538+0000","last_fullsized":"2024-09-12T12:11:13.500538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.499817+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:11:13.499817+0000","last_peered":"2024-09-12T12:11:13.499817+0000","last_clean":"2024-09-12T12:11:13.499817+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:11:13.499817+0000","last_undegraded":"2024-09-12T12:11:13.499817+0000","last_fullsized":"2024-09-12T12:11:13.499817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498213+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:11:13.498213+0000","last_peered":"2024-09-12T12:11:13.498213+0000","last_clean":"2024-09-12T12:11:13.498213+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:11:13.498213+0000","last_undegraded":"2024-09-12T12:11:13.498213+0000","last_fullsized":"2024-09-12T12:11:13.498213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557140+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:11:21.557140+0000","last_peered":"2024-09-12T12:11:21.557140+0000","last_clean":"2024-09-12T12:11:21.557140+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:11:21.557140+0000","last_undegraded":"2024-09-12T12:11:21.557140+0000","last_fullsized":"2024-09-12T12:11:21.557140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":340,"num_read_kb":235,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":427186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1078271,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":427342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1019274,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":426811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1018565,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":427122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":487165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":485815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":428082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:12:19.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:12:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:19 smithi137 ceph-mon[30950]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T12:12:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:12:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:12:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:19 smithi137 ceph-mon[30950]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:19 smithi189 ceph-mon[35475]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T12:12:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:12:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:12:19.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:19 smithi189 ceph-mon[35475]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:19.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:19 smithi171 ceph-mon[35098]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T12:12:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:12:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:12:19.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:19 smithi171 ceph-mon[35098]: from='mgr.14150 172.21.15.137:0/241726393' entity='mgr.x' 2024-09-12T12:12:19.504 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:20.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 -- 172.21.15.137:0/2166096316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874102790 msgr2=0x7f98741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:20.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 --2- 172.21.15.137:0/2166096316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874102790 0x7f98741067e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f986402a410 comp rx=0 tx=0).stop 2024-09-12T12:12:20.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 -- 172.21.15.137:0/2166096316 shutdown_connections 2024-09-12T12:12:20.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 --2- 172.21.15.137:0/2166096316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874106f10 0x7f987410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 --2- 172.21.15.137:0/2166096316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874102790 0x7f98741067e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 --2- 172.21.15.137:0/2166096316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9874101de0 0x7f98741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 -- 172.21.15.137:0/2166096316 >> 172.21.15.137:0/2166096316 conn(0x7f98740fd8c0 msgr2=0x7f98740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 -- 172.21.15.137:0/2166096316 shutdown_connections 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 -- 172.21.15.137:0/2166096316 wait complete. 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 Processor -- start 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 -- start start 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.023+0000 7f987c386640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874101de0 0x7f98741985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987c386640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874102790 0x7f9874198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987c386640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9874106f10 0x7f987419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987c386640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9874112010 con 0x7f9874101de0 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987c386640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9874111e90 con 0x7f9874106f10 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987c386640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9874112190 con 0x7f9874102790 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874101de0 0x7f98741985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874101de0 0x7f98741985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56698/0 (socket says 172.21.15.137:56698) 2024-09-12T12:12:20.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 -- 172.21.15.137:0/2674570330 learned_addr learned my addr 172.21.15.137:0/2674570330 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a923640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9874106f10 0x7f987419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f9879921640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874102790 0x7f9874198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 -- 172.21.15.137:0/2674570330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874102790 msgr2=0x7f9874198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874102790 0x7f9874198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 -- 172.21.15.137:0/2674570330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9874106f10 msgr2=0x7f987419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9874106f10 0x7f987419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 -- 172.21.15.137:0/2674570330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987419d560 con 0x7f9874101de0 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f9879921640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874102790 0x7f9874198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987a122640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874101de0 0x7f98741985b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f987000b4a0 tx=0x7f987000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f986b7fe640 1 -- 172.21.15.137:0/2674570330 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870014070 con 0x7f9874101de0 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f986b7fe640 1 -- 172.21.15.137:0/2674570330 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9870010af0 con 0x7f9874101de0 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f986b7fe640 1 -- 172.21.15.137:0/2674570330 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870017d10 con 0x7f9874101de0 2024-09-12T12:12:20.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987419d850 con 0x7f9874101de0 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.024+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9874074420 con 0x7f9874101de0 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.025+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9874102240 con 0x7f9874101de0 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.026+0000 7f986b7fe640 1 -- 172.21.15.137:0/2674570330 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9870017070 con 0x7f9874101de0 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.027+0000 7f986b7fe640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98600a2c10 0x7f98600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.027+0000 7f9879921640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98600a2c10 0x7f98600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.027+0000 7f986b7fe640 1 -- 172.21.15.137:0/2674570330 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f98700ce510 con 0x7f9874101de0 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.027+0000 7f9879921640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98600a2c10 0x7f98600a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9864000c00 tx=0x7f9864032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:20.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.031+0000 7f986b7fe640 1 -- 172.21.15.137:0/2674570330 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f987009b450 con 0x7f9874101de0 2024-09-12T12:12:20.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.197+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9874062630 con 0x7f98600a2c10 2024-09-12T12:12:20.198 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:20 smithi137 ceph-mon[30950]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:20.198 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:20 smithi137 ceph-mon[30950]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T12:12:20.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.203+0000 7f986b7fe640 1 -- 172.21.15.137:0/2674570330 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258874 (secure 0 0 0) 0x7f9874062630 con 0x7f98600a2c10 2024-09-12T12:12:20.206 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:20.207 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:12:20.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98600a2c10 msgr2=0x7f98600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:20.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98600a2c10 0x7f98600a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9864000c00 tx=0x7f9864032d50 comp rx=0 tx=0).stop 2024-09-12T12:12:20.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874101de0 msgr2=0x7f98741985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:20.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874101de0 0x7f98741985b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f987000b4a0 tx=0x7f987000b960 comp rx=0 tx=0).stop 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 shutdown_connections 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f98600a2c10 0x7f98600a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9874106f10 0x7f987419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874102790 0x7f9874198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 --2- 172.21.15.137:0/2674570330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9874101de0 0x7f98741985b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 >> 172.21.15.137:0/2674570330 conn(0x7f98740fd8c0 msgr2=0x7f98740fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 shutdown_connections 2024-09-12T12:12:20.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:20.206+0000 7f987c386640 1 -- 172.21.15.137:0/2674570330 wait complete. 2024-09-12T12:12:20.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:20 smithi189 ceph-mon[35475]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:20.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:20 smithi189 ceph-mon[35475]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T12:12:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:20 smithi171 ceph-mon[35098]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:20 smithi171 ceph-mon[35098]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-12T12:12:20.945 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-12T12:12:19.125100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653816,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":491,"num_read_kb":364,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356148,"kb_used_data":11028,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585676,"statfs":{"total":1151940427776,"available":1151575732224,"internally_reserved":0,"allocated":11292672,"data_stored":7174356,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":25,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491849+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:11:13.491849+0000","last_peered":"2024-09-12T12:11:13.491849+0000","last_clean":"2024-09-12T12:11:13.491849+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:11:13.491849+0000","last_undegraded":"2024-09-12T12:11:13.491849+0000","last_fullsized":"2024-09-12T12:11:13.491849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200200+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:12:17.200200+0000","last_peered":"2024-09-12T12:12:17.200200+0000","last_clean":"2024-09-12T12:12:17.200200+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:12:17.200200+0000","last_undegraded":"2024-09-12T12:12:17.200200+0000","last_fullsized":"2024-09-12T12:12:17.200200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200138+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:12:17.200138+0000","last_peered":"2024-09-12T12:12:17.200138+0000","last_clean":"2024-09-12T12:12:17.200138+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:12:17.200138+0000","last_undegraded":"2024-09-12T12:12:17.200138+0000","last_fullsized":"2024-09-12T12:12:17.200138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492536+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:11:13.492536+0000","last_peered":"2024-09-12T12:11:13.492536+0000","last_clean":"2024-09-12T12:11:13.492536+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:11:13.492536+0000","last_undegraded":"2024-09-12T12:11:13.492536+0000","last_fullsized":"2024-09-12T12:11:13.492536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034055+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:12:17.034055+0000","last_peered":"2024-09-12T12:12:17.034055+0000","last_clean":"2024-09-12T12:12:17.034055+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:12:17.034055+0000","last_undegraded":"2024-09-12T12:12:17.034055+0000","last_fullsized":"2024-09-12T12:12:17.034055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029133+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:12:17.029133+0000","last_peered":"2024-09-12T12:12:17.029133+0000","last_clean":"2024-09-12T12:12:17.029133+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:12:17.029133+0000","last_undegraded":"2024-09-12T12:12:17.029133+0000","last_fullsized":"2024-09-12T12:12:17.029133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028562+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:12:17.028562+0000","last_peered":"2024-09-12T12:12:17.028562+0000","last_clean":"2024-09-12T12:12:17.028562+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:12:17.028562+0000","last_undegraded":"2024-09-12T12:12:17.028562+0000","last_fullsized":"2024-09-12T12:12:17.028562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498001+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:11:13.498001+0000","last_peered":"2024-09-12T12:11:13.498001+0000","last_clean":"2024-09-12T12:11:13.498001+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:11:13.498001+0000","last_undegraded":"2024-09-12T12:11:13.498001+0000","last_fullsized":"2024-09-12T12:11:13.498001+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029456+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:12:17.029456+0000","last_peered":"2024-09-12T12:12:17.029456+0000","last_clean":"2024-09-12T12:12:17.029456+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:12:17.029456+0000","last_undegraded":"2024-09-12T12:12:17.029456+0000","last_fullsized":"2024-09-12T12:12:17.029456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028748+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:12:17.028748+0000","last_peered":"2024-09-12T12:12:17.028748+0000","last_clean":"2024-09-12T12:12:17.028748+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:12:17.028748+0000","last_undegraded":"2024-09-12T12:12:17.028748+0000","last_fullsized":"2024-09-12T12:12:17.028748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.492580+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:11:13.492580+0000","last_peered":"2024-09-12T12:11:13.492580+0000","last_clean":"2024-09-12T12:11:13.492580+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:11:13.492580+0000","last_undegraded":"2024-09-12T12:11:13.492580+0000","last_fullsized":"2024-09-12T12:11:13.492580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491145+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:11:13.491145+0000","last_peered":"2024-09-12T12:11:13.491145+0000","last_clean":"2024-09-12T12:11:13.491145+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:11:13.491145+0000","last_undegraded":"2024-09-12T12:11:13.491145+0000","last_fullsized":"2024-09-12T12:11:13.491145+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200027+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:12:17.200027+0000","last_peered":"2024-09-12T12:12:17.200027+0000","last_clean":"2024-09-12T12:12:17.200027+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:12:17.200027+0000","last_undegraded":"2024-09-12T12:12:17.200027+0000","last_fullsized":"2024-09-12T12:12:17.200027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.199927+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:12:17.199927+0000","last_peered":"2024-09-12T12:12:17.199927+0000","last_clean":"2024-09-12T12:12:17.199927+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:12:17.199927+0000","last_undegraded":"2024-09-12T12:12:17.199927+0000","last_fullsized":"2024-09-12T12:12:17.199927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200399+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:12:17.200399+0000","last_peered":"2024-09-12T12:12:17.200399+0000","last_clean":"2024-09-12T12:12:17.200399+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:12:17.200399+0000","last_undegraded":"2024-09-12T12:12:17.200399+0000","last_fullsized":"2024-09-12T12:12:17.200399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491907+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:11:13.491907+0000","last_peered":"2024-09-12T12:11:13.491907+0000","last_clean":"2024-09-12T12:11:13.491907+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:11:13.491907+0000","last_undegraded":"2024-09-12T12:11:13.491907+0000","last_fullsized":"2024-09-12T12:11:13.491907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.502053+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:11:13.502053+0000","last_peered":"2024-09-12T12:11:13.502053+0000","last_clean":"2024-09-12T12:11:13.502053+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:11:13.502053+0000","last_undegraded":"2024-09-12T12:11:13.502053+0000","last_fullsized":"2024-09-12T12:11:13.502053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028525+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:12:17.028525+0000","last_peered":"2024-09-12T12:12:17.028525+0000","last_clean":"2024-09-12T12:12:17.028525+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:12:17.028525+0000","last_undegraded":"2024-09-12T12:12:17.028525+0000","last_fullsized":"2024-09-12T12:12:17.028525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491690+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:11:13.491690+0000","last_peered":"2024-09-12T12:11:13.491690+0000","last_clean":"2024-09-12T12:11:13.491690+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:11:13.491690+0000","last_undegraded":"2024-09-12T12:11:13.491690+0000","last_fullsized":"2024-09-12T12:11:13.491690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491185+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:11:13.491185+0000","last_peered":"2024-09-12T12:11:13.491185+0000","last_clean":"2024-09-12T12:11:13.491185+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:11:13.491185+0000","last_undegraded":"2024-09-12T12:11:13.491185+0000","last_fullsized":"2024-09-12T12:11:13.491185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200826+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:12:17.200826+0000","last_peered":"2024-09-12T12:12:17.200826+0000","last_clean":"2024-09-12T12:12:17.200826+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:12:17.200826+0000","last_undegraded":"2024-09-12T12:12:17.200826+0000","last_fullsized":"2024-09-12T12:12:17.200826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497730+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:11:13.497730+0000","last_peered":"2024-09-12T12:11:13.497730+0000","last_clean":"2024-09-12T12:11:13.497730+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:11:13.497730+0000","last_undegraded":"2024-09-12T12:11:13.497730+0000","last_fullsized":"2024-09-12T12:11:13.497730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031314+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:12:16.031314+0000","last_peered":"2024-09-12T12:12:16.031314+0000","last_clean":"2024-09-12T12:12:16.031314+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:12:16.031314+0000","last_undegraded":"2024-09-12T12:12:16.031314+0000","last_fullsized":"2024-09-12T12:12:16.031314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498394+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:11:13.498394+0000","last_peered":"2024-09-12T12:11:13.498394+0000","last_clean":"2024-09-12T12:11:13.498394+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:11:13.498394+0000","last_undegraded":"2024-09-12T12:11:13.498394+0000","last_fullsized":"2024-09-12T12:11:13.498394+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028783+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:12:17.028783+0000","last_peered":"2024-09-12T12:12:17.028783+0000","last_clean":"2024-09-12T12:12:17.028783+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:12:17.028783+0000","last_undegraded":"2024-09-12T12:12:17.028783+0000","last_fullsized":"2024-09-12T12:12:17.028783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491855+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:11:13.491855+0000","last_peered":"2024-09-12T12:11:13.491855+0000","last_clean":"2024-09-12T12:11:13.491855+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:11:13.491855+0000","last_undegraded":"2024-09-12T12:11:13.491855+0000","last_fullsized":"2024-09-12T12:11:13.491855+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491115+0000","last_change":"2024-09-12T12:11:12.500411+0000","last_active":"2024-09-12T12:11:13.491115+0000","last_peered":"2024-09-12T12:11:13.491115+0000","last_clean":"2024-09-12T12:11:13.491115+0000","last_became_active":"2024-09-12T12:11:12.500126+0000","last_became_peered":"2024-09-12T12:11:12.500126+0000","last_unstale":"2024-09-12T12:11:13.491115+0000","last_undegraded":"2024-09-12T12:11:13.491115+0000","last_fullsized":"2024-09-12T12:11:13.491115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:23:49.072306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200084+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:12:17.200084+0000","last_peered":"2024-09-12T12:12:17.200084+0000","last_clean":"2024-09-12T12:12:17.200084+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:12:17.200084+0000","last_undegraded":"2024-09-12T12:12:17.200084+0000","last_fullsized":"2024-09-12T12:12:17.200084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028596+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:12:17.028596+0000","last_peered":"2024-09-12T12:12:17.028596+0000","last_clean":"2024-09-12T12:12:17.028596+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:12:17.028596+0000","last_undegraded":"2024-09-12T12:12:17.028596+0000","last_fullsized":"2024-09-12T12:12:17.028596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030334+0000","last_change":"2024-09-12T12:12:17.030334+0000","last_active":"2024-09-12T12:12:17.030334+0000","last_peered":"2024-09-12T12:12:17.030334+0000","last_clean":"2024-09-12T12:12:17.030334+0000","last_became_active":"2024-09-12T12:12:17.029947+0000","last_became_peered":"2024-09-12T12:12:17.029947+0000","last_unstale":"2024-09-12T12:12:17.030334+0000","last_undegraded":"2024-09-12T12:12:17.030334+0000","last_fullsized":"2024-09-12T12:12:17.030334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:10.299557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491659+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:11:13.491659+0000","last_peered":"2024-09-12T12:11:13.491659+0000","last_clean":"2024-09-12T12:11:13.491659+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:11:13.491659+0000","last_undegraded":"2024-09-12T12:11:13.491659+0000","last_fullsized":"2024-09-12T12:11:13.491659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033339+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:12:17.033339+0000","last_peered":"2024-09-12T12:12:17.033339+0000","last_clean":"2024-09-12T12:12:17.033339+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:12:17.033339+0000","last_undegraded":"2024-09-12T12:12:17.033339+0000","last_fullsized":"2024-09-12T12:12:17.033339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032009+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:12:16.032009+0000","last_peered":"2024-09-12T12:12:16.032009+0000","last_clean":"2024-09-12T12:12:16.032009+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:12:16.032009+0000","last_undegraded":"2024-09-12T12:12:16.032009+0000","last_fullsized":"2024-09-12T12:12:16.032009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-12T12:11:16.259126+0000","last_change":"2024-09-12T12:12:16.032862+0000","last_active":"2024-09-12T12:12:16.032862+0000","last_peered":"2024-09-12T12:11:16.259126+0000","last_clean":"2024-09-12T12:11:16.259126+0000","last_became_active":"2024-09-12T12:11:12.507860+0000","last_became_peered":"2024-09-12T12:11:12.507860+0000","last_unstale":"2024-09-12T12:11:16.259126+0000","last_undegraded":"2024-09-12T12:11:16.259126+0000","last_fullsized":"2024-09-12T12:11:16.259126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029242+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:12:17.029242+0000","last_peered":"2024-09-12T12:12:17.029242+0000","last_clean":"2024-09-12T12:12:17.029242+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:12:17.029242+0000","last_undegraded":"2024-09-12T12:12:17.029242+0000","last_fullsized":"2024-09-12T12:12:17.029242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.556515+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:11:21.556515+0000","last_peered":"2024-09-12T12:11:21.556515+0000","last_clean":"2024-09-12T12:11:21.556515+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:11:21.556515+0000","last_undegraded":"2024-09-12T12:11:21.556515+0000","last_fullsized":"2024-09-12T12:11:21.556515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497911+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:11:13.497911+0000","last_peered":"2024-09-12T12:11:13.497911+0000","last_clean":"2024-09-12T12:11:13.497911+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:11:13.497911+0000","last_undegraded":"2024-09-12T12:11:13.497911+0000","last_fullsized":"2024-09-12T12:11:13.497911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032731+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:12:16.032731+0000","last_peered":"2024-09-12T12:12:16.032731+0000","last_clean":"2024-09-12T12:12:16.032731+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:12:16.032731+0000","last_undegraded":"2024-09-12T12:12:16.032731+0000","last_fullsized":"2024-09-12T12:12:16.032731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501931+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:11:13.501931+0000","last_peered":"2024-09-12T12:11:13.501931+0000","last_clean":"2024-09-12T12:11:13.501931+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:11:13.501931+0000","last_undegraded":"2024-09-12T12:11:13.501931+0000","last_fullsized":"2024-09-12T12:11:13.501931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":83,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029335+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:12:17.029335+0000","last_peered":"2024-09-12T12:12:17.029335+0000","last_clean":"2024-09-12T12:12:17.029335+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:12:17.029335+0000","last_undegraded":"2024-09-12T12:12:17.029335+0000","last_fullsized":"2024-09-12T12:12:17.029335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028966+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:12:17.028966+0000","last_peered":"2024-09-12T12:12:17.028966+0000","last_clean":"2024-09-12T12:12:17.028966+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:12:17.028966+0000","last_undegraded":"2024-09-12T12:12:17.028966+0000","last_fullsized":"2024-09-12T12:12:17.028966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200368+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:12:17.200368+0000","last_peered":"2024-09-12T12:12:17.200368+0000","last_clean":"2024-09-12T12:12:17.200368+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:12:17.200368+0000","last_undegraded":"2024-09-12T12:12:17.200368+0000","last_fullsized":"2024-09-12T12:12:17.200368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032201+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:12:16.032201+0000","last_peered":"2024-09-12T12:12:16.032201+0000","last_clean":"2024-09-12T12:12:16.032201+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:12:16.032201+0000","last_undegraded":"2024-09-12T12:12:16.032201+0000","last_fullsized":"2024-09-12T12:12:16.032201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200747+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:12:17.200747+0000","last_peered":"2024-09-12T12:12:17.200747+0000","last_clean":"2024-09-12T12:12:17.200747+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:12:17.200747+0000","last_undegraded":"2024-09-12T12:12:17.200747+0000","last_fullsized":"2024-09-12T12:12:17.200747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":170,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.544336+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:12:17.544336+0000","last_peered":"2024-09-12T12:12:17.544336+0000","last_clean":"2024-09-12T12:12:17.544336+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:12:17.544336+0000","last_undegraded":"2024-09-12T12:12:17.544336+0000","last_fullsized":"2024-09-12T12:12:17.544336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":117,"num_write":27,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034261+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:12:17.034261+0000","last_peered":"2024-09-12T12:12:17.034261+0000","last_clean":"2024-09-12T12:12:17.034261+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:12:17.034261+0000","last_undegraded":"2024-09-12T12:12:17.034261+0000","last_fullsized":"2024-09-12T12:12:17.034261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028566+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:12:17.028566+0000","last_peered":"2024-09-12T12:12:17.028566+0000","last_clean":"2024-09-12T12:12:17.028566+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:12:17.028566+0000","last_undegraded":"2024-09-12T12:12:17.028566+0000","last_fullsized":"2024-09-12T12:12:17.028566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490815+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:11:13.490815+0000","last_peered":"2024-09-12T12:11:13.490815+0000","last_clean":"2024-09-12T12:11:13.490815+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:11:13.490815+0000","last_undegraded":"2024-09-12T12:11:13.490815+0000","last_fullsized":"2024-09-12T12:11:13.490815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031888+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:12:16.031888+0000","last_peered":"2024-09-12T12:12:16.031888+0000","last_clean":"2024-09-12T12:12:16.031888+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:12:16.031888+0000","last_undegraded":"2024-09-12T12:12:16.031888+0000","last_fullsized":"2024-09-12T12:12:16.031888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029013+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:12:17.029013+0000","last_peered":"2024-09-12T12:12:17.029013+0000","last_clean":"2024-09-12T12:12:17.029013+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:12:17.029013+0000","last_undegraded":"2024-09-12T12:12:17.029013+0000","last_fullsized":"2024-09-12T12:12:17.029013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:12:13.539328+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:12:13.539328+0000","last_peered":"2024-09-12T12:12:13.539328+0000","last_clean":"2024-09-12T12:12:13.539328+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:12:13.539328+0000","last_undegraded":"2024-09-12T12:12:13.539328+0000","last_fullsized":"2024-09-12T12:12:13.539328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031317+0000","last_change":"2024-09-12T12:12:17.031317+0000","last_active":"2024-09-12T12:12:17.031317+0000","last_peered":"2024-09-12T12:12:17.031317+0000","last_clean":"2024-09-12T12:12:17.031317+0000","last_became_active":"2024-09-12T12:12:17.030842+0000","last_became_peered":"2024-09-12T12:12:17.030842+0000","last_unstale":"2024-09-12T12:12:17.031317+0000","last_undegraded":"2024-09-12T12:12:17.031317+0000","last_fullsized":"2024-09-12T12:12:17.031317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:11.616682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033026+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:12:17.033026+0000","last_peered":"2024-09-12T12:12:17.033026+0000","last_clean":"2024-09-12T12:12:17.033026+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:12:17.033026+0000","last_undegraded":"2024-09-12T12:12:17.033026+0000","last_fullsized":"2024-09-12T12:12:17.033026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032581+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:12:16.032581+0000","last_peered":"2024-09-12T12:12:16.032581+0000","last_clean":"2024-09-12T12:12:16.032581+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:12:16.032581+0000","last_undegraded":"2024-09-12T12:12:16.032581+0000","last_fullsized":"2024-09-12T12:12:16.032581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034442+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:12:17.034442+0000","last_peered":"2024-09-12T12:12:17.034442+0000","last_clean":"2024-09-12T12:12:17.034442+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:12:17.034442+0000","last_undegraded":"2024-09-12T12:12:17.034442+0000","last_fullsized":"2024-09-12T12:12:17.034442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491726+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:11:13.491726+0000","last_peered":"2024-09-12T12:11:13.491726+0000","last_clean":"2024-09-12T12:11:13.491726+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:11:13.491726+0000","last_undegraded":"2024-09-12T12:11:13.491726+0000","last_fullsized":"2024-09-12T12:11:13.491726+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497437+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:11:13.497437+0000","last_peered":"2024-09-12T12:11:13.497437+0000","last_clean":"2024-09-12T12:11:13.497437+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:11:13.497437+0000","last_undegraded":"2024-09-12T12:11:13.497437+0000","last_fullsized":"2024-09-12T12:11:13.497437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029297+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:12:17.029297+0000","last_peered":"2024-09-12T12:12:17.029297+0000","last_clean":"2024-09-12T12:12:17.029297+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:12:17.029297+0000","last_undegraded":"2024-09-12T12:12:17.029297+0000","last_fullsized":"2024-09-12T12:12:17.029297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032086+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:12:16.032086+0000","last_peered":"2024-09-12T12:12:16.032086+0000","last_clean":"2024-09-12T12:12:16.032086+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:12:16.032086+0000","last_undegraded":"2024-09-12T12:12:16.032086+0000","last_fullsized":"2024-09-12T12:12:16.032086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029052+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:12:17.029052+0000","last_peered":"2024-09-12T12:12:17.029052+0000","last_clean":"2024-09-12T12:12:17.029052+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:12:17.029052+0000","last_undegraded":"2024-09-12T12:12:17.029052+0000","last_fullsized":"2024-09-12T12:12:17.029052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029406+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:12:17.029406+0000","last_peered":"2024-09-12T12:12:17.029406+0000","last_clean":"2024-09-12T12:12:17.029406+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:12:17.029406+0000","last_undegraded":"2024-09-12T12:12:17.029406+0000","last_fullsized":"2024-09-12T12:12:17.029406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029338+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:12:17.029338+0000","last_peered":"2024-09-12T12:12:17.029338+0000","last_clean":"2024-09-12T12:12:17.029338+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:12:17.029338+0000","last_undegraded":"2024-09-12T12:12:17.029338+0000","last_fullsized":"2024-09-12T12:12:17.029338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029959+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:12:17.029959+0000","last_peered":"2024-09-12T12:12:17.029959+0000","last_clean":"2024-09-12T12:12:17.029959+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:12:17.029959+0000","last_undegraded":"2024-09-12T12:12:17.029959+0000","last_fullsized":"2024-09-12T12:12:17.029959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:22.465059+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:11:22.465059+0000","last_peered":"2024-09-12T12:11:22.465059+0000","last_clean":"2024-09-12T12:11:22.465059+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:11:22.465059+0000","last_undegraded":"2024-09-12T12:11:22.465059+0000","last_fullsized":"2024-09-12T12:11:22.465059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498114+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:11:13.498114+0000","last_peered":"2024-09-12T12:11:13.498114+0000","last_clean":"2024-09-12T12:11:13.498114+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:11:13.498114+0000","last_undegraded":"2024-09-12T12:11:13.498114+0000","last_fullsized":"2024-09-12T12:11:13.498114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029901+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:12:17.029901+0000","last_peered":"2024-09-12T12:12:17.029901+0000","last_clean":"2024-09-12T12:12:17.029901+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:12:17.029901+0000","last_undegraded":"2024-09-12T12:12:17.029901+0000","last_fullsized":"2024-09-12T12:12:17.029901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.490784+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:11:13.490784+0000","last_peered":"2024-09-12T12:11:13.490784+0000","last_clean":"2024-09-12T12:11:13.490784+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:11:13.490784+0000","last_undegraded":"2024-09-12T12:11:13.490784+0000","last_fullsized":"2024-09-12T12:11:13.490784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":77,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:12:13.538767+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:12:13.538767+0000","last_peered":"2024-09-12T12:12:13.538767+0000","last_clean":"2024-09-12T12:12:13.538767+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:12:13.538767+0000","last_undegraded":"2024-09-12T12:12:13.538767+0000","last_fullsized":"2024-09-12T12:12:13.538767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031412+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:12:16.031412+0000","last_peered":"2024-09-12T12:12:16.031412+0000","last_clean":"2024-09-12T12:12:16.031412+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:12:16.031412+0000","last_undegraded":"2024-09-12T12:12:16.031412+0000","last_fullsized":"2024-09-12T12:12:16.031412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028913+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:12:17.028913+0000","last_peered":"2024-09-12T12:12:17.028913+0000","last_clean":"2024-09-12T12:12:17.028913+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:12:17.028913+0000","last_undegraded":"2024-09-12T12:12:17.028913+0000","last_fullsized":"2024-09-12T12:12:17.028913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029412+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:12:17.029412+0000","last_peered":"2024-09-12T12:12:17.029412+0000","last_clean":"2024-09-12T12:12:17.029412+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:12:17.029412+0000","last_undegraded":"2024-09-12T12:12:17.029412+0000","last_fullsized":"2024-09-12T12:12:17.029412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.501897+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:11:13.501897+0000","last_peered":"2024-09-12T12:11:13.501897+0000","last_clean":"2024-09-12T12:11:13.501897+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:11:13.501897+0000","last_undegraded":"2024-09-12T12:11:13.501897+0000","last_fullsized":"2024-09-12T12:11:13.501897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028914+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:12:17.028914+0000","last_peered":"2024-09-12T12:12:17.028914+0000","last_clean":"2024-09-12T12:12:17.028914+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:12:17.028914+0000","last_undegraded":"2024-09-12T12:12:17.028914+0000","last_fullsized":"2024-09-12T12:12:17.028914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034512+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:12:17.034512+0000","last_peered":"2024-09-12T12:12:17.034512+0000","last_clean":"2024-09-12T12:12:17.034512+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:12:17.034512+0000","last_undegraded":"2024-09-12T12:12:17.034512+0000","last_fullsized":"2024-09-12T12:12:17.034512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491176+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:11:13.491176+0000","last_peered":"2024-09-12T12:11:13.491176+0000","last_clean":"2024-09-12T12:11:13.491176+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:11:13.491176+0000","last_undegraded":"2024-09-12T12:11:13.491176+0000","last_fullsized":"2024-09-12T12:11:13.491176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:37.499854+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:11:37.499854+0000","last_peered":"2024-09-12T12:11:37.499854+0000","last_clean":"2024-09-12T12:11:37.499854+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:11:37.499854+0000","last_undegraded":"2024-09-12T12:11:37.499854+0000","last_fullsized":"2024-09-12T12:11:37.499854+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498306+0000","last_change":"2024-09-12T12:11:12.506733+0000","last_active":"2024-09-12T12:11:13.498306+0000","last_peered":"2024-09-12T12:11:13.498306+0000","last_clean":"2024-09-12T12:11:13.498306+0000","last_became_active":"2024-09-12T12:11:12.506447+0000","last_became_peered":"2024-09-12T12:11:12.506447+0000","last_unstale":"2024-09-12T12:11:13.498306+0000","last_undegraded":"2024-09-12T12:11:13.498306+0000","last_fullsized":"2024-09-12T12:11:13.498306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:58:36.338767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.544999+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:11:20.544999+0000","last_peered":"2024-09-12T12:11:20.544999+0000","last_clean":"2024-09-12T12:11:20.544999+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:11:20.544999+0000","last_undegraded":"2024-09-12T12:11:20.544999+0000","last_fullsized":"2024-09-12T12:11:20.544999+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200202+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:12:17.200202+0000","last_peered":"2024-09-12T12:12:17.200202+0000","last_clean":"2024-09-12T12:12:17.200202+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:12:17.200202+0000","last_undegraded":"2024-09-12T12:12:17.200202+0000","last_fullsized":"2024-09-12T12:12:17.200202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200521+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:12:17.200521+0000","last_peered":"2024-09-12T12:12:17.200521+0000","last_clean":"2024-09-12T12:12:17.200521+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:12:17.200521+0000","last_undegraded":"2024-09-12T12:12:17.200521+0000","last_fullsized":"2024-09-12T12:12:17.200521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033160+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:12:17.033160+0000","last_peered":"2024-09-12T12:12:17.033160+0000","last_clean":"2024-09-12T12:12:17.033160+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:12:17.033160+0000","last_undegraded":"2024-09-12T12:12:17.033160+0000","last_fullsized":"2024-09-12T12:12:17.033160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545237+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:11:20.545237+0000","last_peered":"2024-09-12T12:11:20.545237+0000","last_clean":"2024-09-12T12:11:20.545237+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:11:20.545237+0000","last_undegraded":"2024-09-12T12:11:20.545237+0000","last_fullsized":"2024-09-12T12:11:20.545237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.491769+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:11:13.491769+0000","last_peered":"2024-09-12T12:11:13.491769+0000","last_clean":"2024-09-12T12:11:13.491769+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:11:13.491769+0000","last_undegraded":"2024-09-12T12:11:13.491769+0000","last_fullsized":"2024-09-12T12:11:13.491769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498164+0000","last_change":"2024-09-12T12:11:09.477481+0000","last_active":"2024-09-12T12:11:13.498164+0000","last_peered":"2024-09-12T12:11:13.498164+0000","last_clean":"2024-09-12T12:11:13.498164+0000","last_became_active":"2024-09-12T12:11:09.477126+0000","last_became_peered":"2024-09-12T12:11:09.477126+0000","last_unstale":"2024-09-12T12:11:13.498164+0000","last_undegraded":"2024-09-12T12:11:13.498164+0000","last_fullsized":"2024-09-12T12:11:13.498164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:57.467398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.497665+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:11:13.497665+0000","last_peered":"2024-09-12T12:11:13.497665+0000","last_clean":"2024-09-12T12:11:13.497665+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:11:13.497665+0000","last_undegraded":"2024-09-12T12:11:13.497665+0000","last_fullsized":"2024-09-12T12:11:13.497665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:20.545706+0000","last_change":"2024-09-12T12:11:09.484682+0000","last_active":"2024-09-12T12:11:20.545706+0000","last_peered":"2024-09-12T12:11:20.545706+0000","last_clean":"2024-09-12T12:11:20.545706+0000","last_became_active":"2024-09-12T12:11:09.484294+0000","last_became_peered":"2024-09-12T12:11:09.484294+0000","last_unstale":"2024-09-12T12:11:20.545706+0000","last_undegraded":"2024-09-12T12:11:20.545706+0000","last_fullsized":"2024-09-12T12:11:20.545706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:01:39.731705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028769+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:12:17.028769+0000","last_peered":"2024-09-12T12:12:17.028769+0000","last_clean":"2024-09-12T12:12:17.028769+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:12:17.028769+0000","last_undegraded":"2024-09-12T12:12:17.028769+0000","last_fullsized":"2024-09-12T12:12:17.028769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031761+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:12:16.031761+0000","last_peered":"2024-09-12T12:12:16.031761+0000","last_clean":"2024-09-12T12:12:16.031761+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:12:16.031761+0000","last_undegraded":"2024-09-12T12:12:16.031761+0000","last_fullsized":"2024-09-12T12:12:16.031761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034217+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:12:17.034217+0000","last_peered":"2024-09-12T12:12:17.034217+0000","last_clean":"2024-09-12T12:12:17.034217+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:12:17.034217+0000","last_undegraded":"2024-09-12T12:12:17.034217+0000","last_fullsized":"2024-09-12T12:12:17.034217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200709+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:12:17.200709+0000","last_peered":"2024-09-12T12:12:17.200709+0000","last_clean":"2024-09-12T12:12:17.200709+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:12:17.200709+0000","last_undegraded":"2024-09-12T12:12:17.200709+0000","last_fullsized":"2024-09-12T12:12:17.200709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034393+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:12:17.034393+0000","last_peered":"2024-09-12T12:12:17.034393+0000","last_clean":"2024-09-12T12:12:17.034393+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:12:17.034393+0000","last_undegraded":"2024-09-12T12:12:17.034393+0000","last_fullsized":"2024-09-12T12:12:17.034393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200361+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:12:17.200361+0000","last_peered":"2024-09-12T12:12:17.200361+0000","last_clean":"2024-09-12T12:12:17.200361+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:12:17.200361+0000","last_undegraded":"2024-09-12T12:12:17.200361+0000","last_fullsized":"2024-09-12T12:12:17.200361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.499817+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:11:13.499817+0000","last_peered":"2024-09-12T12:11:13.499817+0000","last_clean":"2024-09-12T12:11:13.499817+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:11:13.499817+0000","last_undegraded":"2024-09-12T12:11:13.499817+0000","last_fullsized":"2024-09-12T12:11:13.499817+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:13.498213+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:11:13.498213+0000","last_peered":"2024-09-12T12:11:13.498213+0000","last_clean":"2024-09-12T12:11:13.498213+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:11:13.498213+0000","last_undegraded":"2024-09-12T12:11:13.498213+0000","last_fullsized":"2024-09-12T12:11:13.498213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T12:11:21.557140+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:11:21.557140+0000","last_peered":"2024-09-12T12:11:21.557140+0000","last_clean":"2024-09-12T12:11:21.557140+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:11:21.557140+0000","last_undegraded":"2024-09-12T12:11:21.557140+0000","last_fullsized":"2024-09-12T12:11:21.557140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":355,"num_read_kb":245,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":427186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":1095286,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":427422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168619,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":435927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1036289,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":426811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1018565,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":427641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":427122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":504180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":502830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:12:20.949 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:12:21.199 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:21.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:21 smithi137 ceph-mon[30950]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:21.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:21 smithi189 ceph-mon[35475]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:21.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:21 smithi171 ceph-mon[35098]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 -- 172.21.15.137:0/4110850768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c105e00 msgr2=0x7f7a4c10d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 --2- 172.21.15.137:0/4110850768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c105e00 0x7f7a4c10d980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009ca0 tx=0x7f7a3c02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 -- 172.21.15.137:0/4110850768 shutdown_connections 2024-09-12T12:12:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 --2- 172.21.15.137:0/4110850768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c105e00 0x7f7a4c10d980 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 --2- 172.21.15.137:0/4110850768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c101fc0 0x7f7a4c1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 --2- 172.21.15.137:0/4110850768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a4c101610 0x7f7a4c1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 -- 172.21.15.137:0/4110850768 >> 172.21.15.137:0/4110850768 conn(0x7f7a4c0fc5c0 msgr2=0x7f7a4c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 -- 172.21.15.137:0/4110850768 shutdown_connections 2024-09-12T12:12:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.702+0000 7f7a53351640 1 -- 172.21.15.137:0/4110850768 wait complete. 2024-09-12T12:12:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 Processor -- start 2024-09-12T12:12:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 -- start start 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a4c101610 0x7f7a4c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c101fc0 0x7f7a4c197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c105e00 0x7f7a4c19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c110e60 con 0x7f7a4c101fc0 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c110ce0 con 0x7f7a4c105e00 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a53351640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c110fe0 con 0x7f7a4c101610 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a508ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c101fc0 0x7f7a4c197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a508ec640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c101fc0 0x7f7a4c197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56706/0 (socket says 172.21.15.137:56706) 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a508ec640 1 -- 172.21.15.137:0/1634455252 learned_addr learned my addr 172.21.15.137:0/1634455252 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a510ed640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a4c101610 0x7f7a4c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.703+0000 7f7a518ee640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c105e00 0x7f7a4c19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a508ec640 1 -- 172.21.15.137:0/1634455252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a4c101610 msgr2=0x7f7a4c1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a508ec640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a4c101610 0x7f7a4c1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a508ec640 1 -- 172.21.15.137:0/1634455252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c105e00 msgr2=0x7f7a4c19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a508ec640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c105e00 0x7f7a4c19ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a508ec640 1 -- 172.21.15.137:0/1634455252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c19c170 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a508ec640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c101fc0 0x7f7a4c197700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c970 tx=0x7f7a4000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a4a7fc640 1 -- 172.21.15.137:0/1634455252 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40014070 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a4a7fc640 1 -- 172.21.15.137:0/1634455252 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a40004440 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c19c460 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a4a7fc640 1 -- 172.21.15.137:0/1634455252 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40003e10 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.704+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c1a3d40 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.705+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c1019f0 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.706+0000 7f7a4a7fc640 1 -- 172.21.15.137:0/1634455252 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7a40017070 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.707+0000 7f7a4a7fc640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7a340a2c10 0x7f7a340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.707+0000 7f7a510ed640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7a340a2c10 0x7f7a340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.707+0000 7f7a4a7fc640 1 -- 172.21.15.137:0/1634455252 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f7a400cef70 con 0x7f7a4c101fc0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.707+0000 7f7a510ed640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7a340a2c10 0x7f7a340a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400abe0 tx=0x7f7a4400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.711+0000 7f7a4a7fc640 1 -- 172.21.15.137:0/1634455252 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a4001e020 con 0x7f7a4c101fc0 2024-09-12T12:12:21.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.878+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a4c062630 con 0x7f7a340a2c10 2024-09-12T12:12:21.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.884+0000 7f7a4a7fc640 1 -- 172.21.15.137:0/1634455252 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258883 (secure 0 0 0) 0x7f7a4c062630 con 0x7f7a340a2c10 2024-09-12T12:12:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:21.888 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:12:21.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7a340a2c10 msgr2=0x7f7a340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:21.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7a340a2c10 0x7f7a340a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400abe0 tx=0x7f7a4400a2d0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c101fc0 msgr2=0x7f7a4c197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:21.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c101fc0 0x7f7a4c197700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000c970 tx=0x7f7a4000cd70 comp rx=0 tx=0).stop 2024-09-12T12:12:21.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 shutdown_connections 2024-09-12T12:12:21.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7a340a2c10 0x7f7a340a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c105e00 0x7f7a4c19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7a4c101fc0 0x7f7a4c197700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 --2- 172.21.15.137:0/1634455252 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a4c101610 0x7f7a4c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:21.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 >> 172.21.15.137:0/1634455252 conn(0x7f7a4c0fc5c0 msgr2=0x7f7a4c1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:21.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 shutdown_connections 2024-09-12T12:12:21.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:21.887+0000 7f7a53351640 1 -- 172.21.15.137:0/1634455252 wait complete. 2024-09-12T12:12:22.258 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:22 smithi137 ceph-mon[30950]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:22.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:22 smithi189 ceph-mon[35475]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:22.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:22 smithi171 ceph-mon[35098]: pgmap v317: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:22.641 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-12T12:12:21.125562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653816,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":499,"num_read_kb":367,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356300,"kb_used_data":11148,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585524,"statfs":{"total":1151940427776,"available":1151575576576,"internally_reserved":0,"allocated":11415552,"data_stored":7259431,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001905"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031172+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:12:17.031172+0000","last_peered":"2024-09-12T12:12:17.031172+0000","last_clean":"2024-09-12T12:12:17.031172+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:12:17.031172+0000","last_undegraded":"2024-09-12T12:12:17.031172+0000","last_fullsized":"2024-09-12T12:12:17.031172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200200+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:12:17.200200+0000","last_peered":"2024-09-12T12:12:17.200200+0000","last_clean":"2024-09-12T12:12:17.200200+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:12:17.200200+0000","last_undegraded":"2024-09-12T12:12:17.200200+0000","last_fullsized":"2024-09-12T12:12:17.200200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200138+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:12:17.200138+0000","last_peered":"2024-09-12T12:12:17.200138+0000","last_clean":"2024-09-12T12:12:17.200138+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:12:17.200138+0000","last_undegraded":"2024-09-12T12:12:17.200138+0000","last_fullsized":"2024-09-12T12:12:17.200138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200871+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:12:17.200871+0000","last_peered":"2024-09-12T12:12:17.200871+0000","last_clean":"2024-09-12T12:12:17.200871+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:12:17.200871+0000","last_undegraded":"2024-09-12T12:12:17.200871+0000","last_fullsized":"2024-09-12T12:12:17.200871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034055+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:12:17.034055+0000","last_peered":"2024-09-12T12:12:17.034055+0000","last_clean":"2024-09-12T12:12:17.034055+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:12:17.034055+0000","last_undegraded":"2024-09-12T12:12:17.034055+0000","last_fullsized":"2024-09-12T12:12:17.034055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029133+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:12:17.029133+0000","last_peered":"2024-09-12T12:12:17.029133+0000","last_clean":"2024-09-12T12:12:17.029133+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:12:17.029133+0000","last_undegraded":"2024-09-12T12:12:17.029133+0000","last_fullsized":"2024-09-12T12:12:17.029133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028562+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:12:17.028562+0000","last_peered":"2024-09-12T12:12:17.028562+0000","last_clean":"2024-09-12T12:12:17.028562+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:12:17.028562+0000","last_undegraded":"2024-09-12T12:12:17.028562+0000","last_fullsized":"2024-09-12T12:12:17.028562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033265+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:12:17.033265+0000","last_peered":"2024-09-12T12:12:17.033265+0000","last_clean":"2024-09-12T12:12:17.033265+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:12:17.033265+0000","last_undegraded":"2024-09-12T12:12:17.033265+0000","last_fullsized":"2024-09-12T12:12:17.033265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029456+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:12:17.029456+0000","last_peered":"2024-09-12T12:12:17.029456+0000","last_clean":"2024-09-12T12:12:17.029456+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:12:17.029456+0000","last_undegraded":"2024-09-12T12:12:17.029456+0000","last_fullsized":"2024-09-12T12:12:17.029456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028748+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:12:17.028748+0000","last_peered":"2024-09-12T12:12:17.028748+0000","last_clean":"2024-09-12T12:12:17.028748+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:12:17.028748+0000","last_undegraded":"2024-09-12T12:12:17.028748+0000","last_fullsized":"2024-09-12T12:12:17.028748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200997+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:12:17.200997+0000","last_peered":"2024-09-12T12:12:17.200997+0000","last_clean":"2024-09-12T12:12:17.200997+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:12:17.200997+0000","last_undegraded":"2024-09-12T12:12:17.200997+0000","last_fullsized":"2024-09-12T12:12:17.200997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025292+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:12:17.025292+0000","last_peered":"2024-09-12T12:12:17.025292+0000","last_clean":"2024-09-12T12:12:17.025292+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:12:17.025292+0000","last_undegraded":"2024-09-12T12:12:17.025292+0000","last_fullsized":"2024-09-12T12:12:17.025292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200027+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:12:17.200027+0000","last_peered":"2024-09-12T12:12:17.200027+0000","last_clean":"2024-09-12T12:12:17.200027+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:12:17.200027+0000","last_undegraded":"2024-09-12T12:12:17.200027+0000","last_fullsized":"2024-09-12T12:12:17.200027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.199927+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:12:17.199927+0000","last_peered":"2024-09-12T12:12:17.199927+0000","last_clean":"2024-09-12T12:12:17.199927+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:12:17.199927+0000","last_undegraded":"2024-09-12T12:12:17.199927+0000","last_fullsized":"2024-09-12T12:12:17.199927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200399+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:12:17.200399+0000","last_peered":"2024-09-12T12:12:17.200399+0000","last_clean":"2024-09-12T12:12:17.200399+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:12:17.200399+0000","last_undegraded":"2024-09-12T12:12:17.200399+0000","last_fullsized":"2024-09-12T12:12:17.200399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031383+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:12:17.031383+0000","last_peered":"2024-09-12T12:12:17.031383+0000","last_clean":"2024-09-12T12:12:17.031383+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:12:17.031383+0000","last_undegraded":"2024-09-12T12:12:17.031383+0000","last_fullsized":"2024-09-12T12:12:17.031383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028526+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:12:17.028526+0000","last_peered":"2024-09-12T12:12:17.028526+0000","last_clean":"2024-09-12T12:12:17.028526+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:12:17.028526+0000","last_undegraded":"2024-09-12T12:12:17.028526+0000","last_fullsized":"2024-09-12T12:12:17.028526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028525+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:12:17.028525+0000","last_peered":"2024-09-12T12:12:17.028525+0000","last_clean":"2024-09-12T12:12:17.028525+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:12:17.028525+0000","last_undegraded":"2024-09-12T12:12:17.028525+0000","last_fullsized":"2024-09-12T12:12:17.028525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030717+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:12:17.030717+0000","last_peered":"2024-09-12T12:12:17.030717+0000","last_clean":"2024-09-12T12:12:17.030717+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:12:17.030717+0000","last_undegraded":"2024-09-12T12:12:17.030717+0000","last_fullsized":"2024-09-12T12:12:17.030717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025393+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:12:17.025393+0000","last_peered":"2024-09-12T12:12:17.025393+0000","last_clean":"2024-09-12T12:12:17.025393+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:12:17.025393+0000","last_undegraded":"2024-09-12T12:12:17.025393+0000","last_fullsized":"2024-09-12T12:12:17.025393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200826+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:12:17.200826+0000","last_peered":"2024-09-12T12:12:17.200826+0000","last_clean":"2024-09-12T12:12:17.200826+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:12:17.200826+0000","last_undegraded":"2024-09-12T12:12:17.200826+0000","last_fullsized":"2024-09-12T12:12:17.200826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032817+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:12:17.032817+0000","last_peered":"2024-09-12T12:12:17.032817+0000","last_clean":"2024-09-12T12:12:17.032817+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:12:17.032817+0000","last_undegraded":"2024-09-12T12:12:17.032817+0000","last_fullsized":"2024-09-12T12:12:17.032817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031314+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:12:16.031314+0000","last_peered":"2024-09-12T12:12:16.031314+0000","last_clean":"2024-09-12T12:12:16.031314+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:12:16.031314+0000","last_undegraded":"2024-09-12T12:12:16.031314+0000","last_fullsized":"2024-09-12T12:12:16.031314+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033429+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:12:17.033429+0000","last_peered":"2024-09-12T12:12:17.033429+0000","last_clean":"2024-09-12T12:12:17.033429+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:12:17.033429+0000","last_undegraded":"2024-09-12T12:12:17.033429+0000","last_fullsized":"2024-09-12T12:12:17.033429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028783+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:12:17.028783+0000","last_peered":"2024-09-12T12:12:17.028783+0000","last_clean":"2024-09-12T12:12:17.028783+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:12:17.028783+0000","last_undegraded":"2024-09-12T12:12:17.028783+0000","last_fullsized":"2024-09-12T12:12:17.028783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031113+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:12:17.031113+0000","last_peered":"2024-09-12T12:12:17.031113+0000","last_clean":"2024-09-12T12:12:17.031113+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:12:17.031113+0000","last_undegraded":"2024-09-12T12:12:17.031113+0000","last_fullsized":"2024-09-12T12:12:17.031113+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201138+0000","last_change":"2024-09-12T12:12:17.201138+0000","last_active":"2024-09-12T12:12:17.201138+0000","last_peered":"2024-09-12T12:12:17.201138+0000","last_clean":"2024-09-12T12:12:17.201138+0000","last_became_active":"2024-09-12T12:12:17.200768+0000","last_became_peered":"2024-09-12T12:12:17.200768+0000","last_unstale":"2024-09-12T12:12:17.201138+0000","last_undegraded":"2024-09-12T12:12:17.201138+0000","last_fullsized":"2024-09-12T12:12:17.201138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:48.749285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200084+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:12:17.200084+0000","last_peered":"2024-09-12T12:12:17.200084+0000","last_clean":"2024-09-12T12:12:17.200084+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:12:17.200084+0000","last_undegraded":"2024-09-12T12:12:17.200084+0000","last_fullsized":"2024-09-12T12:12:17.200084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028596+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:12:17.028596+0000","last_peered":"2024-09-12T12:12:17.028596+0000","last_clean":"2024-09-12T12:12:17.028596+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:12:17.028596+0000","last_undegraded":"2024-09-12T12:12:17.028596+0000","last_fullsized":"2024-09-12T12:12:17.028596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030334+0000","last_change":"2024-09-12T12:12:17.030334+0000","last_active":"2024-09-12T12:12:17.030334+0000","last_peered":"2024-09-12T12:12:17.030334+0000","last_clean":"2024-09-12T12:12:17.030334+0000","last_became_active":"2024-09-12T12:12:17.029947+0000","last_became_peered":"2024-09-12T12:12:17.029947+0000","last_unstale":"2024-09-12T12:12:17.030334+0000","last_undegraded":"2024-09-12T12:12:17.030334+0000","last_fullsized":"2024-09-12T12:12:17.030334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:10.299557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030670+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:12:17.030670+0000","last_peered":"2024-09-12T12:12:17.030670+0000","last_clean":"2024-09-12T12:12:17.030670+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:12:17.030670+0000","last_undegraded":"2024-09-12T12:12:17.030670+0000","last_fullsized":"2024-09-12T12:12:17.030670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033339+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:12:17.033339+0000","last_peered":"2024-09-12T12:12:17.033339+0000","last_clean":"2024-09-12T12:12:17.033339+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:12:17.033339+0000","last_undegraded":"2024-09-12T12:12:17.033339+0000","last_fullsized":"2024-09-12T12:12:17.033339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032009+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:12:16.032009+0000","last_peered":"2024-09-12T12:12:16.032009+0000","last_clean":"2024-09-12T12:12:16.032009+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:12:16.032009+0000","last_undegraded":"2024-09-12T12:12:16.032009+0000","last_fullsized":"2024-09-12T12:12:16.032009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-12T12:11:16.259126+0000","last_change":"2024-09-12T12:12:16.032862+0000","last_active":"2024-09-12T12:12:16.032862+0000","last_peered":"2024-09-12T12:11:16.259126+0000","last_clean":"2024-09-12T12:11:16.259126+0000","last_became_active":"2024-09-12T12:11:12.507860+0000","last_became_peered":"2024-09-12T12:11:12.507860+0000","last_unstale":"2024-09-12T12:11:16.259126+0000","last_undegraded":"2024-09-12T12:11:16.259126+0000","last_fullsized":"2024-09-12T12:11:16.259126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029242+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:12:17.029242+0000","last_peered":"2024-09-12T12:12:17.029242+0000","last_clean":"2024-09-12T12:12:17.029242+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:12:17.029242+0000","last_undegraded":"2024-09-12T12:12:17.029242+0000","last_fullsized":"2024-09-12T12:12:17.029242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025082+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:12:17.025082+0000","last_peered":"2024-09-12T12:12:17.025082+0000","last_clean":"2024-09-12T12:12:17.025082+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:12:17.025082+0000","last_undegraded":"2024-09-12T12:12:17.025082+0000","last_fullsized":"2024-09-12T12:12:17.025082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033339+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:12:17.033339+0000","last_peered":"2024-09-12T12:12:17.033339+0000","last_clean":"2024-09-12T12:12:17.033339+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:12:17.033339+0000","last_undegraded":"2024-09-12T12:12:17.033339+0000","last_fullsized":"2024-09-12T12:12:17.033339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032731+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:12:16.032731+0000","last_peered":"2024-09-12T12:12:16.032731+0000","last_clean":"2024-09-12T12:12:16.032731+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:12:16.032731+0000","last_undegraded":"2024-09-12T12:12:16.032731+0000","last_fullsized":"2024-09-12T12:12:16.032731+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028352+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:12:17.028352+0000","last_peered":"2024-09-12T12:12:17.028352+0000","last_clean":"2024-09-12T12:12:17.028352+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:12:17.028352+0000","last_undegraded":"2024-09-12T12:12:17.028352+0000","last_fullsized":"2024-09-12T12:12:17.028352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":83,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029335+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:12:17.029335+0000","last_peered":"2024-09-12T12:12:17.029335+0000","last_clean":"2024-09-12T12:12:17.029335+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:12:17.029335+0000","last_undegraded":"2024-09-12T12:12:17.029335+0000","last_fullsized":"2024-09-12T12:12:17.029335+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028966+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:12:17.028966+0000","last_peered":"2024-09-12T12:12:17.028966+0000","last_clean":"2024-09-12T12:12:17.028966+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:12:17.028966+0000","last_undegraded":"2024-09-12T12:12:17.028966+0000","last_fullsized":"2024-09-12T12:12:17.028966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200368+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:12:17.200368+0000","last_peered":"2024-09-12T12:12:17.200368+0000","last_clean":"2024-09-12T12:12:17.200368+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:12:17.200368+0000","last_undegraded":"2024-09-12T12:12:17.200368+0000","last_fullsized":"2024-09-12T12:12:17.200368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032201+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:12:16.032201+0000","last_peered":"2024-09-12T12:12:16.032201+0000","last_clean":"2024-09-12T12:12:16.032201+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:12:16.032201+0000","last_undegraded":"2024-09-12T12:12:16.032201+0000","last_fullsized":"2024-09-12T12:12:16.032201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200747+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:12:17.200747+0000","last_peered":"2024-09-12T12:12:17.200747+0000","last_clean":"2024-09-12T12:12:17.200747+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:12:17.200747+0000","last_undegraded":"2024-09-12T12:12:17.200747+0000","last_fullsized":"2024-09-12T12:12:17.200747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":170,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.544336+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:12:17.544336+0000","last_peered":"2024-09-12T12:12:17.544336+0000","last_clean":"2024-09-12T12:12:17.544336+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:12:17.544336+0000","last_undegraded":"2024-09-12T12:12:17.544336+0000","last_fullsized":"2024-09-12T12:12:17.544336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":117,"num_write":27,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034261+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:12:17.034261+0000","last_peered":"2024-09-12T12:12:17.034261+0000","last_clean":"2024-09-12T12:12:17.034261+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:12:17.034261+0000","last_undegraded":"2024-09-12T12:12:17.034261+0000","last_fullsized":"2024-09-12T12:12:17.034261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028566+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:12:17.028566+0000","last_peered":"2024-09-12T12:12:17.028566+0000","last_clean":"2024-09-12T12:12:17.028566+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:12:17.028566+0000","last_undegraded":"2024-09-12T12:12:17.028566+0000","last_fullsized":"2024-09-12T12:12:17.028566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025123+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:12:17.025123+0000","last_peered":"2024-09-12T12:12:17.025123+0000","last_clean":"2024-09-12T12:12:17.025123+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:12:17.025123+0000","last_undegraded":"2024-09-12T12:12:17.025123+0000","last_fullsized":"2024-09-12T12:12:17.025123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031888+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:12:16.031888+0000","last_peered":"2024-09-12T12:12:16.031888+0000","last_clean":"2024-09-12T12:12:16.031888+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:12:16.031888+0000","last_undegraded":"2024-09-12T12:12:16.031888+0000","last_fullsized":"2024-09-12T12:12:16.031888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029013+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:12:17.029013+0000","last_peered":"2024-09-12T12:12:17.029013+0000","last_clean":"2024-09-12T12:12:17.029013+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:12:17.029013+0000","last_undegraded":"2024-09-12T12:12:17.029013+0000","last_fullsized":"2024-09-12T12:12:17.029013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":89,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:18.546255+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:12:18.546255+0000","last_peered":"2024-09-12T12:12:18.546255+0000","last_clean":"2024-09-12T12:12:18.546255+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:12:18.546255+0000","last_undegraded":"2024-09-12T12:12:18.546255+0000","last_fullsized":"2024-09-12T12:12:18.546255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031317+0000","last_change":"2024-09-12T12:12:17.031317+0000","last_active":"2024-09-12T12:12:17.031317+0000","last_peered":"2024-09-12T12:12:17.031317+0000","last_clean":"2024-09-12T12:12:17.031317+0000","last_became_active":"2024-09-12T12:12:17.030842+0000","last_became_peered":"2024-09-12T12:12:17.030842+0000","last_unstale":"2024-09-12T12:12:17.031317+0000","last_undegraded":"2024-09-12T12:12:17.031317+0000","last_fullsized":"2024-09-12T12:12:17.031317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:11.616682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033026+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:12:17.033026+0000","last_peered":"2024-09-12T12:12:17.033026+0000","last_clean":"2024-09-12T12:12:17.033026+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:12:17.033026+0000","last_undegraded":"2024-09-12T12:12:17.033026+0000","last_fullsized":"2024-09-12T12:12:17.033026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032581+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:12:16.032581+0000","last_peered":"2024-09-12T12:12:16.032581+0000","last_clean":"2024-09-12T12:12:16.032581+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:12:16.032581+0000","last_undegraded":"2024-09-12T12:12:16.032581+0000","last_fullsized":"2024-09-12T12:12:16.032581+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034442+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:12:17.034442+0000","last_peered":"2024-09-12T12:12:17.034442+0000","last_clean":"2024-09-12T12:12:17.034442+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:12:17.034442+0000","last_undegraded":"2024-09-12T12:12:17.034442+0000","last_fullsized":"2024-09-12T12:12:17.034442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031307+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:12:17.031307+0000","last_peered":"2024-09-12T12:12:17.031307+0000","last_clean":"2024-09-12T12:12:17.031307+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:12:17.031307+0000","last_undegraded":"2024-09-12T12:12:17.031307+0000","last_fullsized":"2024-09-12T12:12:17.031307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025314+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:12:17.025314+0000","last_peered":"2024-09-12T12:12:17.025314+0000","last_clean":"2024-09-12T12:12:17.025314+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:12:17.025314+0000","last_undegraded":"2024-09-12T12:12:17.025314+0000","last_fullsized":"2024-09-12T12:12:17.025314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029297+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:12:17.029297+0000","last_peered":"2024-09-12T12:12:17.029297+0000","last_clean":"2024-09-12T12:12:17.029297+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:12:17.029297+0000","last_undegraded":"2024-09-12T12:12:17.029297+0000","last_fullsized":"2024-09-12T12:12:17.029297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.032086+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:12:16.032086+0000","last_peered":"2024-09-12T12:12:16.032086+0000","last_clean":"2024-09-12T12:12:16.032086+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:12:16.032086+0000","last_undegraded":"2024-09-12T12:12:16.032086+0000","last_fullsized":"2024-09-12T12:12:16.032086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029052+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:12:17.029052+0000","last_peered":"2024-09-12T12:12:17.029052+0000","last_clean":"2024-09-12T12:12:17.029052+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:12:17.029052+0000","last_undegraded":"2024-09-12T12:12:17.029052+0000","last_fullsized":"2024-09-12T12:12:17.029052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029406+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:12:17.029406+0000","last_peered":"2024-09-12T12:12:17.029406+0000","last_clean":"2024-09-12T12:12:17.029406+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:12:17.029406+0000","last_undegraded":"2024-09-12T12:12:17.029406+0000","last_fullsized":"2024-09-12T12:12:17.029406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029338+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:12:17.029338+0000","last_peered":"2024-09-12T12:12:17.029338+0000","last_clean":"2024-09-12T12:12:17.029338+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:12:17.029338+0000","last_undegraded":"2024-09-12T12:12:17.029338+0000","last_fullsized":"2024-09-12T12:12:17.029338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029959+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:12:17.029959+0000","last_peered":"2024-09-12T12:12:17.029959+0000","last_clean":"2024-09-12T12:12:17.029959+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:12:17.029959+0000","last_undegraded":"2024-09-12T12:12:17.029959+0000","last_fullsized":"2024-09-12T12:12:17.029959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200957+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:12:17.200957+0000","last_peered":"2024-09-12T12:12:17.200957+0000","last_clean":"2024-09-12T12:12:17.200957+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:12:17.200957+0000","last_undegraded":"2024-09-12T12:12:17.200957+0000","last_fullsized":"2024-09-12T12:12:17.200957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032847+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:12:17.032847+0000","last_peered":"2024-09-12T12:12:17.032847+0000","last_clean":"2024-09-12T12:12:17.032847+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:12:17.032847+0000","last_undegraded":"2024-09-12T12:12:17.032847+0000","last_fullsized":"2024-09-12T12:12:17.032847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029901+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:12:17.029901+0000","last_peered":"2024-09-12T12:12:17.029901+0000","last_clean":"2024-09-12T12:12:17.029901+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:12:17.029901+0000","last_undegraded":"2024-09-12T12:12:17.029901+0000","last_fullsized":"2024-09-12T12:12:17.029901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025050+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:12:17.025050+0000","last_peered":"2024-09-12T12:12:17.025050+0000","last_clean":"2024-09-12T12:12:17.025050+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:12:17.025050+0000","last_undegraded":"2024-09-12T12:12:17.025050+0000","last_fullsized":"2024-09-12T12:12:17.025050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":86,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:18.545728+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:12:18.545728+0000","last_peered":"2024-09-12T12:12:18.545728+0000","last_clean":"2024-09-12T12:12:18.545728+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:12:18.545728+0000","last_undegraded":"2024-09-12T12:12:18.545728+0000","last_fullsized":"2024-09-12T12:12:18.545728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031412+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:12:16.031412+0000","last_peered":"2024-09-12T12:12:16.031412+0000","last_clean":"2024-09-12T12:12:16.031412+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:12:16.031412+0000","last_undegraded":"2024-09-12T12:12:16.031412+0000","last_fullsized":"2024-09-12T12:12:16.031412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028913+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:12:17.028913+0000","last_peered":"2024-09-12T12:12:17.028913+0000","last_clean":"2024-09-12T12:12:17.028913+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:12:17.028913+0000","last_undegraded":"2024-09-12T12:12:17.028913+0000","last_fullsized":"2024-09-12T12:12:17.028913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029412+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:12:17.029412+0000","last_peered":"2024-09-12T12:12:17.029412+0000","last_clean":"2024-09-12T12:12:17.029412+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:12:17.029412+0000","last_undegraded":"2024-09-12T12:12:17.029412+0000","last_fullsized":"2024-09-12T12:12:17.029412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028570+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:12:17.028570+0000","last_peered":"2024-09-12T12:12:17.028570+0000","last_clean":"2024-09-12T12:12:17.028570+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:12:17.028570+0000","last_undegraded":"2024-09-12T12:12:17.028570+0000","last_fullsized":"2024-09-12T12:12:17.028570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028914+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:12:17.028914+0000","last_peered":"2024-09-12T12:12:17.028914+0000","last_clean":"2024-09-12T12:12:17.028914+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:12:17.028914+0000","last_undegraded":"2024-09-12T12:12:17.028914+0000","last_fullsized":"2024-09-12T12:12:17.028914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034512+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:12:17.034512+0000","last_peered":"2024-09-12T12:12:17.034512+0000","last_clean":"2024-09-12T12:12:17.034512+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:12:17.034512+0000","last_undegraded":"2024-09-12T12:12:17.034512+0000","last_fullsized":"2024-09-12T12:12:17.034512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025398+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:12:17.025398+0000","last_peered":"2024-09-12T12:12:17.025398+0000","last_clean":"2024-09-12T12:12:17.025398+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:12:17.025398+0000","last_undegraded":"2024-09-12T12:12:17.025398+0000","last_fullsized":"2024-09-12T12:12:17.025398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028567+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:12:17.028567+0000","last_peered":"2024-09-12T12:12:17.028567+0000","last_clean":"2024-09-12T12:12:17.028567+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:12:17.028567+0000","last_undegraded":"2024-09-12T12:12:17.028567+0000","last_fullsized":"2024-09-12T12:12:17.028567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032777+0000","last_change":"2024-09-12T12:12:17.032314+0000","last_active":"2024-09-12T12:12:17.032777+0000","last_peered":"2024-09-12T12:12:17.032777+0000","last_clean":"2024-09-12T12:12:17.032777+0000","last_became_active":"2024-09-12T12:12:17.032022+0000","last_became_peered":"2024-09-12T12:12:17.032022+0000","last_unstale":"2024-09-12T12:12:17.032777+0000","last_undegraded":"2024-09-12T12:12:17.032777+0000","last_fullsized":"2024-09-12T12:12:17.032777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:55.111688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201240+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:12:17.201240+0000","last_peered":"2024-09-12T12:12:17.201240+0000","last_clean":"2024-09-12T12:12:17.201240+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:12:17.201240+0000","last_undegraded":"2024-09-12T12:12:17.201240+0000","last_fullsized":"2024-09-12T12:12:17.201240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200202+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:12:17.200202+0000","last_peered":"2024-09-12T12:12:17.200202+0000","last_clean":"2024-09-12T12:12:17.200202+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:12:17.200202+0000","last_undegraded":"2024-09-12T12:12:17.200202+0000","last_fullsized":"2024-09-12T12:12:17.200202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200521+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:12:17.200521+0000","last_peered":"2024-09-12T12:12:17.200521+0000","last_clean":"2024-09-12T12:12:17.200521+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:12:17.200521+0000","last_undegraded":"2024-09-12T12:12:17.200521+0000","last_fullsized":"2024-09-12T12:12:17.200521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033160+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:12:17.033160+0000","last_peered":"2024-09-12T12:12:17.033160+0000","last_clean":"2024-09-12T12:12:17.033160+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:12:17.033160+0000","last_undegraded":"2024-09-12T12:12:17.033160+0000","last_fullsized":"2024-09-12T12:12:17.033160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032969+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:12:17.032969+0000","last_peered":"2024-09-12T12:12:17.032969+0000","last_clean":"2024-09-12T12:12:17.032969+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:12:17.032969+0000","last_undegraded":"2024-09-12T12:12:17.032969+0000","last_fullsized":"2024-09-12T12:12:17.032969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030761+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:12:17.030761+0000","last_peered":"2024-09-12T12:12:17.030761+0000","last_clean":"2024-09-12T12:12:17.030761+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:12:17.030761+0000","last_undegraded":"2024-09-12T12:12:17.030761+0000","last_fullsized":"2024-09-12T12:12:17.030761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.039006+0000","last_change":"2024-09-12T12:12:17.038499+0000","last_active":"2024-09-12T12:12:17.039006+0000","last_peered":"2024-09-12T12:12:17.039006+0000","last_clean":"2024-09-12T12:12:17.039006+0000","last_became_active":"2024-09-12T12:12:17.034622+0000","last_became_peered":"2024-09-12T12:12:17.034622+0000","last_unstale":"2024-09-12T12:12:17.039006+0000","last_undegraded":"2024-09-12T12:12:17.039006+0000","last_fullsized":"2024-09-12T12:12:17.039006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:30.684068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033360+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:12:17.033360+0000","last_peered":"2024-09-12T12:12:17.033360+0000","last_clean":"2024-09-12T12:12:17.033360+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:12:17.033360+0000","last_undegraded":"2024-09-12T12:12:17.033360+0000","last_fullsized":"2024-09-12T12:12:17.033360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.042021+0000","last_change":"2024-09-12T12:12:17.041616+0000","last_active":"2024-09-12T12:12:17.042021+0000","last_peered":"2024-09-12T12:12:17.042021+0000","last_clean":"2024-09-12T12:12:17.042021+0000","last_became_active":"2024-09-12T12:12:17.035895+0000","last_became_peered":"2024-09-12T12:12:17.035895+0000","last_unstale":"2024-09-12T12:12:17.042021+0000","last_undegraded":"2024-09-12T12:12:17.042021+0000","last_fullsized":"2024-09-12T12:12:17.042021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:11.722682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028769+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:12:17.028769+0000","last_peered":"2024-09-12T12:12:17.028769+0000","last_clean":"2024-09-12T12:12:17.028769+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:12:17.028769+0000","last_undegraded":"2024-09-12T12:12:17.028769+0000","last_fullsized":"2024-09-12T12:12:17.028769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T12:12:16.031761+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:12:16.031761+0000","last_peered":"2024-09-12T12:12:16.031761+0000","last_clean":"2024-09-12T12:12:16.031761+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:12:16.031761+0000","last_undegraded":"2024-09-12T12:12:16.031761+0000","last_fullsized":"2024-09-12T12:12:16.031761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034217+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:12:17.034217+0000","last_peered":"2024-09-12T12:12:17.034217+0000","last_clean":"2024-09-12T12:12:17.034217+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:12:17.034217+0000","last_undegraded":"2024-09-12T12:12:17.034217+0000","last_fullsized":"2024-09-12T12:12:17.034217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200709+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:12:17.200709+0000","last_peered":"2024-09-12T12:12:17.200709+0000","last_clean":"2024-09-12T12:12:17.200709+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:12:17.200709+0000","last_undegraded":"2024-09-12T12:12:17.200709+0000","last_fullsized":"2024-09-12T12:12:17.200709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034393+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:12:17.034393+0000","last_peered":"2024-09-12T12:12:17.034393+0000","last_clean":"2024-09-12T12:12:17.034393+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:12:17.034393+0000","last_undegraded":"2024-09-12T12:12:17.034393+0000","last_fullsized":"2024-09-12T12:12:17.034393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200361+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:12:17.200361+0000","last_peered":"2024-09-12T12:12:17.200361+0000","last_clean":"2024-09-12T12:12:17.200361+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:12:17.200361+0000","last_undegraded":"2024-09-12T12:12:17.200361+0000","last_fullsized":"2024-09-12T12:12:17.200361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033161+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:12:17.033161+0000","last_peered":"2024-09-12T12:12:17.033161+0000","last_clean":"2024-09-12T12:12:17.033161+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:12:17.033161+0000","last_undegraded":"2024-09-12T12:12:17.033161+0000","last_fullsized":"2024-09-12T12:12:17.033161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032939+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:12:17.032939+0000","last_peered":"2024-09-12T12:12:17.032939+0000","last_clean":"2024-09-12T12:12:17.032939+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:12:17.032939+0000","last_undegraded":"2024-09-12T12:12:17.032939+0000","last_fullsized":"2024-09-12T12:12:17.032939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028701+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:12:17.028701+0000","last_peered":"2024-09-12T12:12:17.028701+0000","last_clean":"2024-09-12T12:12:17.028701+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:12:17.028701+0000","last_undegraded":"2024-09-12T12:12:17.028701+0000","last_fullsized":"2024-09-12T12:12:17.028701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":363,"num_read_kb":248,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":444201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":1095286,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939686,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":444437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168619,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":435927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397552,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1036289,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":443826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790009,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1018565,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":444656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":504180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":502830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:12:22.645 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-12T12:12:22.645 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 82, 'state': 'peering', 'last_fresh': '2024-09-12T12:11:16.259126+0000', 'last_change': '2024-09-12T12:12:16.032862+0000', 'last_active': '2024-09-12T12:12:16.032862+0000', 'last_peered': '2024-09-12T12:11:16.259126+0000', 'last_clean': '2024-09-12T12:11:16.259126+0000', 'last_became_active': '2024-09-12T12:11:12.507860+0000', 'last_became_peered': '2024-09-12T12:11:12.507860+0000', 'last_unstale': '2024-09-12T12:11:16.259126+0000', 'last_undegraded': '2024-09-12T12:11:16.259126+0000', 'last_fullsized': '2024-09-12T12:11:16.259126+0000', 'mapping_epoch': 83, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 81, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T12:11:11.470920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T12:11:11.470920+0000', 'last_clean_scrub_stamp': '2024-09-12T12:11:11.470920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 8, 3], 'acting': [5, 8, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3, 4, 8], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T12:12:22.645 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-12T12:12:22.896 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:23.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:23 smithi137 ceph-mon[30950]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:23.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:23 smithi189 ceph-mon[35475]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:23.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:23 smithi171 ceph-mon[35098]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1658790079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0100e20 msgr2=0x7f9cc0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1658790079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0100e20 0x7f9cc0101200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009900 tx=0x7f9cbc02a480 comp rx=0 tx=0).stop 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1658790079 shutdown_connections 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1658790079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cc0102ef0 0x7f9cc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1658790079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc0102570 0x7f9cc01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1658790079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0100e20 0x7f9cc0101200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1658790079 >> 172.21.15.137:0/1658790079 conn(0x7f9cc00fc7d0 msgr2=0x7f9cc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1658790079 shutdown_connections 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.461+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1658790079 wait complete. 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 Processor -- start 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 -- start start 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc0100e20 0x7f9cc010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cc0102570 0x7f9cc010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0102ef0 0x7f9cc010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc01120a0 con 0x7f9cc0102ef0 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0111f20 con 0x7f9cc0102570 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc7ec4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0112220 con 0x7f9cc0100e20 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0102ef0 0x7f9cc010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0102ef0 0x7f9cc010ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56730/0 (socket says 172.21.15.137:56730) 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 -- 172.21.15.137:0/1345079695 learned_addr learned my addr 172.21.15.137:0/1345079695 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc5c60640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc0100e20 0x7f9cc010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 -- 172.21.15.137:0/1345079695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc0100e20 msgr2=0x7f9cc010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc0100e20 0x7f9cc010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 -- 172.21.15.137:0/1345079695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cc0102570 msgr2=0x7f9cc010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cc0102570 0x7f9cc010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 -- 172.21.15.137:0/1345079695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc01a6a60 con 0x7f9cc0102ef0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc5c60640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc0100e20 0x7f9cc010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.462+0000 7f9cc6461640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0102ef0 0x7f9cc010ef50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800eaa0 tx=0x7f9cb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.463+0000 7f9cb6ffd640 1 -- 172.21.15.137:0/1345079695 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800cdb0 con 0x7f9cc0102ef0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.463+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc01a6d50 con 0x7f9cc0102ef0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.463+0000 7f9cb6ffd640 1 -- 172.21.15.137:0/1345079695 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb8005020 con 0x7f9cc0102ef0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.463+0000 7f9cb6ffd640 1 -- 172.21.15.137:0/1345079695 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80116b0 con 0x7f9cc0102ef0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.463+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc01a7240 con 0x7f9cc0102ef0 2024-09-12T12:12:23.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.464+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc0101200 con 0x7f9cc0102ef0 2024-09-12T12:12:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.465+0000 7f9cb6ffd640 1 -- 172.21.15.137:0/1345079695 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9cb8005560 con 0x7f9cc0102ef0 2024-09-12T12:12:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.466+0000 7f9cb6ffd640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9cac0a2c10 0x7f9cac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.466+0000 7f9cc5c60640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9cac0a2c10 0x7f9cac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.466+0000 7f9cb6ffd640 1 -- 172.21.15.137:0/1345079695 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f9cb80cf7f0 con 0x7f9cc0102ef0 2024-09-12T12:12:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.466+0000 7f9cc5c60640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9cac0a2c10 0x7f9cac0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc000c00 tx=0x7f9cbc004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.469+0000 7f9cb6ffd640 1 -- 172.21.15.137:0/1345079695 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9cb8016030 con 0x7f9cc0102ef0 2024-09-12T12:12:23.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.700+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9cc010fa00 con 0x7f9cc0102ef0 2024-09-12T12:12:23.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.701+0000 7f9cb6ffd640 1 -- 172.21.15.137:0/1345079695 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2563 (secure 0 0 0) 0x7f9cb809c620 con 0x7f9cc0102ef0 2024-09-12T12:12:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:23.702 INFO:teuthology.orchestra.run.smithi137.stdout:{"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":440,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1726143019,"num_in_osds":12,"osd_in_since":1726142996,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":96},{"state_name":"peering","count":1}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":653816,"bytes_used":364851200,"bytes_avail":1151575576576,"bytes_total":1151940427776,"inactive_pgs_ratio":0.010309278033673763,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"read_bytes_sec":3583,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":23,"btime":"2024-09-12T12:11:24:550801+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24427},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24446},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24448},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24454},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24475},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24476}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.137:8443/"}},"servicemap":{"epoch":25,"modified":"2024-09-12T12:12:19.125289+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T12:12:23.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9cac0a2c10 msgr2=0x7f9cac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:23.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9cac0a2c10 0x7f9cac0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc000c00 tx=0x7f9cbc004100 comp rx=0 tx=0).stop 2024-09-12T12:12:23.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0102ef0 msgr2=0x7f9cc010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:23.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0102ef0 0x7f9cc010ef50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800eaa0 tx=0x7f9cb800ef60 comp rx=0 tx=0).stop 2024-09-12T12:12:23.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 shutdown_connections 2024-09-12T12:12:23.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9cac0a2c10 0x7f9cac0a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc0102ef0 0x7f9cc010ef50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9cc0102570 0x7f9cc010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 --2- 172.21.15.137:0/1345079695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc0100e20 0x7f9cc010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:23.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 >> 172.21.15.137:0/1345079695 conn(0x7f9cc00fc7d0 msgr2=0x7f9cc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:23.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 shutdown_connections 2024-09-12T12:12:23.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:23.703+0000 7f9cc7ec4640 1 -- 172.21.15.137:0/1345079695 wait complete. 2024-09-12T12:12:24.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:24 smithi189 ceph-mon[35475]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:24.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:24 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1345079695' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T12:12:24.351 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:24 smithi171 ceph-mon[35098]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:24.351 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:24 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1345079695' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T12:12:24.387 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:24 smithi137 ceph-mon[30950]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:24.387 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:24 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1345079695' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T12:12:24.418 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-12T12:12:24.418 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:12:24.672 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 -- 172.21.15.137:0/739062597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 msgr2=0x7fafac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/739062597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac1029b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7faf94009900 tx=0x7faf9402a480 comp rx=0 tx=0).stop 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 -- 172.21.15.137:0/739062597 shutdown_connections 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/739062597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafac102ef0 0x7fafac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/739062597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac1029b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/739062597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafac100e20 0x7fafac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 -- 172.21.15.137:0/739062597 >> 172.21.15.137:0/739062597 conn(0x7fafac0fc7d0 msgr2=0x7fafac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:25.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 -- 172.21.15.137:0/739062597 shutdown_connections 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.156+0000 7fafb2cdd640 1 -- 172.21.15.137:0/739062597 wait complete. 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 Processor -- start 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 -- start start 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafac100e20 0x7fafac1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafac102ef0 0x7fafac19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fafac1120a0 con 0x7fafac102570 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fafac111f20 con 0x7fafac100e20 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb2cdd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fafac112220 con 0x7fafac102ef0 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafabfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafabfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56746/0 (socket says 172.21.15.137:56746) 2024-09-12T12:12:25.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafabfff640 1 -- 172.21.15.137:0/3090756703 learned_addr learned my addr 172.21.15.137:0/3090756703 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb0a79640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafac100e20 0x7fafac1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.157+0000 7fafb127a640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafac102ef0 0x7fafac19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafabfff640 1 -- 172.21.15.137:0/3090756703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafac102ef0 msgr2=0x7fafac19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafabfff640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafac102ef0 0x7fafac19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafabfff640 1 -- 172.21.15.137:0/3090756703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafac100e20 msgr2=0x7fafac1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafabfff640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafac100e20 0x7fafac1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafabfff640 1 -- 172.21.15.137:0/3090756703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac19d570 con 0x7fafac102570 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafabfff640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac198b00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf94006010 tx=0x7faf94003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafa9ffb640 1 -- 172.21.15.137:0/3090756703 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9403f070 con 0x7fafac102570 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafa9ffb640 1 -- 172.21.15.137:0/3090756703 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf94031c30 con 0x7fafac102570 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac19d800 con 0x7fafac102570 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafa9ffb640 1 -- 172.21.15.137:0/3090756703 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94035030 con 0x7fafac102570 2024-09-12T12:12:25.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.158+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac1a5140 con 0x7fafac102570 2024-09-12T12:12:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.159+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafac1021b0 con 0x7fafac102570 2024-09-12T12:12:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.160+0000 7fafa9ffb640 1 -- 172.21.15.137:0/3090756703 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7faf94042070 con 0x7fafac102570 2024-09-12T12:12:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.161+0000 7fafa9ffb640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf980a2c10 0x7faf980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.161+0000 7fafb0a79640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf980a2c10 0x7faf980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.161+0000 7fafa9ffb640 1 -- 172.21.15.137:0/3090756703 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7faf940efd70 con 0x7fafac102570 2024-09-12T12:12:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.162+0000 7fafb0a79640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf980a2c10 0x7faf980a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.164+0000 7fafa9ffb640 1 -- 172.21.15.137:0/3090756703 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf94031de0 con 0x7fafac102570 2024-09-12T12:12:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.331+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafac062630 con 0x7faf980a2c10 2024-09-12T12:12:25.332 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:25 smithi137 ceph-mon[30950]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T12:12:25.332 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:25 smithi137 ceph-mon[30950]: Cluster is now healthy 2024-09-12T12:12:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.337+0000 7fafa9ffb640 1 -- 172.21.15.137:0/3090756703 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258872 (secure 0 0 0) 0x7fafac062630 con 0x7faf980a2c10 2024-09-12T12:12:25.340 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:25.341 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:12:25.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf980a2c10 msgr2=0x7faf980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:25.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf980a2c10 0x7faf980a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c009760 comp rx=0 tx=0).stop 2024-09-12T12:12:25.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 msgr2=0x7fafac198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:25.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac198b00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf94006010 tx=0x7faf94003c10 comp rx=0 tx=0).stop 2024-09-12T12:12:25.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 shutdown_connections 2024-09-12T12:12:25.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7faf980a2c10 0x7faf980a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafac102ef0 0x7fafac19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fafac102570 0x7fafac198b00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 --2- 172.21.15.137:0/3090756703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fafac100e20 0x7fafac1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:25.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 >> 172.21.15.137:0/3090756703 conn(0x7fafac0fc7d0 msgr2=0x7fafac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:25.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 shutdown_connections 2024-09-12T12:12:25.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:25.340+0000 7fafb2cdd640 1 -- 172.21.15.137:0/3090756703 wait complete. 2024-09-12T12:12:25.344 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:25 smithi189 ceph-mon[35475]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T12:12:25.344 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:25 smithi189 ceph-mon[35475]: Cluster is now healthy 2024-09-12T12:12:25.364 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:25 smithi171 ceph-mon[35098]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T12:12:25.364 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:25 smithi171 ceph-mon[35098]: Cluster is now healthy 2024-09-12T12:12:26.100 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-12T12:12:25.126689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653816,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":519,"num_read_kb":377,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356344,"kb_used_data":11184,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585480,"statfs":{"total":1151940427776,"available":1151575531520,"internally_reserved":0,"allocated":11452416,"data_stored":7284876,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":23,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.102580"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031172+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:12:17.031172+0000","last_peered":"2024-09-12T12:12:17.031172+0000","last_clean":"2024-09-12T12:12:17.031172+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:12:17.031172+0000","last_undegraded":"2024-09-12T12:12:17.031172+0000","last_fullsized":"2024-09-12T12:12:17.031172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200200+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:12:17.200200+0000","last_peered":"2024-09-12T12:12:17.200200+0000","last_clean":"2024-09-12T12:12:17.200200+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:12:17.200200+0000","last_undegraded":"2024-09-12T12:12:17.200200+0000","last_fullsized":"2024-09-12T12:12:17.200200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200138+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:12:17.200138+0000","last_peered":"2024-09-12T12:12:17.200138+0000","last_clean":"2024-09-12T12:12:17.200138+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:12:17.200138+0000","last_undegraded":"2024-09-12T12:12:17.200138+0000","last_fullsized":"2024-09-12T12:12:17.200138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200871+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:12:17.200871+0000","last_peered":"2024-09-12T12:12:17.200871+0000","last_clean":"2024-09-12T12:12:17.200871+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:12:17.200871+0000","last_undegraded":"2024-09-12T12:12:17.200871+0000","last_fullsized":"2024-09-12T12:12:17.200871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034055+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:12:17.034055+0000","last_peered":"2024-09-12T12:12:17.034055+0000","last_clean":"2024-09-12T12:12:17.034055+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:12:17.034055+0000","last_undegraded":"2024-09-12T12:12:17.034055+0000","last_fullsized":"2024-09-12T12:12:17.034055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029133+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:12:17.029133+0000","last_peered":"2024-09-12T12:12:17.029133+0000","last_clean":"2024-09-12T12:12:17.029133+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:12:17.029133+0000","last_undegraded":"2024-09-12T12:12:17.029133+0000","last_fullsized":"2024-09-12T12:12:17.029133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028562+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:12:17.028562+0000","last_peered":"2024-09-12T12:12:17.028562+0000","last_clean":"2024-09-12T12:12:17.028562+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:12:17.028562+0000","last_undegraded":"2024-09-12T12:12:17.028562+0000","last_fullsized":"2024-09-12T12:12:17.028562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033265+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:12:17.033265+0000","last_peered":"2024-09-12T12:12:17.033265+0000","last_clean":"2024-09-12T12:12:17.033265+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:12:17.033265+0000","last_undegraded":"2024-09-12T12:12:17.033265+0000","last_fullsized":"2024-09-12T12:12:17.033265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029456+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:12:17.029456+0000","last_peered":"2024-09-12T12:12:17.029456+0000","last_clean":"2024-09-12T12:12:17.029456+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:12:17.029456+0000","last_undegraded":"2024-09-12T12:12:17.029456+0000","last_fullsized":"2024-09-12T12:12:17.029456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028748+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:12:17.028748+0000","last_peered":"2024-09-12T12:12:17.028748+0000","last_clean":"2024-09-12T12:12:17.028748+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:12:17.028748+0000","last_undegraded":"2024-09-12T12:12:17.028748+0000","last_fullsized":"2024-09-12T12:12:17.028748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200997+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:12:17.200997+0000","last_peered":"2024-09-12T12:12:17.200997+0000","last_clean":"2024-09-12T12:12:17.200997+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:12:17.200997+0000","last_undegraded":"2024-09-12T12:12:17.200997+0000","last_fullsized":"2024-09-12T12:12:17.200997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025292+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:12:17.025292+0000","last_peered":"2024-09-12T12:12:17.025292+0000","last_clean":"2024-09-12T12:12:17.025292+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:12:17.025292+0000","last_undegraded":"2024-09-12T12:12:17.025292+0000","last_fullsized":"2024-09-12T12:12:17.025292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200027+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:12:17.200027+0000","last_peered":"2024-09-12T12:12:17.200027+0000","last_clean":"2024-09-12T12:12:17.200027+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:12:17.200027+0000","last_undegraded":"2024-09-12T12:12:17.200027+0000","last_fullsized":"2024-09-12T12:12:17.200027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.199927+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:12:17.199927+0000","last_peered":"2024-09-12T12:12:17.199927+0000","last_clean":"2024-09-12T12:12:17.199927+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:12:17.199927+0000","last_undegraded":"2024-09-12T12:12:17.199927+0000","last_fullsized":"2024-09-12T12:12:17.199927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200399+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:12:17.200399+0000","last_peered":"2024-09-12T12:12:17.200399+0000","last_clean":"2024-09-12T12:12:17.200399+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:12:17.200399+0000","last_undegraded":"2024-09-12T12:12:17.200399+0000","last_fullsized":"2024-09-12T12:12:17.200399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031383+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:12:17.031383+0000","last_peered":"2024-09-12T12:12:17.031383+0000","last_clean":"2024-09-12T12:12:17.031383+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:12:17.031383+0000","last_undegraded":"2024-09-12T12:12:17.031383+0000","last_fullsized":"2024-09-12T12:12:17.031383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028526+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:12:17.028526+0000","last_peered":"2024-09-12T12:12:17.028526+0000","last_clean":"2024-09-12T12:12:17.028526+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:12:17.028526+0000","last_undegraded":"2024-09-12T12:12:17.028526+0000","last_fullsized":"2024-09-12T12:12:17.028526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028525+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:12:17.028525+0000","last_peered":"2024-09-12T12:12:17.028525+0000","last_clean":"2024-09-12T12:12:17.028525+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:12:17.028525+0000","last_undegraded":"2024-09-12T12:12:17.028525+0000","last_fullsized":"2024-09-12T12:12:17.028525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030717+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:12:17.030717+0000","last_peered":"2024-09-12T12:12:17.030717+0000","last_clean":"2024-09-12T12:12:17.030717+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:12:17.030717+0000","last_undegraded":"2024-09-12T12:12:17.030717+0000","last_fullsized":"2024-09-12T12:12:17.030717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025393+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:12:17.025393+0000","last_peered":"2024-09-12T12:12:17.025393+0000","last_clean":"2024-09-12T12:12:17.025393+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:12:17.025393+0000","last_undegraded":"2024-09-12T12:12:17.025393+0000","last_fullsized":"2024-09-12T12:12:17.025393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200826+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:12:17.200826+0000","last_peered":"2024-09-12T12:12:17.200826+0000","last_clean":"2024-09-12T12:12:17.200826+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:12:17.200826+0000","last_undegraded":"2024-09-12T12:12:17.200826+0000","last_fullsized":"2024-09-12T12:12:17.200826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032817+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:12:17.032817+0000","last_peered":"2024-09-12T12:12:17.032817+0000","last_clean":"2024-09-12T12:12:17.032817+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:12:17.032817+0000","last_undegraded":"2024-09-12T12:12:17.032817+0000","last_fullsized":"2024-09-12T12:12:17.032817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026551+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:12:17.026551+0000","last_peered":"2024-09-12T12:12:17.026551+0000","last_clean":"2024-09-12T12:12:17.026551+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:12:17.026551+0000","last_undegraded":"2024-09-12T12:12:17.026551+0000","last_fullsized":"2024-09-12T12:12:17.026551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033429+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:12:17.033429+0000","last_peered":"2024-09-12T12:12:17.033429+0000","last_clean":"2024-09-12T12:12:17.033429+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:12:17.033429+0000","last_undegraded":"2024-09-12T12:12:17.033429+0000","last_fullsized":"2024-09-12T12:12:17.033429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028783+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:12:17.028783+0000","last_peered":"2024-09-12T12:12:17.028783+0000","last_clean":"2024-09-12T12:12:17.028783+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:12:17.028783+0000","last_undegraded":"2024-09-12T12:12:17.028783+0000","last_fullsized":"2024-09-12T12:12:17.028783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031113+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:12:17.031113+0000","last_peered":"2024-09-12T12:12:17.031113+0000","last_clean":"2024-09-12T12:12:17.031113+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:12:17.031113+0000","last_undegraded":"2024-09-12T12:12:17.031113+0000","last_fullsized":"2024-09-12T12:12:17.031113+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201138+0000","last_change":"2024-09-12T12:12:17.201138+0000","last_active":"2024-09-12T12:12:17.201138+0000","last_peered":"2024-09-12T12:12:17.201138+0000","last_clean":"2024-09-12T12:12:17.201138+0000","last_became_active":"2024-09-12T12:12:17.200768+0000","last_became_peered":"2024-09-12T12:12:17.200768+0000","last_unstale":"2024-09-12T12:12:17.201138+0000","last_undegraded":"2024-09-12T12:12:17.201138+0000","last_fullsized":"2024-09-12T12:12:17.201138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:48.749285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200084+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:12:17.200084+0000","last_peered":"2024-09-12T12:12:17.200084+0000","last_clean":"2024-09-12T12:12:17.200084+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:12:17.200084+0000","last_undegraded":"2024-09-12T12:12:17.200084+0000","last_fullsized":"2024-09-12T12:12:17.200084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028596+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:12:17.028596+0000","last_peered":"2024-09-12T12:12:17.028596+0000","last_clean":"2024-09-12T12:12:17.028596+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:12:17.028596+0000","last_undegraded":"2024-09-12T12:12:17.028596+0000","last_fullsized":"2024-09-12T12:12:17.028596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030334+0000","last_change":"2024-09-12T12:12:17.030334+0000","last_active":"2024-09-12T12:12:17.030334+0000","last_peered":"2024-09-12T12:12:17.030334+0000","last_clean":"2024-09-12T12:12:17.030334+0000","last_became_active":"2024-09-12T12:12:17.029947+0000","last_became_peered":"2024-09-12T12:12:17.029947+0000","last_unstale":"2024-09-12T12:12:17.030334+0000","last_undegraded":"2024-09-12T12:12:17.030334+0000","last_fullsized":"2024-09-12T12:12:17.030334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:10.299557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030670+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:12:17.030670+0000","last_peered":"2024-09-12T12:12:17.030670+0000","last_clean":"2024-09-12T12:12:17.030670+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:12:17.030670+0000","last_undegraded":"2024-09-12T12:12:17.030670+0000","last_fullsized":"2024-09-12T12:12:17.030670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033339+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:12:17.033339+0000","last_peered":"2024-09-12T12:12:17.033339+0000","last_clean":"2024-09-12T12:12:17.033339+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:12:17.033339+0000","last_undegraded":"2024-09-12T12:12:17.033339+0000","last_fullsized":"2024-09-12T12:12:17.033339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":90,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:19.547483+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:12:19.547483+0000","last_peered":"2024-09-12T12:12:19.547483+0000","last_clean":"2024-09-12T12:12:19.547483+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:12:19.547483+0000","last_undegraded":"2024-09-12T12:12:19.547483+0000","last_fullsized":"2024-09-12T12:12:19.547483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.037302+0000","last_change":"2024-09-12T12:12:17.037302+0000","last_active":"2024-09-12T12:12:17.037302+0000","last_peered":"2024-09-12T12:12:17.037302+0000","last_clean":"2024-09-12T12:12:17.037302+0000","last_became_active":"2024-09-12T12:12:17.036907+0000","last_became_peered":"2024-09-12T12:12:17.036907+0000","last_unstale":"2024-09-12T12:12:17.037302+0000","last_undegraded":"2024-09-12T12:12:17.037302+0000","last_fullsized":"2024-09-12T12:12:17.037302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:06:37.061690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029242+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:12:17.029242+0000","last_peered":"2024-09-12T12:12:17.029242+0000","last_clean":"2024-09-12T12:12:17.029242+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:12:17.029242+0000","last_undegraded":"2024-09-12T12:12:17.029242+0000","last_fullsized":"2024-09-12T12:12:17.029242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025082+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:12:17.025082+0000","last_peered":"2024-09-12T12:12:17.025082+0000","last_clean":"2024-09-12T12:12:17.025082+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:12:17.025082+0000","last_undegraded":"2024-09-12T12:12:17.025082+0000","last_fullsized":"2024-09-12T12:12:17.025082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033339+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:12:17.033339+0000","last_peered":"2024-09-12T12:12:17.033339+0000","last_clean":"2024-09-12T12:12:17.033339+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:12:17.033339+0000","last_undegraded":"2024-09-12T12:12:17.033339+0000","last_fullsized":"2024-09-12T12:12:17.033339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026843+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:12:17.026843+0000","last_peered":"2024-09-12T12:12:17.026843+0000","last_clean":"2024-09-12T12:12:17.026843+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:12:17.026843+0000","last_undegraded":"2024-09-12T12:12:17.026843+0000","last_fullsized":"2024-09-12T12:12:17.026843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028352+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:12:17.028352+0000","last_peered":"2024-09-12T12:12:17.028352+0000","last_clean":"2024-09-12T12:12:17.028352+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:12:17.028352+0000","last_undegraded":"2024-09-12T12:12:17.028352+0000","last_fullsized":"2024-09-12T12:12:17.028352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":88,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:21.549593+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:12:21.549593+0000","last_peered":"2024-09-12T12:12:21.549593+0000","last_clean":"2024-09-12T12:12:21.549593+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:12:21.549593+0000","last_undegraded":"2024-09-12T12:12:21.549593+0000","last_fullsized":"2024-09-12T12:12:21.549593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028966+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:12:17.028966+0000","last_peered":"2024-09-12T12:12:17.028966+0000","last_clean":"2024-09-12T12:12:17.028966+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:12:17.028966+0000","last_undegraded":"2024-09-12T12:12:17.028966+0000","last_fullsized":"2024-09-12T12:12:17.028966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200368+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:12:17.200368+0000","last_peered":"2024-09-12T12:12:17.200368+0000","last_clean":"2024-09-12T12:12:17.200368+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:12:17.200368+0000","last_undegraded":"2024-09-12T12:12:17.200368+0000","last_fullsized":"2024-09-12T12:12:17.200368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.027181+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:12:17.027181+0000","last_peered":"2024-09-12T12:12:17.027181+0000","last_clean":"2024-09-12T12:12:17.027181+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:12:17.027181+0000","last_undegraded":"2024-09-12T12:12:17.027181+0000","last_fullsized":"2024-09-12T12:12:17.027181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200747+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:12:17.200747+0000","last_peered":"2024-09-12T12:12:17.200747+0000","last_clean":"2024-09-12T12:12:17.200747+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:12:17.200747+0000","last_undegraded":"2024-09-12T12:12:17.200747+0000","last_fullsized":"2024-09-12T12:12:17.200747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":180,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:22.550996+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:12:22.550996+0000","last_peered":"2024-09-12T12:12:22.550996+0000","last_clean":"2024-09-12T12:12:22.550996+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:12:22.550996+0000","last_undegraded":"2024-09-12T12:12:22.550996+0000","last_fullsized":"2024-09-12T12:12:22.550996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":122,"num_write":27,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034261+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:12:17.034261+0000","last_peered":"2024-09-12T12:12:17.034261+0000","last_clean":"2024-09-12T12:12:17.034261+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:12:17.034261+0000","last_undegraded":"2024-09-12T12:12:17.034261+0000","last_fullsized":"2024-09-12T12:12:17.034261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028566+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:12:17.028566+0000","last_peered":"2024-09-12T12:12:17.028566+0000","last_clean":"2024-09-12T12:12:17.028566+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:12:17.028566+0000","last_undegraded":"2024-09-12T12:12:17.028566+0000","last_fullsized":"2024-09-12T12:12:17.028566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025123+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:12:17.025123+0000","last_peered":"2024-09-12T12:12:17.025123+0000","last_clean":"2024-09-12T12:12:17.025123+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:12:17.025123+0000","last_undegraded":"2024-09-12T12:12:17.025123+0000","last_fullsized":"2024-09-12T12:12:17.025123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026795+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:12:17.026795+0000","last_peered":"2024-09-12T12:12:17.026795+0000","last_clean":"2024-09-12T12:12:17.026795+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:12:17.026795+0000","last_undegraded":"2024-09-12T12:12:17.026795+0000","last_fullsized":"2024-09-12T12:12:17.026795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029013+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:12:17.029013+0000","last_peered":"2024-09-12T12:12:17.029013+0000","last_clean":"2024-09-12T12:12:17.029013+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:12:17.029013+0000","last_undegraded":"2024-09-12T12:12:17.029013+0000","last_fullsized":"2024-09-12T12:12:17.029013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":89,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:18.546255+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:12:18.546255+0000","last_peered":"2024-09-12T12:12:18.546255+0000","last_clean":"2024-09-12T12:12:18.546255+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:12:18.546255+0000","last_undegraded":"2024-09-12T12:12:18.546255+0000","last_fullsized":"2024-09-12T12:12:18.546255+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031317+0000","last_change":"2024-09-12T12:12:17.031317+0000","last_active":"2024-09-12T12:12:17.031317+0000","last_peered":"2024-09-12T12:12:17.031317+0000","last_clean":"2024-09-12T12:12:17.031317+0000","last_became_active":"2024-09-12T12:12:17.030842+0000","last_became_peered":"2024-09-12T12:12:17.030842+0000","last_unstale":"2024-09-12T12:12:17.031317+0000","last_undegraded":"2024-09-12T12:12:17.031317+0000","last_fullsized":"2024-09-12T12:12:17.031317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:11.616682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033026+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:12:17.033026+0000","last_peered":"2024-09-12T12:12:17.033026+0000","last_clean":"2024-09-12T12:12:17.033026+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:12:17.033026+0000","last_undegraded":"2024-09-12T12:12:17.033026+0000","last_fullsized":"2024-09-12T12:12:17.033026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026755+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:12:17.026755+0000","last_peered":"2024-09-12T12:12:17.026755+0000","last_clean":"2024-09-12T12:12:17.026755+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:12:17.026755+0000","last_undegraded":"2024-09-12T12:12:17.026755+0000","last_fullsized":"2024-09-12T12:12:17.026755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034442+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:12:17.034442+0000","last_peered":"2024-09-12T12:12:17.034442+0000","last_clean":"2024-09-12T12:12:17.034442+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:12:17.034442+0000","last_undegraded":"2024-09-12T12:12:17.034442+0000","last_fullsized":"2024-09-12T12:12:17.034442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031307+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:12:17.031307+0000","last_peered":"2024-09-12T12:12:17.031307+0000","last_clean":"2024-09-12T12:12:17.031307+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:12:17.031307+0000","last_undegraded":"2024-09-12T12:12:17.031307+0000","last_fullsized":"2024-09-12T12:12:17.031307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025314+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:12:17.025314+0000","last_peered":"2024-09-12T12:12:17.025314+0000","last_clean":"2024-09-12T12:12:17.025314+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:12:17.025314+0000","last_undegraded":"2024-09-12T12:12:17.025314+0000","last_fullsized":"2024-09-12T12:12:17.025314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029297+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:12:17.029297+0000","last_peered":"2024-09-12T12:12:17.029297+0000","last_clean":"2024-09-12T12:12:17.029297+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:12:17.029297+0000","last_undegraded":"2024-09-12T12:12:17.029297+0000","last_fullsized":"2024-09-12T12:12:17.029297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.027293+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:12:17.027293+0000","last_peered":"2024-09-12T12:12:17.027293+0000","last_clean":"2024-09-12T12:12:17.027293+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:12:17.027293+0000","last_undegraded":"2024-09-12T12:12:17.027293+0000","last_fullsized":"2024-09-12T12:12:17.027293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029052+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:12:17.029052+0000","last_peered":"2024-09-12T12:12:17.029052+0000","last_clean":"2024-09-12T12:12:17.029052+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:12:17.029052+0000","last_undegraded":"2024-09-12T12:12:17.029052+0000","last_fullsized":"2024-09-12T12:12:17.029052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029406+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:12:17.029406+0000","last_peered":"2024-09-12T12:12:17.029406+0000","last_clean":"2024-09-12T12:12:17.029406+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:12:17.029406+0000","last_undegraded":"2024-09-12T12:12:17.029406+0000","last_fullsized":"2024-09-12T12:12:17.029406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029338+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:12:17.029338+0000","last_peered":"2024-09-12T12:12:17.029338+0000","last_clean":"2024-09-12T12:12:17.029338+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:12:17.029338+0000","last_undegraded":"2024-09-12T12:12:17.029338+0000","last_fullsized":"2024-09-12T12:12:17.029338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029959+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:12:17.029959+0000","last_peered":"2024-09-12T12:12:17.029959+0000","last_clean":"2024-09-12T12:12:17.029959+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:12:17.029959+0000","last_undegraded":"2024-09-12T12:12:17.029959+0000","last_fullsized":"2024-09-12T12:12:17.029959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200957+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:12:17.200957+0000","last_peered":"2024-09-12T12:12:17.200957+0000","last_clean":"2024-09-12T12:12:17.200957+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:12:17.200957+0000","last_undegraded":"2024-09-12T12:12:17.200957+0000","last_fullsized":"2024-09-12T12:12:17.200957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032847+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:12:17.032847+0000","last_peered":"2024-09-12T12:12:17.032847+0000","last_clean":"2024-09-12T12:12:17.032847+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:12:17.032847+0000","last_undegraded":"2024-09-12T12:12:17.032847+0000","last_fullsized":"2024-09-12T12:12:17.032847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029901+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:12:17.029901+0000","last_peered":"2024-09-12T12:12:17.029901+0000","last_clean":"2024-09-12T12:12:17.029901+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:12:17.029901+0000","last_undegraded":"2024-09-12T12:12:17.029901+0000","last_fullsized":"2024-09-12T12:12:17.029901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025050+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:12:17.025050+0000","last_peered":"2024-09-12T12:12:17.025050+0000","last_clean":"2024-09-12T12:12:17.025050+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:12:17.025050+0000","last_undegraded":"2024-09-12T12:12:17.025050+0000","last_fullsized":"2024-09-12T12:12:17.025050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":86,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:18.545728+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:12:18.545728+0000","last_peered":"2024-09-12T12:12:18.545728+0000","last_clean":"2024-09-12T12:12:18.545728+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:12:18.545728+0000","last_undegraded":"2024-09-12T12:12:18.545728+0000","last_fullsized":"2024-09-12T12:12:18.545728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026631+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:12:17.026631+0000","last_peered":"2024-09-12T12:12:17.026631+0000","last_clean":"2024-09-12T12:12:17.026631+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:12:17.026631+0000","last_undegraded":"2024-09-12T12:12:17.026631+0000","last_fullsized":"2024-09-12T12:12:17.026631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028913+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:12:17.028913+0000","last_peered":"2024-09-12T12:12:17.028913+0000","last_clean":"2024-09-12T12:12:17.028913+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:12:17.028913+0000","last_undegraded":"2024-09-12T12:12:17.028913+0000","last_fullsized":"2024-09-12T12:12:17.028913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029412+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:12:17.029412+0000","last_peered":"2024-09-12T12:12:17.029412+0000","last_clean":"2024-09-12T12:12:17.029412+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:12:17.029412+0000","last_undegraded":"2024-09-12T12:12:17.029412+0000","last_fullsized":"2024-09-12T12:12:17.029412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028570+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:12:17.028570+0000","last_peered":"2024-09-12T12:12:17.028570+0000","last_clean":"2024-09-12T12:12:17.028570+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:12:17.028570+0000","last_undegraded":"2024-09-12T12:12:17.028570+0000","last_fullsized":"2024-09-12T12:12:17.028570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028914+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:12:17.028914+0000","last_peered":"2024-09-12T12:12:17.028914+0000","last_clean":"2024-09-12T12:12:17.028914+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:12:17.028914+0000","last_undegraded":"2024-09-12T12:12:17.028914+0000","last_fullsized":"2024-09-12T12:12:17.028914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034512+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:12:17.034512+0000","last_peered":"2024-09-12T12:12:17.034512+0000","last_clean":"2024-09-12T12:12:17.034512+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:12:17.034512+0000","last_undegraded":"2024-09-12T12:12:17.034512+0000","last_fullsized":"2024-09-12T12:12:17.034512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025398+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:12:17.025398+0000","last_peered":"2024-09-12T12:12:17.025398+0000","last_clean":"2024-09-12T12:12:17.025398+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:12:17.025398+0000","last_undegraded":"2024-09-12T12:12:17.025398+0000","last_fullsized":"2024-09-12T12:12:17.025398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028567+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:12:17.028567+0000","last_peered":"2024-09-12T12:12:17.028567+0000","last_clean":"2024-09-12T12:12:17.028567+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:12:17.028567+0000","last_undegraded":"2024-09-12T12:12:17.028567+0000","last_fullsized":"2024-09-12T12:12:17.028567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032777+0000","last_change":"2024-09-12T12:12:17.032314+0000","last_active":"2024-09-12T12:12:17.032777+0000","last_peered":"2024-09-12T12:12:17.032777+0000","last_clean":"2024-09-12T12:12:17.032777+0000","last_became_active":"2024-09-12T12:12:17.032022+0000","last_became_peered":"2024-09-12T12:12:17.032022+0000","last_unstale":"2024-09-12T12:12:17.032777+0000","last_undegraded":"2024-09-12T12:12:17.032777+0000","last_fullsized":"2024-09-12T12:12:17.032777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:55.111688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201240+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:12:17.201240+0000","last_peered":"2024-09-12T12:12:17.201240+0000","last_clean":"2024-09-12T12:12:17.201240+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:12:17.201240+0000","last_undegraded":"2024-09-12T12:12:17.201240+0000","last_fullsized":"2024-09-12T12:12:17.201240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200202+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:12:17.200202+0000","last_peered":"2024-09-12T12:12:17.200202+0000","last_clean":"2024-09-12T12:12:17.200202+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:12:17.200202+0000","last_undegraded":"2024-09-12T12:12:17.200202+0000","last_fullsized":"2024-09-12T12:12:17.200202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200521+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:12:17.200521+0000","last_peered":"2024-09-12T12:12:17.200521+0000","last_clean":"2024-09-12T12:12:17.200521+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:12:17.200521+0000","last_undegraded":"2024-09-12T12:12:17.200521+0000","last_fullsized":"2024-09-12T12:12:17.200521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033160+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:12:17.033160+0000","last_peered":"2024-09-12T12:12:17.033160+0000","last_clean":"2024-09-12T12:12:17.033160+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:12:17.033160+0000","last_undegraded":"2024-09-12T12:12:17.033160+0000","last_fullsized":"2024-09-12T12:12:17.033160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032969+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:12:17.032969+0000","last_peered":"2024-09-12T12:12:17.032969+0000","last_clean":"2024-09-12T12:12:17.032969+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:12:17.032969+0000","last_undegraded":"2024-09-12T12:12:17.032969+0000","last_fullsized":"2024-09-12T12:12:17.032969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030761+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:12:17.030761+0000","last_peered":"2024-09-12T12:12:17.030761+0000","last_clean":"2024-09-12T12:12:17.030761+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:12:17.030761+0000","last_undegraded":"2024-09-12T12:12:17.030761+0000","last_fullsized":"2024-09-12T12:12:17.030761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.039006+0000","last_change":"2024-09-12T12:12:17.038499+0000","last_active":"2024-09-12T12:12:17.039006+0000","last_peered":"2024-09-12T12:12:17.039006+0000","last_clean":"2024-09-12T12:12:17.039006+0000","last_became_active":"2024-09-12T12:12:17.034622+0000","last_became_peered":"2024-09-12T12:12:17.034622+0000","last_unstale":"2024-09-12T12:12:17.039006+0000","last_undegraded":"2024-09-12T12:12:17.039006+0000","last_fullsized":"2024-09-12T12:12:17.039006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:30.684068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033360+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:12:17.033360+0000","last_peered":"2024-09-12T12:12:17.033360+0000","last_clean":"2024-09-12T12:12:17.033360+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:12:17.033360+0000","last_undegraded":"2024-09-12T12:12:17.033360+0000","last_fullsized":"2024-09-12T12:12:17.033360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.042021+0000","last_change":"2024-09-12T12:12:17.041616+0000","last_active":"2024-09-12T12:12:17.042021+0000","last_peered":"2024-09-12T12:12:17.042021+0000","last_clean":"2024-09-12T12:12:17.042021+0000","last_became_active":"2024-09-12T12:12:17.035895+0000","last_became_peered":"2024-09-12T12:12:17.035895+0000","last_unstale":"2024-09-12T12:12:17.042021+0000","last_undegraded":"2024-09-12T12:12:17.042021+0000","last_fullsized":"2024-09-12T12:12:17.042021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:11.722682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028769+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:12:17.028769+0000","last_peered":"2024-09-12T12:12:17.028769+0000","last_clean":"2024-09-12T12:12:17.028769+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:12:17.028769+0000","last_undegraded":"2024-09-12T12:12:17.028769+0000","last_fullsized":"2024-09-12T12:12:17.028769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.027073+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:12:17.027073+0000","last_peered":"2024-09-12T12:12:17.027073+0000","last_clean":"2024-09-12T12:12:17.027073+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:12:17.027073+0000","last_undegraded":"2024-09-12T12:12:17.027073+0000","last_fullsized":"2024-09-12T12:12:17.027073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034217+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:12:17.034217+0000","last_peered":"2024-09-12T12:12:17.034217+0000","last_clean":"2024-09-12T12:12:17.034217+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:12:17.034217+0000","last_undegraded":"2024-09-12T12:12:17.034217+0000","last_fullsized":"2024-09-12T12:12:17.034217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200709+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:12:17.200709+0000","last_peered":"2024-09-12T12:12:17.200709+0000","last_clean":"2024-09-12T12:12:17.200709+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:12:17.200709+0000","last_undegraded":"2024-09-12T12:12:17.200709+0000","last_fullsized":"2024-09-12T12:12:17.200709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034393+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:12:17.034393+0000","last_peered":"2024-09-12T12:12:17.034393+0000","last_clean":"2024-09-12T12:12:17.034393+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:12:17.034393+0000","last_undegraded":"2024-09-12T12:12:17.034393+0000","last_fullsized":"2024-09-12T12:12:17.034393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200361+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:12:17.200361+0000","last_peered":"2024-09-12T12:12:17.200361+0000","last_clean":"2024-09-12T12:12:17.200361+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:12:17.200361+0000","last_undegraded":"2024-09-12T12:12:17.200361+0000","last_fullsized":"2024-09-12T12:12:17.200361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033161+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:12:17.033161+0000","last_peered":"2024-09-12T12:12:17.033161+0000","last_clean":"2024-09-12T12:12:17.033161+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:12:17.033161+0000","last_undegraded":"2024-09-12T12:12:17.033161+0000","last_fullsized":"2024-09-12T12:12:17.033161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032939+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:12:17.032939+0000","last_peered":"2024-09-12T12:12:17.032939+0000","last_clean":"2024-09-12T12:12:17.032939+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:12:17.032939+0000","last_undegraded":"2024-09-12T12:12:17.032939+0000","last_fullsized":"2024-09-12T12:12:17.032939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028701+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:12:17.028701+0000","last_peered":"2024-09-12T12:12:17.028701+0000","last_clean":"2024-09-12T12:12:17.028701+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:12:17.028701+0000","last_undegraded":"2024-09-12T12:12:17.028701+0000","last_fullsized":"2024-09-12T12:12:17.028701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":383,"num_read_kb":258,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":444201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":1095286,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939687,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":444437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168620,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":444357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397553,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1036289,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":443826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1035580,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":444656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":504180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":502830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:12:26.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:26 smithi189 ceph-mon[35475]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:26.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:26 smithi171 ceph-mon[35098]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:26.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:26 smithi137 ceph-mon[30950]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:27.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:27 smithi189 ceph-mon[35475]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:27.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:27 smithi171 ceph-mon[35098]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:27 smithi137 ceph-mon[30950]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:28.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:28 smithi189 ceph-mon[35475]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:28.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:28 smithi171 ceph-mon[35098]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:28 smithi137 ceph-mon[30950]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:29.105 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:12:29.366 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:29.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 -- 172.21.15.137:0/3411241516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 msgr2=0x7f9f3810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 --2- 172.21.15.137:0/3411241516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3810da40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9f30009ca0 tx=0x7f9f3002a950 comp rx=0 tx=0).stop 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 -- 172.21.15.137:0/3411241516 shutdown_connections 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 --2- 172.21.15.137:0/3411241516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3810da40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 --2- 172.21.15.137:0/3411241516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f38102570 0x7f9f381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 --2- 172.21.15.137:0/3411241516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f38100e20 0x7f9f38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 -- 172.21.15.137:0/3411241516 >> 172.21.15.137:0/3411241516 conn(0x7f9f380fc7d0 msgr2=0x7f9f380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 -- 172.21.15.137:0/3411241516 shutdown_connections 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.891+0000 7f9f3df32640 1 -- 172.21.15.137:0/3411241516 wait complete. 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 Processor -- start 2024-09-12T12:12:29.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 -- start start 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f38100e20 0x7f9f38198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f38102570 0x7f9f381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f381120a0 con 0x7f9f38102ef0 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38111f20 con 0x7f9f38100e20 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.892+0000 7f9f3df32640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f38112220 con 0x7f9f38102570 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60794/0 (socket says 172.21.15.137:60794) 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 -- 172.21.15.137:0/1535428762 learned_addr learned my addr 172.21.15.137:0/1535428762 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f2effd640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f38102570 0x7f9f381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f377fe640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f38100e20 0x7f9f38198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 -- 172.21.15.137:0/1535428762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f38102570 msgr2=0x7f9f381989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:29.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f38102570 0x7f9f381989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 -- 172.21.15.137:0/1535428762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f38100e20 msgr2=0x7f9f38198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f38100e20 0x7f9f38198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 -- 172.21.15.137:0/1535428762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3819d410 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f2effd640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f38102570 0x7f9f381989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f37fff640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3819cd30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006010 tx=0x7f9f3002f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f357fa640 1 -- 172.21.15.137:0/1535428762 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30034040 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3819d6a0 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f357fa640 1 -- 172.21.15.137:0/1535428762 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f30003e00 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f357fa640 1 -- 172.21.15.137:0/1535428762 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30045480 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.893+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f381a4fe0 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.894+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f38101200 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.895+0000 7f9f357fa640 1 -- 172.21.15.137:0/1535428762 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9f30037020 con 0x7f9f38102ef0 2024-09-12T12:12:29.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.896+0000 7f9f357fa640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9f240a2c90 0x7f9f240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:29.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.896+0000 7f9f377fe640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9f240a2c90 0x7f9f240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:29.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.896+0000 7f9f357fa640 1 -- 172.21.15.137:0/1535428762 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f9f300f0190 con 0x7f9f38102ef0 2024-09-12T12:12:29.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.897+0000 7f9f377fe640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9f240a2c90 0x7f9f240a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f280058c0 tx=0x7f9f28005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:29.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:29.899+0000 7f9f357fa640 1 -- 172.21.15.137:0/1535428762 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f300443e0 con 0x7f9f38102ef0 2024-09-12T12:12:30.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.065+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f38106fb0 con 0x7f9f240a2c90 2024-09-12T12:12:30.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.071+0000 7f9f357fa640 1 -- 172.21.15.137:0/1535428762 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258873 (secure 0 0 0) 0x7f9f38106fb0 con 0x7f9f240a2c90 2024-09-12T12:12:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:30.075 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.073+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9f240a2c90 msgr2=0x7f9f240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.073+0000 7f9f3df32640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9f240a2c90 0x7f9f240a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9f280058c0 tx=0x7f9f28005c60 comp rx=0 tx=0).stop 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.073+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 msgr2=0x7f9f3819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.073+0000 7f9f3df32640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3819cd30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9f30006010 tx=0x7f9f3002f6f0 comp rx=0 tx=0).stop 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 shutdown_connections 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f9f240a2c90 0x7f9f240a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f38102ef0 0x7f9f3819cd30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:30.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f38102570 0x7f9f381989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:30.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 --2- 172.21.15.137:0/1535428762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f38100e20 0x7f9f38198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:30.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 >> 172.21.15.137:0/1535428762 conn(0x7f9f380fc7d0 msgr2=0x7f9f381068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:30.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 shutdown_connections 2024-09-12T12:12:30.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:30.074+0000 7f9f3df32640 1 -- 172.21.15.137:0/1535428762 wait complete. 2024-09-12T12:12:30.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:30 smithi171 ceph-mon[35098]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:30.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:30 smithi137 ceph-mon[30950]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:30.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:30 smithi189 ceph-mon[35475]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:30.821 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-12T12:12:29.127788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653816,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":549,"num_read_kb":392,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356344,"kb_used_data":11184,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585480,"statfs":{"total":1151940427776,"available":1151575531520,"internally_reserved":0,"allocated":11452416,"data_stored":7284876,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":38,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003158"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031172+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:12:17.031172+0000","last_peered":"2024-09-12T12:12:17.031172+0000","last_clean":"2024-09-12T12:12:17.031172+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:12:17.031172+0000","last_undegraded":"2024-09-12T12:12:17.031172+0000","last_fullsized":"2024-09-12T12:12:17.031172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200200+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:12:17.200200+0000","last_peered":"2024-09-12T12:12:17.200200+0000","last_clean":"2024-09-12T12:12:17.200200+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:12:17.200200+0000","last_undegraded":"2024-09-12T12:12:17.200200+0000","last_fullsized":"2024-09-12T12:12:17.200200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200138+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:12:17.200138+0000","last_peered":"2024-09-12T12:12:17.200138+0000","last_clean":"2024-09-12T12:12:17.200138+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:12:17.200138+0000","last_undegraded":"2024-09-12T12:12:17.200138+0000","last_fullsized":"2024-09-12T12:12:17.200138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200871+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:12:17.200871+0000","last_peered":"2024-09-12T12:12:17.200871+0000","last_clean":"2024-09-12T12:12:17.200871+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:12:17.200871+0000","last_undegraded":"2024-09-12T12:12:17.200871+0000","last_fullsized":"2024-09-12T12:12:17.200871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034055+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:12:17.034055+0000","last_peered":"2024-09-12T12:12:17.034055+0000","last_clean":"2024-09-12T12:12:17.034055+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:12:17.034055+0000","last_undegraded":"2024-09-12T12:12:17.034055+0000","last_fullsized":"2024-09-12T12:12:17.034055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029133+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:12:17.029133+0000","last_peered":"2024-09-12T12:12:17.029133+0000","last_clean":"2024-09-12T12:12:17.029133+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:12:17.029133+0000","last_undegraded":"2024-09-12T12:12:17.029133+0000","last_fullsized":"2024-09-12T12:12:17.029133+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028562+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:12:17.028562+0000","last_peered":"2024-09-12T12:12:17.028562+0000","last_clean":"2024-09-12T12:12:17.028562+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:12:17.028562+0000","last_undegraded":"2024-09-12T12:12:17.028562+0000","last_fullsized":"2024-09-12T12:12:17.028562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033265+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:12:17.033265+0000","last_peered":"2024-09-12T12:12:17.033265+0000","last_clean":"2024-09-12T12:12:17.033265+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:12:17.033265+0000","last_undegraded":"2024-09-12T12:12:17.033265+0000","last_fullsized":"2024-09-12T12:12:17.033265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029456+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:12:17.029456+0000","last_peered":"2024-09-12T12:12:17.029456+0000","last_clean":"2024-09-12T12:12:17.029456+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:12:17.029456+0000","last_undegraded":"2024-09-12T12:12:17.029456+0000","last_fullsized":"2024-09-12T12:12:17.029456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028748+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:12:17.028748+0000","last_peered":"2024-09-12T12:12:17.028748+0000","last_clean":"2024-09-12T12:12:17.028748+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:12:17.028748+0000","last_undegraded":"2024-09-12T12:12:17.028748+0000","last_fullsized":"2024-09-12T12:12:17.028748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200997+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:12:17.200997+0000","last_peered":"2024-09-12T12:12:17.200997+0000","last_clean":"2024-09-12T12:12:17.200997+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:12:17.200997+0000","last_undegraded":"2024-09-12T12:12:17.200997+0000","last_fullsized":"2024-09-12T12:12:17.200997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025292+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:12:17.025292+0000","last_peered":"2024-09-12T12:12:17.025292+0000","last_clean":"2024-09-12T12:12:17.025292+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:12:17.025292+0000","last_undegraded":"2024-09-12T12:12:17.025292+0000","last_fullsized":"2024-09-12T12:12:17.025292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200027+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:12:17.200027+0000","last_peered":"2024-09-12T12:12:17.200027+0000","last_clean":"2024-09-12T12:12:17.200027+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:12:17.200027+0000","last_undegraded":"2024-09-12T12:12:17.200027+0000","last_fullsized":"2024-09-12T12:12:17.200027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.199927+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:12:17.199927+0000","last_peered":"2024-09-12T12:12:17.199927+0000","last_clean":"2024-09-12T12:12:17.199927+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:12:17.199927+0000","last_undegraded":"2024-09-12T12:12:17.199927+0000","last_fullsized":"2024-09-12T12:12:17.199927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200399+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:12:17.200399+0000","last_peered":"2024-09-12T12:12:17.200399+0000","last_clean":"2024-09-12T12:12:17.200399+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:12:17.200399+0000","last_undegraded":"2024-09-12T12:12:17.200399+0000","last_fullsized":"2024-09-12T12:12:17.200399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031383+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:12:17.031383+0000","last_peered":"2024-09-12T12:12:17.031383+0000","last_clean":"2024-09-12T12:12:17.031383+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:12:17.031383+0000","last_undegraded":"2024-09-12T12:12:17.031383+0000","last_fullsized":"2024-09-12T12:12:17.031383+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028526+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:12:17.028526+0000","last_peered":"2024-09-12T12:12:17.028526+0000","last_clean":"2024-09-12T12:12:17.028526+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:12:17.028526+0000","last_undegraded":"2024-09-12T12:12:17.028526+0000","last_fullsized":"2024-09-12T12:12:17.028526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028525+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:12:17.028525+0000","last_peered":"2024-09-12T12:12:17.028525+0000","last_clean":"2024-09-12T12:12:17.028525+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:12:17.028525+0000","last_undegraded":"2024-09-12T12:12:17.028525+0000","last_fullsized":"2024-09-12T12:12:17.028525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030717+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:12:17.030717+0000","last_peered":"2024-09-12T12:12:17.030717+0000","last_clean":"2024-09-12T12:12:17.030717+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:12:17.030717+0000","last_undegraded":"2024-09-12T12:12:17.030717+0000","last_fullsized":"2024-09-12T12:12:17.030717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025393+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:12:17.025393+0000","last_peered":"2024-09-12T12:12:17.025393+0000","last_clean":"2024-09-12T12:12:17.025393+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:12:17.025393+0000","last_undegraded":"2024-09-12T12:12:17.025393+0000","last_fullsized":"2024-09-12T12:12:17.025393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200826+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:12:17.200826+0000","last_peered":"2024-09-12T12:12:17.200826+0000","last_clean":"2024-09-12T12:12:17.200826+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:12:17.200826+0000","last_undegraded":"2024-09-12T12:12:17.200826+0000","last_fullsized":"2024-09-12T12:12:17.200826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032817+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:12:17.032817+0000","last_peered":"2024-09-12T12:12:17.032817+0000","last_clean":"2024-09-12T12:12:17.032817+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:12:17.032817+0000","last_undegraded":"2024-09-12T12:12:17.032817+0000","last_fullsized":"2024-09-12T12:12:17.032817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026551+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:12:17.026551+0000","last_peered":"2024-09-12T12:12:17.026551+0000","last_clean":"2024-09-12T12:12:17.026551+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:12:17.026551+0000","last_undegraded":"2024-09-12T12:12:17.026551+0000","last_fullsized":"2024-09-12T12:12:17.026551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033429+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:12:17.033429+0000","last_peered":"2024-09-12T12:12:17.033429+0000","last_clean":"2024-09-12T12:12:17.033429+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:12:17.033429+0000","last_undegraded":"2024-09-12T12:12:17.033429+0000","last_fullsized":"2024-09-12T12:12:17.033429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028783+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:12:17.028783+0000","last_peered":"2024-09-12T12:12:17.028783+0000","last_clean":"2024-09-12T12:12:17.028783+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:12:17.028783+0000","last_undegraded":"2024-09-12T12:12:17.028783+0000","last_fullsized":"2024-09-12T12:12:17.028783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031113+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:12:17.031113+0000","last_peered":"2024-09-12T12:12:17.031113+0000","last_clean":"2024-09-12T12:12:17.031113+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:12:17.031113+0000","last_undegraded":"2024-09-12T12:12:17.031113+0000","last_fullsized":"2024-09-12T12:12:17.031113+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201138+0000","last_change":"2024-09-12T12:12:17.201138+0000","last_active":"2024-09-12T12:12:17.201138+0000","last_peered":"2024-09-12T12:12:17.201138+0000","last_clean":"2024-09-12T12:12:17.201138+0000","last_became_active":"2024-09-12T12:12:17.200768+0000","last_became_peered":"2024-09-12T12:12:17.200768+0000","last_unstale":"2024-09-12T12:12:17.201138+0000","last_undegraded":"2024-09-12T12:12:17.201138+0000","last_fullsized":"2024-09-12T12:12:17.201138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:48.749285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200084+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:12:17.200084+0000","last_peered":"2024-09-12T12:12:17.200084+0000","last_clean":"2024-09-12T12:12:17.200084+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:12:17.200084+0000","last_undegraded":"2024-09-12T12:12:17.200084+0000","last_fullsized":"2024-09-12T12:12:17.200084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028596+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:12:17.028596+0000","last_peered":"2024-09-12T12:12:17.028596+0000","last_clean":"2024-09-12T12:12:17.028596+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:12:17.028596+0000","last_undegraded":"2024-09-12T12:12:17.028596+0000","last_fullsized":"2024-09-12T12:12:17.028596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030334+0000","last_change":"2024-09-12T12:12:17.030334+0000","last_active":"2024-09-12T12:12:17.030334+0000","last_peered":"2024-09-12T12:12:17.030334+0000","last_clean":"2024-09-12T12:12:17.030334+0000","last_became_active":"2024-09-12T12:12:17.029947+0000","last_became_peered":"2024-09-12T12:12:17.029947+0000","last_unstale":"2024-09-12T12:12:17.030334+0000","last_undegraded":"2024-09-12T12:12:17.030334+0000","last_fullsized":"2024-09-12T12:12:17.030334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:10.299557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030670+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:12:17.030670+0000","last_peered":"2024-09-12T12:12:17.030670+0000","last_clean":"2024-09-12T12:12:17.030670+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:12:17.030670+0000","last_undegraded":"2024-09-12T12:12:17.030670+0000","last_fullsized":"2024-09-12T12:12:17.030670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033339+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:12:17.033339+0000","last_peered":"2024-09-12T12:12:17.033339+0000","last_clean":"2024-09-12T12:12:17.033339+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:12:17.033339+0000","last_undegraded":"2024-09-12T12:12:17.033339+0000","last_fullsized":"2024-09-12T12:12:17.033339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":95,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:24.554432+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:12:24.554432+0000","last_peered":"2024-09-12T12:12:24.554432+0000","last_clean":"2024-09-12T12:12:24.554432+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:12:24.554432+0000","last_undegraded":"2024-09-12T12:12:24.554432+0000","last_fullsized":"2024-09-12T12:12:24.554432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.037302+0000","last_change":"2024-09-12T12:12:17.037302+0000","last_active":"2024-09-12T12:12:17.037302+0000","last_peered":"2024-09-12T12:12:17.037302+0000","last_clean":"2024-09-12T12:12:17.037302+0000","last_became_active":"2024-09-12T12:12:17.036907+0000","last_became_peered":"2024-09-12T12:12:17.036907+0000","last_unstale":"2024-09-12T12:12:17.037302+0000","last_undegraded":"2024-09-12T12:12:17.037302+0000","last_fullsized":"2024-09-12T12:12:17.037302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:06:37.061690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029242+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:12:17.029242+0000","last_peered":"2024-09-12T12:12:17.029242+0000","last_clean":"2024-09-12T12:12:17.029242+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:12:17.029242+0000","last_undegraded":"2024-09-12T12:12:17.029242+0000","last_fullsized":"2024-09-12T12:12:17.029242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025082+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:12:17.025082+0000","last_peered":"2024-09-12T12:12:17.025082+0000","last_clean":"2024-09-12T12:12:17.025082+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:12:17.025082+0000","last_undegraded":"2024-09-12T12:12:17.025082+0000","last_fullsized":"2024-09-12T12:12:17.025082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033339+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:12:17.033339+0000","last_peered":"2024-09-12T12:12:17.033339+0000","last_clean":"2024-09-12T12:12:17.033339+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:12:17.033339+0000","last_undegraded":"2024-09-12T12:12:17.033339+0000","last_fullsized":"2024-09-12T12:12:17.033339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026843+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:12:17.026843+0000","last_peered":"2024-09-12T12:12:17.026843+0000","last_clean":"2024-09-12T12:12:17.026843+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:12:17.026843+0000","last_undegraded":"2024-09-12T12:12:17.026843+0000","last_fullsized":"2024-09-12T12:12:17.026843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028352+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:12:17.028352+0000","last_peered":"2024-09-12T12:12:17.028352+0000","last_clean":"2024-09-12T12:12:17.028352+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:12:17.028352+0000","last_undegraded":"2024-09-12T12:12:17.028352+0000","last_fullsized":"2024-09-12T12:12:17.028352+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":93,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:26.556298+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:12:26.556298+0000","last_peered":"2024-09-12T12:12:26.556298+0000","last_clean":"2024-09-12T12:12:26.556298+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:12:26.556298+0000","last_undegraded":"2024-09-12T12:12:26.556298+0000","last_fullsized":"2024-09-12T12:12:26.556298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201332+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:12:17.201332+0000","last_peered":"2024-09-12T12:12:17.201332+0000","last_clean":"2024-09-12T12:12:17.201332+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:12:17.201332+0000","last_undegraded":"2024-09-12T12:12:17.201332+0000","last_fullsized":"2024-09-12T12:12:17.201332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028966+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:12:17.028966+0000","last_peered":"2024-09-12T12:12:17.028966+0000","last_clean":"2024-09-12T12:12:17.028966+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:12:17.028966+0000","last_undegraded":"2024-09-12T12:12:17.028966+0000","last_fullsized":"2024-09-12T12:12:17.028966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200368+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:12:17.200368+0000","last_peered":"2024-09-12T12:12:17.200368+0000","last_clean":"2024-09-12T12:12:17.200368+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:12:17.200368+0000","last_undegraded":"2024-09-12T12:12:17.200368+0000","last_fullsized":"2024-09-12T12:12:17.200368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.027181+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:12:17.027181+0000","last_peered":"2024-09-12T12:12:17.027181+0000","last_clean":"2024-09-12T12:12:17.027181+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:12:17.027181+0000","last_undegraded":"2024-09-12T12:12:17.027181+0000","last_fullsized":"2024-09-12T12:12:17.027181+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200747+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:12:17.200747+0000","last_peered":"2024-09-12T12:12:17.200747+0000","last_clean":"2024-09-12T12:12:17.200747+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:12:17.200747+0000","last_undegraded":"2024-09-12T12:12:17.200747+0000","last_fullsized":"2024-09-12T12:12:17.200747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'26","reported_seq":190,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:27.557687+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:12:27.557687+0000","last_peered":"2024-09-12T12:12:27.557687+0000","last_clean":"2024-09-12T12:12:27.557687+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:12:27.557687+0000","last_undegraded":"2024-09-12T12:12:27.557687+0000","last_fullsized":"2024-09-12T12:12:27.557687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":140,"num_read_kb":127,"num_write":27,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034261+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:12:17.034261+0000","last_peered":"2024-09-12T12:12:17.034261+0000","last_clean":"2024-09-12T12:12:17.034261+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:12:17.034261+0000","last_undegraded":"2024-09-12T12:12:17.034261+0000","last_fullsized":"2024-09-12T12:12:17.034261+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028566+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:12:17.028566+0000","last_peered":"2024-09-12T12:12:17.028566+0000","last_clean":"2024-09-12T12:12:17.028566+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:12:17.028566+0000","last_undegraded":"2024-09-12T12:12:17.028566+0000","last_fullsized":"2024-09-12T12:12:17.028566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025123+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:12:17.025123+0000","last_peered":"2024-09-12T12:12:17.025123+0000","last_clean":"2024-09-12T12:12:17.025123+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:12:17.025123+0000","last_undegraded":"2024-09-12T12:12:17.025123+0000","last_fullsized":"2024-09-12T12:12:17.025123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026795+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:12:17.026795+0000","last_peered":"2024-09-12T12:12:17.026795+0000","last_clean":"2024-09-12T12:12:17.026795+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:12:17.026795+0000","last_undegraded":"2024-09-12T12:12:17.026795+0000","last_fullsized":"2024-09-12T12:12:17.026795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029013+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:12:17.029013+0000","last_peered":"2024-09-12T12:12:17.029013+0000","last_clean":"2024-09-12T12:12:17.029013+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:12:17.029013+0000","last_undegraded":"2024-09-12T12:12:17.029013+0000","last_fullsized":"2024-09-12T12:12:17.029013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":94,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:23.553200+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:12:23.553200+0000","last_peered":"2024-09-12T12:12:23.553200+0000","last_clean":"2024-09-12T12:12:23.553200+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:12:23.553200+0000","last_undegraded":"2024-09-12T12:12:23.553200+0000","last_fullsized":"2024-09-12T12:12:23.553200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031317+0000","last_change":"2024-09-12T12:12:17.031317+0000","last_active":"2024-09-12T12:12:17.031317+0000","last_peered":"2024-09-12T12:12:17.031317+0000","last_clean":"2024-09-12T12:12:17.031317+0000","last_became_active":"2024-09-12T12:12:17.030842+0000","last_became_peered":"2024-09-12T12:12:17.030842+0000","last_unstale":"2024-09-12T12:12:17.031317+0000","last_undegraded":"2024-09-12T12:12:17.031317+0000","last_fullsized":"2024-09-12T12:12:17.031317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:11.616682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033026+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:12:17.033026+0000","last_peered":"2024-09-12T12:12:17.033026+0000","last_clean":"2024-09-12T12:12:17.033026+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:12:17.033026+0000","last_undegraded":"2024-09-12T12:12:17.033026+0000","last_fullsized":"2024-09-12T12:12:17.033026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026755+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:12:17.026755+0000","last_peered":"2024-09-12T12:12:17.026755+0000","last_clean":"2024-09-12T12:12:17.026755+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:12:17.026755+0000","last_undegraded":"2024-09-12T12:12:17.026755+0000","last_fullsized":"2024-09-12T12:12:17.026755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034442+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:12:17.034442+0000","last_peered":"2024-09-12T12:12:17.034442+0000","last_clean":"2024-09-12T12:12:17.034442+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:12:17.034442+0000","last_undegraded":"2024-09-12T12:12:17.034442+0000","last_fullsized":"2024-09-12T12:12:17.034442+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.031307+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:12:17.031307+0000","last_peered":"2024-09-12T12:12:17.031307+0000","last_clean":"2024-09-12T12:12:17.031307+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:12:17.031307+0000","last_undegraded":"2024-09-12T12:12:17.031307+0000","last_fullsized":"2024-09-12T12:12:17.031307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025314+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:12:17.025314+0000","last_peered":"2024-09-12T12:12:17.025314+0000","last_clean":"2024-09-12T12:12:17.025314+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:12:17.025314+0000","last_undegraded":"2024-09-12T12:12:17.025314+0000","last_fullsized":"2024-09-12T12:12:17.025314+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029297+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:12:17.029297+0000","last_peered":"2024-09-12T12:12:17.029297+0000","last_clean":"2024-09-12T12:12:17.029297+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:12:17.029297+0000","last_undegraded":"2024-09-12T12:12:17.029297+0000","last_fullsized":"2024-09-12T12:12:17.029297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.027293+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:12:17.027293+0000","last_peered":"2024-09-12T12:12:17.027293+0000","last_clean":"2024-09-12T12:12:17.027293+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:12:17.027293+0000","last_undegraded":"2024-09-12T12:12:17.027293+0000","last_fullsized":"2024-09-12T12:12:17.027293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029052+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:12:17.029052+0000","last_peered":"2024-09-12T12:12:17.029052+0000","last_clean":"2024-09-12T12:12:17.029052+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:12:17.029052+0000","last_undegraded":"2024-09-12T12:12:17.029052+0000","last_fullsized":"2024-09-12T12:12:17.029052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029406+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:12:17.029406+0000","last_peered":"2024-09-12T12:12:17.029406+0000","last_clean":"2024-09-12T12:12:17.029406+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:12:17.029406+0000","last_undegraded":"2024-09-12T12:12:17.029406+0000","last_fullsized":"2024-09-12T12:12:17.029406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029338+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:12:17.029338+0000","last_peered":"2024-09-12T12:12:17.029338+0000","last_clean":"2024-09-12T12:12:17.029338+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:12:17.029338+0000","last_undegraded":"2024-09-12T12:12:17.029338+0000","last_fullsized":"2024-09-12T12:12:17.029338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029959+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:12:17.029959+0000","last_peered":"2024-09-12T12:12:17.029959+0000","last_clean":"2024-09-12T12:12:17.029959+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:12:17.029959+0000","last_undegraded":"2024-09-12T12:12:17.029959+0000","last_fullsized":"2024-09-12T12:12:17.029959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200957+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:12:17.200957+0000","last_peered":"2024-09-12T12:12:17.200957+0000","last_clean":"2024-09-12T12:12:17.200957+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:12:17.200957+0000","last_undegraded":"2024-09-12T12:12:17.200957+0000","last_fullsized":"2024-09-12T12:12:17.200957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032847+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:12:17.032847+0000","last_peered":"2024-09-12T12:12:17.032847+0000","last_clean":"2024-09-12T12:12:17.032847+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:12:17.032847+0000","last_undegraded":"2024-09-12T12:12:17.032847+0000","last_fullsized":"2024-09-12T12:12:17.032847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029901+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:12:17.029901+0000","last_peered":"2024-09-12T12:12:17.029901+0000","last_clean":"2024-09-12T12:12:17.029901+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:12:17.029901+0000","last_undegraded":"2024-09-12T12:12:17.029901+0000","last_fullsized":"2024-09-12T12:12:17.029901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025050+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:12:17.025050+0000","last_peered":"2024-09-12T12:12:17.025050+0000","last_clean":"2024-09-12T12:12:17.025050+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:12:17.025050+0000","last_undegraded":"2024-09-12T12:12:17.025050+0000","last_fullsized":"2024-09-12T12:12:17.025050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":91,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:23.552580+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:12:23.552580+0000","last_peered":"2024-09-12T12:12:23.552580+0000","last_clean":"2024-09-12T12:12:23.552580+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:12:23.552580+0000","last_undegraded":"2024-09-12T12:12:23.552580+0000","last_fullsized":"2024-09-12T12:12:23.552580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.026631+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:12:17.026631+0000","last_peered":"2024-09-12T12:12:17.026631+0000","last_clean":"2024-09-12T12:12:17.026631+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:12:17.026631+0000","last_undegraded":"2024-09-12T12:12:17.026631+0000","last_fullsized":"2024-09-12T12:12:17.026631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028913+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:12:17.028913+0000","last_peered":"2024-09-12T12:12:17.028913+0000","last_clean":"2024-09-12T12:12:17.028913+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:12:17.028913+0000","last_undegraded":"2024-09-12T12:12:17.028913+0000","last_fullsized":"2024-09-12T12:12:17.028913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.029412+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:12:17.029412+0000","last_peered":"2024-09-12T12:12:17.029412+0000","last_clean":"2024-09-12T12:12:17.029412+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:12:17.029412+0000","last_undegraded":"2024-09-12T12:12:17.029412+0000","last_fullsized":"2024-09-12T12:12:17.029412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028570+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:12:17.028570+0000","last_peered":"2024-09-12T12:12:17.028570+0000","last_clean":"2024-09-12T12:12:17.028570+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:12:17.028570+0000","last_undegraded":"2024-09-12T12:12:17.028570+0000","last_fullsized":"2024-09-12T12:12:17.028570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028914+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:12:17.028914+0000","last_peered":"2024-09-12T12:12:17.028914+0000","last_clean":"2024-09-12T12:12:17.028914+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:12:17.028914+0000","last_undegraded":"2024-09-12T12:12:17.028914+0000","last_fullsized":"2024-09-12T12:12:17.028914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034512+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:12:17.034512+0000","last_peered":"2024-09-12T12:12:17.034512+0000","last_clean":"2024-09-12T12:12:17.034512+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:12:17.034512+0000","last_undegraded":"2024-09-12T12:12:17.034512+0000","last_fullsized":"2024-09-12T12:12:17.034512+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.025398+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:12:17.025398+0000","last_peered":"2024-09-12T12:12:17.025398+0000","last_clean":"2024-09-12T12:12:17.025398+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:12:17.025398+0000","last_undegraded":"2024-09-12T12:12:17.025398+0000","last_fullsized":"2024-09-12T12:12:17.025398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028567+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:12:17.028567+0000","last_peered":"2024-09-12T12:12:17.028567+0000","last_clean":"2024-09-12T12:12:17.028567+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:12:17.028567+0000","last_undegraded":"2024-09-12T12:12:17.028567+0000","last_fullsized":"2024-09-12T12:12:17.028567+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032777+0000","last_change":"2024-09-12T12:12:17.032314+0000","last_active":"2024-09-12T12:12:17.032777+0000","last_peered":"2024-09-12T12:12:17.032777+0000","last_clean":"2024-09-12T12:12:17.032777+0000","last_became_active":"2024-09-12T12:12:17.032022+0000","last_became_peered":"2024-09-12T12:12:17.032022+0000","last_unstale":"2024-09-12T12:12:17.032777+0000","last_undegraded":"2024-09-12T12:12:17.032777+0000","last_fullsized":"2024-09-12T12:12:17.032777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:55.111688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.201240+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:12:17.201240+0000","last_peered":"2024-09-12T12:12:17.201240+0000","last_clean":"2024-09-12T12:12:17.201240+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:12:17.201240+0000","last_undegraded":"2024-09-12T12:12:17.201240+0000","last_fullsized":"2024-09-12T12:12:17.201240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200202+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:12:17.200202+0000","last_peered":"2024-09-12T12:12:17.200202+0000","last_clean":"2024-09-12T12:12:17.200202+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:12:17.200202+0000","last_undegraded":"2024-09-12T12:12:17.200202+0000","last_fullsized":"2024-09-12T12:12:17.200202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200521+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:12:17.200521+0000","last_peered":"2024-09-12T12:12:17.200521+0000","last_clean":"2024-09-12T12:12:17.200521+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:12:17.200521+0000","last_undegraded":"2024-09-12T12:12:17.200521+0000","last_fullsized":"2024-09-12T12:12:17.200521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033160+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:12:17.033160+0000","last_peered":"2024-09-12T12:12:17.033160+0000","last_clean":"2024-09-12T12:12:17.033160+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:12:17.033160+0000","last_undegraded":"2024-09-12T12:12:17.033160+0000","last_fullsized":"2024-09-12T12:12:17.033160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032969+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:12:17.032969+0000","last_peered":"2024-09-12T12:12:17.032969+0000","last_clean":"2024-09-12T12:12:17.032969+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:12:17.032969+0000","last_undegraded":"2024-09-12T12:12:17.032969+0000","last_fullsized":"2024-09-12T12:12:17.032969+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.030761+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:12:17.030761+0000","last_peered":"2024-09-12T12:12:17.030761+0000","last_clean":"2024-09-12T12:12:17.030761+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:12:17.030761+0000","last_undegraded":"2024-09-12T12:12:17.030761+0000","last_fullsized":"2024-09-12T12:12:17.030761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.039006+0000","last_change":"2024-09-12T12:12:17.038499+0000","last_active":"2024-09-12T12:12:17.039006+0000","last_peered":"2024-09-12T12:12:17.039006+0000","last_clean":"2024-09-12T12:12:17.039006+0000","last_became_active":"2024-09-12T12:12:17.034622+0000","last_became_peered":"2024-09-12T12:12:17.034622+0000","last_unstale":"2024-09-12T12:12:17.039006+0000","last_undegraded":"2024-09-12T12:12:17.039006+0000","last_fullsized":"2024-09-12T12:12:17.039006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:30.684068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033360+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:12:17.033360+0000","last_peered":"2024-09-12T12:12:17.033360+0000","last_clean":"2024-09-12T12:12:17.033360+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:12:17.033360+0000","last_undegraded":"2024-09-12T12:12:17.033360+0000","last_fullsized":"2024-09-12T12:12:17.033360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.042021+0000","last_change":"2024-09-12T12:12:17.041616+0000","last_active":"2024-09-12T12:12:17.042021+0000","last_peered":"2024-09-12T12:12:17.042021+0000","last_clean":"2024-09-12T12:12:17.042021+0000","last_became_active":"2024-09-12T12:12:17.035895+0000","last_became_peered":"2024-09-12T12:12:17.035895+0000","last_unstale":"2024-09-12T12:12:17.042021+0000","last_undegraded":"2024-09-12T12:12:17.042021+0000","last_fullsized":"2024-09-12T12:12:17.042021+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:11.722682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028769+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:12:17.028769+0000","last_peered":"2024-09-12T12:12:17.028769+0000","last_clean":"2024-09-12T12:12:17.028769+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:12:17.028769+0000","last_undegraded":"2024-09-12T12:12:17.028769+0000","last_fullsized":"2024-09-12T12:12:17.028769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.027073+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:12:17.027073+0000","last_peered":"2024-09-12T12:12:17.027073+0000","last_clean":"2024-09-12T12:12:17.027073+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:12:17.027073+0000","last_undegraded":"2024-09-12T12:12:17.027073+0000","last_fullsized":"2024-09-12T12:12:17.027073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034217+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:12:17.034217+0000","last_peered":"2024-09-12T12:12:17.034217+0000","last_clean":"2024-09-12T12:12:17.034217+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:12:17.034217+0000","last_undegraded":"2024-09-12T12:12:17.034217+0000","last_fullsized":"2024-09-12T12:12:17.034217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200709+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:12:17.200709+0000","last_peered":"2024-09-12T12:12:17.200709+0000","last_clean":"2024-09-12T12:12:17.200709+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:12:17.200709+0000","last_undegraded":"2024-09-12T12:12:17.200709+0000","last_fullsized":"2024-09-12T12:12:17.200709+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.034393+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:12:17.034393+0000","last_peered":"2024-09-12T12:12:17.034393+0000","last_clean":"2024-09-12T12:12:17.034393+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:12:17.034393+0000","last_undegraded":"2024-09-12T12:12:17.034393+0000","last_fullsized":"2024-09-12T12:12:17.034393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.200361+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:12:17.200361+0000","last_peered":"2024-09-12T12:12:17.200361+0000","last_clean":"2024-09-12T12:12:17.200361+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:12:17.200361+0000","last_undegraded":"2024-09-12T12:12:17.200361+0000","last_fullsized":"2024-09-12T12:12:17.200361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.033161+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:12:17.033161+0000","last_peered":"2024-09-12T12:12:17.033161+0000","last_clean":"2024-09-12T12:12:17.033161+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:12:17.033161+0000","last_undegraded":"2024-09-12T12:12:17.033161+0000","last_fullsized":"2024-09-12T12:12:17.033161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.032939+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:12:17.032939+0000","last_peered":"2024-09-12T12:12:17.032939+0000","last_clean":"2024-09-12T12:12:17.032939+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:12:17.032939+0000","last_undegraded":"2024-09-12T12:12:17.032939+0000","last_fullsized":"2024-09-12T12:12:17.032939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-12T12:12:17.028701+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:12:17.028701+0000","last_peered":"2024-09-12T12:12:17.028701+0000","last_clean":"2024-09-12T12:12:17.028701+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:12:17.028701+0000","last_undegraded":"2024-09-12T12:12:17.028701+0000","last_fullsized":"2024-09-12T12:12:17.028701+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":413,"num_read_kb":273,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281347,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":444201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":1095286,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939687,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":444437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168621,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":444357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397554,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1036289,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":443826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1035580,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":444656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":504180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":502830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":933888,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:12:30.879 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T12:12:30.879 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T12:12:30.879 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T12:12:30.879 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T12:12:31.129 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:31 smithi171 ceph-mon[35098]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:31.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:31 smithi137 ceph-mon[30950]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:31.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:31 smithi189 ceph-mon[35475]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:12:31.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 -- 172.21.15.137:0/2775747686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 msgr2=0x7fbe34103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:31.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2775747686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34103110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009cd0 tx=0x7fbe2402a8e0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 -- 172.21.15.137:0/2775747686 shutdown_connections 2024-09-12T12:12:31.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2775747686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe341082a0 0x7fbe3410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2775747686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe341036e0 0x7fbe34107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2775747686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34103110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 -- 172.21.15.137:0/2775747686 >> 172.21.15.137:0/2775747686 conn(0x7fbe340fc7d0 msgr2=0x7fbe340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 -- 172.21.15.137:0/2775747686 shutdown_connections 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.658+0000 7fbe389e6640 1 -- 172.21.15.137:0/2775747686 wait complete. 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 Processor -- start 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 -- start start 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe341036e0 0x7fbe34198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe341082a0 0x7fbe3419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe34112100 con 0x7fbe34102d30 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbe34111f80 con 0x7fbe341082a0 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe389e6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbe34112280 con 0x7fbe341036e0 2024-09-12T12:12:31.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60808/0 (socket says 172.21.15.137:60808) 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 -- 172.21.15.137:0/2316095462 learned_addr learned my addr 172.21.15.137:0/2316095462 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe32d9d640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe341082a0 0x7fbe3419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe31d9b640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe341036e0 0x7fbe34198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 -- 172.21.15.137:0/2316095462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe341036e0 msgr2=0x7fbe34198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe341036e0 0x7fbe34198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 -- 172.21.15.137:0/2316095462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe341082a0 msgr2=0x7fbe3419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe341082a0 0x7fbe3419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 -- 172.21.15.137:0/2316095462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3419d520 con 0x7fbe34102d30 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe32d9d640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe341082a0 0x7fbe3419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe31d9b640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe341036e0 0x7fbe34198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.659+0000 7fbe3259c640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34198570 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe2402ae00 tx=0x7fbe24003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:31.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.660+0000 7fbe237fe640 1 -- 172.21.15.137:0/2316095462 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24004240 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.660+0000 7fbe237fe640 1 -- 172.21.15.137:0/2316095462 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe240305a0 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.660+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3419d7b0 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.660+0000 7fbe237fe640 1 -- 172.21.15.137:0/2316095462 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2403a660 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.660+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe341a50f0 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.661+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe34103110 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.662+0000 7fbe237fe640 1 -- 172.21.15.137:0/2316095462 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbe24030ac0 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.662+0000 7fbe237fe640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbe1c0a2c10 0x7fbe1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.662+0000 7fbe31d9b640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbe1c0a2c10 0x7fbe1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.663+0000 7fbe237fe640 1 -- 172.21.15.137:0/2316095462 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fbe2402e030 con 0x7fbe34102d30 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.663+0000 7fbe31d9b640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbe1c0a2c10 0x7fbe1c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe340feda0 tx=0x7fbe28005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:31.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.667+0000 7fbe237fe640 1 -- 172.21.15.137:0/2316095462 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe240098b0 con 0x7fbe34102d30 2024-09-12T12:12:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.869+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbe3410a2b0 con 0x7fbe34102d30 2024-09-12T12:12:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.870+0000 7fbe237fe640 1 -- 172.21.15.137:0/2316095462 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbe240bd160 con 0x7fbe34102d30 2024-09-12T12:12:31.870 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:31.871 INFO:teuthology.orchestra.run.smithi137.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T12:12:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbe1c0a2c10 msgr2=0x7fbe1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbe1c0a2c10 0x7fbe1c0a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbe340feda0 tx=0x7fbe28005d90 comp rx=0 tx=0).stop 2024-09-12T12:12:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 msgr2=0x7fbe34198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34198570 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe2402ae00 tx=0x7fbe24003ba0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 shutdown_connections 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fbe1c0a2c10 0x7fbe1c0a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe341082a0 0x7fbe3419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe341036e0 0x7fbe34198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 --2- 172.21.15.137:0/2316095462 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe34102d30 0x7fbe34198570 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 >> 172.21.15.137:0/2316095462 conn(0x7fbe340fc7d0 msgr2=0x7fbe34105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 shutdown_connections 2024-09-12T12:12:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:31.872+0000 7fbe389e6640 1 -- 172.21.15.137:0/2316095462 wait complete. 2024-09-12T12:12:32.453 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:32 smithi171 ceph-mon[35098]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:32.453 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:32 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2316095462' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T12:12:32.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:32 smithi137 ceph-mon[30950]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:32.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:32 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2316095462' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T12:12:32.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:32 smithi189 ceph-mon[35475]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T12:12:32.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:32 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2316095462' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T12:12:32.622 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T12:12:32.622 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:12:32.868 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:33.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 -- 172.21.15.137:0/927648750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710102790 msgr2=0x7ff7101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:33.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 --2- 172.21.15.137:0/927648750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710102790 0x7ff7101067e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009900 tx=0x7ff6f802a410 comp rx=0 tx=0).stop 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 -- 172.21.15.137:0/927648750 shutdown_connections 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 --2- 172.21.15.137:0/927648750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff710106f10 0x7ff71010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 --2- 172.21.15.137:0/927648750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710102790 0x7ff7101067e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 --2- 172.21.15.137:0/927648750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff710101de0 0x7ff7101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 -- 172.21.15.137:0/927648750 >> 172.21.15.137:0/927648750 conn(0x7ff7100fd8c0 msgr2=0x7ff7100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 -- 172.21.15.137:0/927648750 shutdown_connections 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 -- 172.21.15.137:0/927648750 wait complete. 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 Processor -- start 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.352+0000 7ff71536c640 1 -- start start 2024-09-12T12:12:33.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff71536c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff710101de0 0x7ff710198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff71536c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff710102790 0x7ff710198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff71536c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710106f10 0x7ff71019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff71536c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff710111f90 con 0x7ff710106f10 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff71536c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff710111e10 con 0x7ff710101de0 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff71536c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff710112110 con 0x7ff710102790 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710106f10 0x7ff71019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710106f10 0x7ff71019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60826/0 (socket says 172.21.15.137:60826) 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 -- 172.21.15.137:0/630264978 learned_addr learned my addr 172.21.15.137:0/630264978 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70effd640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff710101de0 0x7ff710198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:33.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 -- 172.21.15.137:0/630264978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff710102790 msgr2=0x7ff710198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff710102790 0x7ff710198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 -- 172.21.15.137:0/630264978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff710101de0 msgr2=0x7ff710198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff710101de0 0x7ff710198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 -- 172.21.15.137:0/630264978 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71019d4c0 con 0x7ff710106f10 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70f7fe640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710106f10 0x7ff71019cde0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff70800cad0 tx=0x7ff70800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70cff9640 1 -- 172.21.15.137:0/630264978 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70800ad60 con 0x7ff710106f10 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70cff9640 1 -- 172.21.15.137:0/630264978 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff708005020 con 0x7ff710106f10 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff70cff9640 1 -- 172.21.15.137:0/630264978 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70800f6b0 con 0x7ff710106f10 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.353+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71019d7b0 con 0x7ff710106f10 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.354+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7101a5090 con 0x7ff710106f10 2024-09-12T12:12:33.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.354+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7101021c0 con 0x7ff710106f10 2024-09-12T12:12:33.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.356+0000 7ff70cff9640 1 -- 172.21.15.137:0/630264978 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff708005560 con 0x7ff710106f10 2024-09-12T12:12:33.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.356+0000 7ff70cff9640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff6fc0a2c10 0x7ff6fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:33.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.357+0000 7ff70effd640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff6fc0a2c10 0x7ff6fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:33.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.357+0000 7ff70cff9640 1 -- 172.21.15.137:0/630264978 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7ff7080cf240 con 0x7ff710106f10 2024-09-12T12:12:33.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.357+0000 7ff70effd640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff6fc0a2c10 0x7ff6fc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff700009990 tx=0x7ff700009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:33.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.361+0000 7ff70cff9640 1 -- 172.21.15.137:0/630264978 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff708014030 con 0x7ff710106f10 2024-09-12T12:12:33.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.563+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff710109b60 con 0x7ff710106f10 2024-09-12T12:12:33.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.563+0000 7ff70cff9640 1 -- 172.21.15.137:0/630264978 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7ff70809c070 con 0x7ff710106f10 2024-09-12T12:12:33.564 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:33.565 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"btime":"2024-09-12T12:11:24:550801+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:11:24.095061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:12:33.565 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-09-12T12:12:33.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff6fc0a2c10 msgr2=0x7ff6fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:33.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff6fc0a2c10 0x7ff6fc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff700009990 tx=0x7ff700009110 comp rx=0 tx=0).stop 2024-09-12T12:12:33.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710106f10 msgr2=0x7ff71019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:33.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710106f10 0x7ff71019cde0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff70800cad0 tx=0x7ff70800ced0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 shutdown_connections 2024-09-12T12:12:33.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff6fc0a2c10 0x7ff6fc0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff710106f10 0x7ff71019cde0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff710102790 0x7ff710198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.565+0000 7ff71536c640 1 --2- 172.21.15.137:0/630264978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff710101de0 0x7ff710198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:33.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.566+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 >> 172.21.15.137:0/630264978 conn(0x7ff7100fd8c0 msgr2=0x7ff7100fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:33.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.566+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 shutdown_connections 2024-09-12T12:12:33.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:33.566+0000 7ff71536c640 1 -- 172.21.15.137:0/630264978 wait complete. 2024-09-12T12:12:34.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:34 smithi137 ceph-mon[30950]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:34.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:34 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/630264978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:34.285 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:12:34.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:34 smithi171 ceph-mon[35098]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:34.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:34 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/630264978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:34.534 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:34.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:34 smithi189 ceph-mon[35475]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:34.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:34 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/630264978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 -- 172.21.15.137:0/2799430051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c106a80 msgr2=0x7f299c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2799430051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c106a80 0x7f299c106e60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2990009930 tx=0x7f299002a420 comp rx=0 tx=0).stop 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 -- 172.21.15.137:0/2799430051 shutdown_connections 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2799430051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c101420 0x7f299c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2799430051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c100a80 0x7f299c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2799430051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c106a80 0x7f299c106e60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 -- 172.21.15.137:0/2799430051 >> 172.21.15.137:0/2799430051 conn(0x7f299c0fc7a0 msgr2=0x7f299c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 -- 172.21.15.137:0/2799430051 shutdown_connections 2024-09-12T12:12:35.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 -- 172.21.15.137:0/2799430051 wait complete. 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 Processor -- start 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.028+0000 7f29a3f15640 1 -- start start 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c100a80 0x7f299c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c101420 0x7f299c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c106a80 0x7f299c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f299c110f20 con 0x7f299c100a80 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c110da0 con 0x7f299c101420 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1110a0 con 0x7f299c106a80 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c100a80 0x7f299c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c100a80 0x7f299c1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60844/0 (socket says 172.21.15.137:60844) 2024-09-12T12:12:35.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 -- 172.21.15.137:0/2345610668 learned_addr learned my addr 172.21.15.137:0/2345610668 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a24b2640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c106a80 0x7f299c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a14b0640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c101420 0x7f299c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 -- 172.21.15.137:0/2345610668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c106a80 msgr2=0x7f299c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c106a80 0x7f299c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 -- 172.21.15.137:0/2345610668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c101420 msgr2=0x7f299c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c101420 0x7f299c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 -- 172.21.15.137:0/2345610668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c19c490 con 0x7f299c100a80 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a1cb1640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c100a80 0x7f299c1974e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2990009a50 tx=0x7f2990004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f298effd640 1 -- 172.21.15.137:0/2345610668 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990046070 con 0x7f299c100a80 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c19c720 con 0x7f299c100a80 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f298effd640 1 -- 172.21.15.137:0/2345610668 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29900044a0 con 0x7f299c100a80 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f298effd640 1 -- 172.21.15.137:0/2345610668 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990035050 con 0x7f299c100a80 2024-09-12T12:12:35.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.029+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c06a790 con 0x7f299c100a80 2024-09-12T12:12:35.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.030+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299c1021c0 con 0x7f299c100a80 2024-09-12T12:12:35.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.031+0000 7f298effd640 1 -- 172.21.15.137:0/2345610668 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2990049070 con 0x7f299c100a80 2024-09-12T12:12:35.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.032+0000 7f298effd640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f29880a2c10 0x7f29880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:35.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.032+0000 7f29a14b0640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f29880a2c10 0x7f29880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:35.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.032+0000 7f298effd640 1 -- 172.21.15.137:0/2345610668 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f29900f0e90 con 0x7f299c100a80 2024-09-12T12:12:35.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.032+0000 7f29a14b0640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f29880a2c10 0x7f29880a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f299c0fee40 tx=0x7f2998008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:35.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.036+0000 7f298effd640 1 -- 172.21.15.137:0/2345610668 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f299007e1b0 con 0x7f299c100a80 2024-09-12T12:12:35.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.238+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f299c06b1b0 con 0x7f299c100a80 2024-09-12T12:12:35.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.239+0000 7f298effd640 1 -- 172.21.15.137:0/2345610668 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7f29900bdcc0 con 0x7f299c100a80 2024-09-12T12:12:35.239 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:35.239 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"btime":"2024-09-12T12:11:24:550801+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:11:24.095061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:12:35.240 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f29880a2c10 msgr2=0x7f29880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f29880a2c10 0x7f29880a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f299c0fee40 tx=0x7f2998008040 comp rx=0 tx=0).stop 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c100a80 msgr2=0x7f299c1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c100a80 0x7f299c1974e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2990009a50 tx=0x7f2990004270 comp rx=0 tx=0).stop 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 shutdown_connections 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f29880a2c10 0x7f29880a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c106a80 0x7f299c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c101420 0x7f299c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 --2- 172.21.15.137:0/2345610668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c100a80 0x7f299c1974e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:35.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 >> 172.21.15.137:0/2345610668 conn(0x7f299c0fc7a0 msgr2=0x7f299c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:35.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 shutdown_connections 2024-09-12T12:12:35.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:35.241+0000 7f29a3f15640 1 -- 172.21.15.137:0/2345610668 wait complete. 2024-09-12T12:12:35.968 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:12:36.230 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:36.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:36 smithi171 ceph-mon[35098]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:36.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:36 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2345610668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:36.519 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:36 smithi137 ceph-mon[30950]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:36.520 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:36 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2345610668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:36.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:36 smithi189 ceph-mon[35475]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:36.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:36 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2345610668' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:36.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 -- 172.21.15.137:0/226751550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102570 msgr2=0x7f670c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:36.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 --2- 172.21.15.137:0/226751550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102570 0x7f670c1029b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6700009900 tx=0x7f670002a480 comp rx=0 tx=0).stop 2024-09-12T12:12:36.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 -- 172.21.15.137:0/226751550 shutdown_connections 2024-09-12T12:12:36.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 --2- 172.21.15.137:0/226751550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f670c102ef0 0x7f670c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 --2- 172.21.15.137:0/226751550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102570 0x7f670c1029b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 --2- 172.21.15.137:0/226751550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f670c100e20 0x7f670c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 -- 172.21.15.137:0/226751550 >> 172.21.15.137:0/226751550 conn(0x7f670c0fc7d0 msgr2=0x7f670c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 -- 172.21.15.137:0/226751550 shutdown_connections 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 -- 172.21.15.137:0/226751550 wait complete. 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.780+0000 7f67129c1640 1 Processor -- start 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f67129c1640 1 -- start start 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f67129c1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f670c100e20 0x7f670c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f67129c1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f670c102570 0x7f670c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f67129c1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102ef0 0x7f670c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:36.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f67129c1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f670c110f80 con 0x7f670c102ef0 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f67129c1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f670c110e00 con 0x7f670c100e20 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f67129c1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f670c111100 con 0x7f670c102570 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102ef0 0x7f670c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102ef0 0x7f670c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60856/0 (socket says 172.21.15.137:60856) 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 -- 172.21.15.137:0/2567371013 learned_addr learned my addr 172.21.15.137:0/2567371013 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f670bfff640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f670c100e20 0x7f670c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f670b7fe640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f670c102570 0x7f670c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 -- 172.21.15.137:0/2567371013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f670c102570 msgr2=0x7f670c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:36.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f670c102570 0x7f670c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 -- 172.21.15.137:0/2567371013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f670c100e20 msgr2=0x7f670c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f670c100e20 0x7f670c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.781+0000 7f6710f5e640 1 -- 172.21.15.137:0/2567371013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c1a69b0 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.782+0000 7f6710f5e640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102ef0 0x7f670c0fd700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f670400cad0 tx=0x7f670400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.782+0000 7f67097fa640 1 -- 172.21.15.137:0/2567371013 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670400ae20 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.782+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c1a6ca0 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.782+0000 7f67097fa640 1 -- 172.21.15.137:0/2567371013 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6704005020 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.782+0000 7f67097fa640 1 -- 172.21.15.137:0/2567371013 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670400f6f0 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.782+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c1a71e0 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.783+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c101200 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.784+0000 7f67097fa640 1 -- 172.21.15.137:0/2567371013 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f67040051d0 con 0x7f670c102ef0 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.785+0000 7f67097fa640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66f40a2c10 0x7f66f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:36.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.785+0000 7f670bfff640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66f40a2c10 0x7f66f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:36.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.785+0000 7f67097fa640 1 -- 172.21.15.137:0/2567371013 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f67040cf570 con 0x7f670c102ef0 2024-09-12T12:12:36.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.785+0000 7f670bfff640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66f40a2c10 0x7f66f40a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005fd0 tx=0x7f66fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:36.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.788+0000 7f67097fa640 1 -- 172.21.15.137:0/2567371013 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6704014030 con 0x7f670c102ef0 2024-09-12T12:12:36.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.956+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f670c1a7470 con 0x7f670c102ef0 2024-09-12T12:12:36.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.958+0000 7f67097fa640 1 -- 172.21.15.137:0/2567371013 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22734 (secure 0 0 0) 0x7f670409c3a0 con 0x7f670c102ef0 2024-09-12T12:12:36.958 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:36.959 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":84,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:12:17.022250+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:12:36.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66f40a2c10 msgr2=0x7f66f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66f40a2c10 0x7f66f40a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f66fc005fd0 tx=0x7f66fc00b390 comp rx=0 tx=0).stop 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102ef0 msgr2=0x7f670c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102ef0 0x7f670c0fd700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f670400cad0 tx=0x7f670400cf90 comp rx=0 tx=0).stop 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 shutdown_connections 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f66f40a2c10 0x7f66f40a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c102ef0 0x7f670c0fd700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f670c102570 0x7f670c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 --2- 172.21.15.137:0/2567371013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f670c100e20 0x7f670c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 >> 172.21.15.137:0/2567371013 conn(0x7f670c0fc7d0 msgr2=0x7f670c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 shutdown_connections 2024-09-12T12:12:36.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:36.960+0000 7f67129c1640 1 -- 172.21.15.137:0/2567371013 wait complete. 2024-09-12T12:12:37.330 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:37 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2567371013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:12:37.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:37 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2567371013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:12:37.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:37 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2567371013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:12:37.694 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:12:37.945 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:38.368 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:38 smithi137 ceph-mon[30950]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:12:38.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 -- 172.21.15.137:0/2237669801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102ef0 msgr2=0x7f24a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:38.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 --2- 172.21.15.137:0/2237669801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102ef0 0x7f24a810da40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2498009ca0 tx=0x7f249802a950 comp rx=0 tx=0).stop 2024-09-12T12:12:38.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 -- 172.21.15.137:0/2237669801 shutdown_connections 2024-09-12T12:12:38.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 --2- 172.21.15.137:0/2237669801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102ef0 0x7f24a810da40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 --2- 172.21.15.137:0/2237669801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8102570 0x7f24a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 --2- 172.21.15.137:0/2237669801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a8100e20 0x7f24a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 -- 172.21.15.137:0/2237669801 >> 172.21.15.137:0/2237669801 conn(0x7f24a80fc7d0 msgr2=0x7f24a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 -- 172.21.15.137:0/2237669801 shutdown_connections 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 -- 172.21.15.137:0/2237669801 wait complete. 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 Processor -- start 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 -- start start 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8100e20 0x7f24a80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102570 0x7f24a80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.449+0000 7f24aff7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a8102ef0 0x7f24a80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24aff7b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8110f80 con 0x7f24a8102570 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24aff7b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8110e00 con 0x7f24a8100e20 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24aff7b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24a8111100 con 0x7f24a8102ef0 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102570 0x7f24a80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102570 0x7f24a80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60872/0 (socket says 172.21.15.137:60872) 2024-09-12T12:12:38.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 -- 172.21.15.137:0/1420482337 learned_addr learned my addr 172.21.15.137:0/1420482337 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24add17640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8100e20 0x7f24a80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 -- 172.21.15.137:0/1420482337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a8102ef0 msgr2=0x7f24a80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a8102ef0 0x7f24a80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 -- 172.21.15.137:0/1420482337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8100e20 msgr2=0x7f24a80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8100e20 0x7f24a80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 -- 172.21.15.137:0/1420482337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a81a6890 con 0x7f24a8102570 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24add17640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8100e20 0x7f24a80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24ad516640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102570 0x7f24a80fd1c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e9f0 tx=0x7f249c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24a6ffd640 1 -- 172.21.15.137:0/1420482337 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c00cd80 con 0x7f24a8102570 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a81a6b80 con 0x7f24a8102570 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24a6ffd640 1 -- 172.21.15.137:0/1420482337 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249c010910 con 0x7f24a8102570 2024-09-12T12:12:38.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24a6ffd640 1 -- 172.21.15.137:0/1420482337 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c018990 con 0x7f24a8102570 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.450+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a81a70c0 con 0x7f24a8102570 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.451+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a8101200 con 0x7f24a8102570 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.452+0000 7f24a6ffd640 1 -- 172.21.15.137:0/1420482337 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f249c010ac0 con 0x7f24a8102570 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.453+0000 7f24a6ffd640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f24900a2c10 0x7f24900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.453+0000 7f24add17640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f24900a2c10 0x7f24900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.453+0000 7f24a6ffd640 1 -- 172.21.15.137:0/1420482337 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f249c0cf950 con 0x7f24a8102570 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.454+0000 7f24add17640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f24900a2c10 0x7f24900a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24a000baa0 tx=0x7f24a0005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:38.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.456+0000 7f24a6ffd640 1 -- 172.21.15.137:0/1420482337 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f249c09c780 con 0x7f24a8102570 2024-09-12T12:12:38.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:38 smithi189 ceph-mon[35475]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:12:38.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.657+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24a80fe1b0 con 0x7f24a8102570 2024-09-12T12:12:38.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.658+0000 7f24a6ffd640 1 -- 172.21.15.137:0/1420482337 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7f249c09c080 con 0x7f24a8102570 2024-09-12T12:12:38.659 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:38.659 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"btime":"2024-09-12T12:11:24:550801+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:11:24.095061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:12:38.659 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-09-12T12:12:38.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f24900a2c10 msgr2=0x7f24900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:38.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f24900a2c10 0x7f24900a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24a000baa0 tx=0x7f24a0005d90 comp rx=0 tx=0).stop 2024-09-12T12:12:38.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102570 msgr2=0x7f24a80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:38.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102570 0x7f24a80fd1c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e9f0 tx=0x7f249c00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 shutdown_connections 2024-09-12T12:12:38.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f24900a2c10 0x7f24900a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a8102ef0 0x7f24a80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24a8102570 0x7f24a80fd1c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 --2- 172.21.15.137:0/1420482337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a8100e20 0x7f24a80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:38.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 >> 172.21.15.137:0/1420482337 conn(0x7f24a80fc7d0 msgr2=0x7f24a806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:38.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 shutdown_connections 2024-09-12T12:12:38.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:38.660+0000 7f24aff7b640 1 -- 172.21.15.137:0/1420482337 wait complete. 2024-09-12T12:12:38.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:38 smithi171 ceph-mon[35098]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:12:39.475 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:39 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1420482337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:39.507 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T12:11:13.196665+0000', 'modified': '2024-09-12T12:11:24.095061+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24476, 'mds_1': 24475, 'mds_2': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6833/4107944214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 4107944214}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 4107944214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/1673181827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 1673181827}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 1673181827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/2972240277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 2972240277}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 2972240277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.189:6837/2107760146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 2107760146}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 2107760146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.189:6839/4067701297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 4067701297}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 4067701297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.171:6839/1705495422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1705495422}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1705495422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24476, 24475, 24454]} 2024-09-12T12:12:39.507 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T12:12:39.507 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T12:12:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:39 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1420482337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:39.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:39 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1420482337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:40.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.234+0000 7f34c7ee5640 1 -- 172.21.15.137:0/3220964884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c0106ba0 msgr2=0x7f34c0106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.234+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/3220964884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c0106ba0 0x7f34c0106f80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009ca0 tx=0x7f34b002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- 172.21.15.137:0/3220964884 shutdown_connections 2024-09-12T12:12:40.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/3220964884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34c0100a70 0x7f34c010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/3220964884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34c01000d0 0x7f34c0100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/3220964884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c0106ba0 0x7f34c0106f80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- 172.21.15.137:0/3220964884 >> 172.21.15.137:0/3220964884 conn(0x7f34c0077210 msgr2=0x7f34c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- 172.21.15.137:0/3220964884 shutdown_connections 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- 172.21.15.137:0/3220964884 wait complete. 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 Processor -- start 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- start start 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c01000d0 0x7f34c01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34c0100a70 0x7f34c0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34c0106ba0 0x7f34c019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34c0112130 con 0x7f34c01000d0 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34c0111fb0 con 0x7f34c0100a70 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.235+0000 7f34c7ee5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34c01122b0 con 0x7f34c0106ba0 2024-09-12T12:12:40.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c01000d0 0x7f34c01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c01000d0 0x7f34c01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57332/0 (socket says 172.21.15.137:57332) 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 -- 172.21.15.137:0/1692539664 learned_addr learned my addr 172.21.15.137:0/1692539664 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5480640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34c0100a70 0x7f34c0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 -- 172.21.15.137:0/1692539664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34c0106ba0 msgr2=0x7f34c019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c6482640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34c0106ba0 0x7f34c019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34c0106ba0 0x7f34c019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 -- 172.21.15.137:0/1692539664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34c0100a70 msgr2=0x7f34c0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34c0100a70 0x7f34c0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 -- 172.21.15.137:0/1692539664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c019d560 con 0x7f34c01000d0 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5480640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34c0100a70 0x7f34c0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c5c81640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c01000d0 0x7f34c01985b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f34b0006010 tx=0x7f34b0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34beffd640 1 -- 172.21.15.137:0/1692539664 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00317d0 con 0x7f34c01000d0 2024-09-12T12:12:40.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34beffd640 1 -- 172.21.15.137:0/1692539664 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b0031970 con 0x7f34c01000d0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c019d7f0 con 0x7f34c01000d0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34beffd640 1 -- 172.21.15.137:0/1692539664 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0030d70 con 0x7f34c01000d0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.236+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c01a5130 con 0x7f34c01000d0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34beffd640 1 -- 172.21.15.137:0/1692539664 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f34b0004040 con 0x7f34c01000d0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 Processor -- start 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 -- start start 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795559db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a795564670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795566fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7954c0090 con 0x55a795564bb0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a795467a70 con 0x55a7955599d0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34ca2be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7954bc050 con 0x55a79555a2f0 2024-09-12T12:12:40.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34c5480640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795566fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34c5480640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795566fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57342/0 (socket says 172.21.15.137:57342) 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34c5480640 1 -- 172.21.15.137:0/2449270025 learned_addr learned my addr 172.21.15.137:0/2449270025 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34c5c81640 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a795564670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34c6482640 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795559db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.239+0000 7f34beffd640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34a80a2c10 0x7f34a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c5480640 1 -- 172.21.15.137:0/2449270025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 msgr2=0x55a795564670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c5480640 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a795564670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c5480640 1 -- 172.21.15.137:0/2449270025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 msgr2=0x55a795559db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c5480640 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795559db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c6482640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34a80a2c10 0x7f34a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c5480640 1 -- 172.21.15.137:0/2449270025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a795474130 con 0x55a795564bb0 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c5c81640 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a795564670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:40.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34beffd640 1 -- 172.21.15.137:0/1692539664 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f34b00f0730 con 0x7f34c01000d0 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c5480640 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795566fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007800 tx=0x7f34b4007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b280f00a941d86d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34a7fff640 1 -- 172.21.15.137:0/2449270025 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b400d950 con 0x55a795564bb0 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34a7fff640 1 -- 172.21.15.137:0/2449270025 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b4018070 con 0x55a795564bb0 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34c6482640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34a80a2c10 0x7f34a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34b800aa00 tx=0x7f34b80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.240+0000 7f34a7fff640 1 -- 172.21.15.137:0/2449270025 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b40123e0 con 0x55a795564bb0 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 -- 172.21.15.137:0/2449270025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 msgr2=0x55a795566fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795566fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007800 tx=0x7f34b4007cc0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 -- 172.21.15.137:0/2449270025 shutdown_connections 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795566fa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a795564670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 --2- 172.21.15.137:0/2449270025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795559db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 -- 172.21.15.137:0/2449270025 >> 172.21.15.137:0/2449270025 conn(0x55a795555f60 msgr2=0x55a795556340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:40.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 -- 172.21.15.137:0/2449270025 shutdown_connections 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 -- 172.21.15.137:0/2449270025 wait complete. 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 -- start start 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.241+0000 7f34ca2be740 1 Processor -- start 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795564350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a7955616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795561be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a795567780 con 0x55a795564bb0 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a79547af20 con 0x55a7955599d0 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a795567900 con 0x55a79555a2f0 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 10 client.0 _ll_register_callbacks cb 0x55a795444030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795561be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795561be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57352/0 (socket says 172.21.15.137:57352) 2024-09-12T12:12:40.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 -- 172.21.15.137:0/1823493126 learned_addr learned my addr 172.21.15.137:0/1823493126 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c5480640 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a7955616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c5c81640 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795564350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 -- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 msgr2=0x55a7955616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a7955616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 -- 172.21.15.137:0/1823493126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 msgr2=0x55a795564350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795564350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 -- 172.21.15.137:0/1823493126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7952312b0 con 0x55a795564bb0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34c6482640 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795561be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f34b8014670 tx=0x7f34b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0554cb5feed06db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34a57fa640 1 -- 172.21.15.137:0/1823493126 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b800f040 con 0x55a795564bb0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b4018070 con 0x55a795564bb0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.242+0000 7f34a57fa640 1 -- 172.21.15.137:0/1823493126 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f34b8013840 con 0x55a795564bb0 2024-09-12T12:12:40.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34a57fa640 1 -- 172.21.15.137:0/1823493126 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f34b8013a40 con 0x55a795564bb0 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 10 client.14991 fetch_fsmap learned FSMap version 23 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a795572ff0 con 0x55a795564bb0 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34a57fa640 1 -- 172.21.15.137:0/1823493126 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b4018070 con 0x55a795564bb0 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34a57fa640 1 -- 172.21.15.137:0/1823493126 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f34b8029400 con 0x55a795564bb0 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 10 client.14991 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 10 client.14991 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 20 client.14991 populate_metadata read hostname 'smithi137' 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955756a0 0x55a795576b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 4 client.14991 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 --> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7954cd600 con 0x55a7955756a0 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.243+0000 7f34c5c81640 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955756a0 0x55a795576b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.244+0000 7f34a57fa640 10 client.14991 ms_handle_connect on v2:172.21.15.189:6836/2107760146 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.244+0000 7f34c5c81640 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955756a0 0x55a795576b10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecb6f3f88bbe7447 server_cookie=41ceab99478e2f53 in_seq=0 out_seq=0 2024-09-12T12:12:40.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.244+0000 7f34a57fa640 1 -- 172.21.15.137:0/1823493126 <== mds.2 v2:172.21.15.189:6836/2107760146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7954cd600 con 0x55a7955756a0 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.244+0000 7f34a57fa640 10 client.14991 handle_command_reply: tid=0 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.244+0000 7f34ca2be740 1 client.14991 shutdown 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 msgr2=0x55a795561be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795561be0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f34b8014670 tx=0x7f34b8008040 comp rx=0 tx=0).stop 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 shutdown_connections 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955756a0 0x55a795576b10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a795564bb0 0x55a795561be0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a79555a2f0 0x55a7955616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34a57fa640 0 client.14991 ms_handle_reset on v2:172.21.15.189:6836/2107760146 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 --2- 172.21.15.137:0/1823493126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795564350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 >> 172.21.15.137:0/1823493126 conn(0x55a795555f60 msgr2=0x55a79555bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 shutdown_connections 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 1 -- 172.21.15.137:0/1823493126 wait complete. 2024-09-12T12:12:40.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.245+0000 7f34ca2be740 20 client.14991 trim_cache size 0 max 16384 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 Processor -- start 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 -- start start 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795561660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795561ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a795564bb0 0x55a79555ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a795574e50 con 0x55a79555a2f0 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a795575520 con 0x55a7955599d0 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.262+0000 7f34ca2be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a795575bf0 con 0x55a795564bb0 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795561ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795561ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57362/0 (socket says 172.21.15.137:57362) 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 -- 172.21.15.137:0/3131087841 learned_addr learned my addr 172.21.15.137:0/3131087841 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c6482640 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a795564bb0 0x55a79555ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5c81640 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795561660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 -- 172.21.15.137:0/3131087841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a795564bb0 msgr2=0x55a79555ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a795564bb0 0x55a79555ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 -- 172.21.15.137:0/3131087841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 msgr2=0x55a795561660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795561660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 -- 172.21.15.137:0/3131087841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a795555d90 con 0x55a79555a2f0 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c6482640 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a795564bb0 0x55a79555ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34c5480640 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795561ba0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007720 tx=0x7f34b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9853ac2cd989773 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34a77fe640 1 -- 172.21.15.137:0/3131087841 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4013480 con 0x55a79555a2f0 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34a77fe640 1 -- 172.21.15.137:0/3131087841 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b4018040 con 0x55a79555a2f0 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.263+0000 7f34a77fe640 1 -- 172.21.15.137:0/3131087841 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b401d9c0 con 0x55a79555a2f0 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 -- 172.21.15.137:0/3131087841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 msgr2=0x55a795561ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795561ba0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007720 tx=0x7f34b400dae0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 -- 172.21.15.137:0/3131087841 shutdown_connections 2024-09-12T12:12:40.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a795564bb0 0x55a79555ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795561ba0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3131087841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7955599d0 0x55a795561660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 -- 172.21.15.137:0/3131087841 >> 172.21.15.137:0/3131087841 conn(0x55a795555f60 msgr2=0x55a795556340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 -- 172.21.15.137:0/3131087841 shutdown_connections 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 -- 172.21.15.137:0/3131087841 wait complete. 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 -- start start 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.264+0000 7f34ca2be740 1 Processor -- start 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7955599d0 0x55a7955783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795578930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a795564bb0 0x55a7955ddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a795576990 con 0x55a79555a2f0 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a7955762c0 con 0x55a795564bb0 2024-09-12T12:12:40.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a795577060 con 0x55a7955599d0 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 10 client.0 _ll_register_callbacks cb 0x7f34b8006da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795578930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795578930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57378/0 (socket says 172.21.15.137:57378) 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 -- 172.21.15.137:0/3930047159 learned_addr learned my addr 172.21.15.137:0/3930047159 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 -- 172.21.15.137:0/3930047159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7955599d0 msgr2=0x55a7955783f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7955599d0 0x55a7955783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 -- 172.21.15.137:0/3930047159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a795564bb0 msgr2=0x55a7955ddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a795564bb0 0x55a7955ddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 -- 172.21.15.137:0/3930047159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7955de360 con 0x55a79555a2f0 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34c5480640 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795578930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f34b400d530 tx=0x7f34b40136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5affdec1bcf50e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34a57fa640 1 -- 172.21.15.137:0/3930047159 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b401d740 con 0x55a79555a2f0 2024-09-12T12:12:40.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34a57fa640 1 -- 172.21.15.137:0/3930047159 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f34b401d8e0 con 0x55a79555a2f0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b4018040 con 0x55a79555a2f0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34a57fa640 1 -- 172.21.15.137:0/3930047159 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f34b401dae0 con 0x55a79555a2f0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 10 client.14997 fetch_fsmap learned FSMap version 23 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.265+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a7955df640 con 0x55a79555a2f0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34a57fa640 1 -- 172.21.15.137:0/3930047159 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34b4018040 con 0x55a79555a2f0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34a57fa640 1 -- 172.21.15.137:0/3930047159 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f34b403caa0 con 0x55a79555a2f0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34ca2be740 10 client.14997 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34ca2be740 10 client.14997 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34ca2be740 20 client.14997 populate_metadata read hostname 'smithi137' 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955e3ae0 0x55a7955e27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34ca2be740 4 client.14997 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 --> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] -- command(tid 0: {"prefix": "status"}) -- 0x55a7954c7b10 con 0x55a7955e3ae0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.266+0000 7f34c5c81640 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955e3ae0 0x55a7955e27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:40.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34c5c81640 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955e3ae0 0x55a7955e27f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30725e84876dd4cb server_cookie=e1020967156a25b6 in_seq=0 out_seq=0 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34a57fa640 10 client.14997 ms_handle_connect on v2:172.21.15.189:6836/2107760146 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34a57fa640 1 -- 172.21.15.137:0/3930047159 <== mds.2 v2:172.21.15.189:6836/2107760146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a7954c7b10 con 0x55a7955e3ae0 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34a57fa640 10 client.14997 handle_command_reply: tid=0 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 client.14997 shutdown 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 msgr2=0x55a795578930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795578930 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f34b400d530 tx=0x7f34b40136b0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 2, 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24454, 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 78.719952450999997, 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 84, 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 112.40910312600001 2024-09-12T12:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 shutdown_connections 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x55a7955e3ae0 0x55a7955e27f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a795564bb0 0x55a7955ddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a79555a2f0 0x55a795578930 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 --2- 172.21.15.137:0/3930047159 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7955599d0 0x55a7955783f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34a57fa640 0 client.14997 ms_handle_reset on v2:172.21.15.189:6836/2107760146 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 >> 172.21.15.137:0/3930047159 conn(0x55a795555f60 msgr2=0x55a795556340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 shutdown_connections 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 1 -- 172.21.15.137:0/3930047159 wait complete. 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.267+0000 7f34ca2be740 20 client.14997 trim_cache size 0 max 16384 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34a80a2c10 msgr2=0x7f34a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34a80a2c10 0x7f34a80a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34b800aa00 tx=0x7f34b80046c0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c01000d0 msgr2=0x7f34c01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c01000d0 0x7f34c01985b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f34b0006010 tx=0x7f34b0003f80 comp rx=0 tx=0).stop 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 shutdown_connections 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f34a80a2c10 0x7f34a80a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34c0106ba0 0x7f34c019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34c0100a70 0x7f34c0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 --2- 172.21.15.137:0/1692539664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34c01000d0 0x7f34c01985b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 >> 172.21.15.137:0/1692539664 conn(0x7f34c0077210 msgr2=0x7f34c0104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 shutdown_connections 2024-09-12T12:12:40.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:40.268+0000 7f34c7ee5640 1 -- 172.21.15.137:0/1692539664 wait complete. 2024-09-12T12:12:40.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:40 smithi137 ceph-mon[30950]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:40.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:40 smithi189 ceph-mon[35475]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:40.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:40 smithi171 ceph-mon[35098]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:41.013 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T12:12:41.261 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:41.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 -- 172.21.15.137:0/2246100706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8108180 msgr2=0x7fa5a810eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2246100706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8108180 0x7fa5a810eb40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009ca0 tx=0x7fa5a002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 -- 172.21.15.137:0/2246100706 shutdown_connections 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2246100706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8108180 0x7fa5a810eb40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2246100706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a8101290 0x7fa5a8107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2246100706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81008e0 0x7fa5a8100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 -- 172.21.15.137:0/2246100706 >> 172.21.15.137:0/2246100706 conn(0x7fa5a8077210 msgr2=0x7fa5a8077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 -- 172.21.15.137:0/2246100706 shutdown_connections 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.735+0000 7fa5afb08640 1 -- 172.21.15.137:0/2246100706 wait complete. 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 Processor -- start 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 -- start start 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81008e0 0x7fa5a81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8101290 0x7fa5a8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a8108180 0x7fa5a819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8112150 con 0x7fa5a8101290 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a8111fd0 con 0x7fa5a8108180 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5afb08640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a81122d0 con 0x7fa5a81008e0 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5ad0a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8101290 0x7fa5a8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5ad0a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8101290 0x7fa5a8198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57402/0 (socket says 172.21.15.137:57402) 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/2851778957 learned_addr learned my addr 172.21.15.137:0/2851778957 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81008e0 0x7fa5a81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.736+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a8108180 0x7fa5a819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/2851778957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81008e0 msgr2=0x7fa5a81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81008e0 0x7fa5a81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/2851778957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a8108180 msgr2=0x7fa5a819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a8108180 0x7fa5a819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/2851778957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a819d560 con 0x7fa5a8101290 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8101290 0x7fa5a8198af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400b450 tx=0x7fa5a400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa59affd640 1 -- 172.21.15.137:0/2851778957 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4015020 con 0x7fa5a8101290 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa59affd640 1 -- 172.21.15.137:0/2851778957 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a4011d10 con 0x7fa5a8101290 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a819d850 con 0x7fa5a8101290 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa59affd640 1 -- 172.21.15.137:0/2851778957 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400f2e0 con 0x7fa5a8101290 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.737+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81a5130 con 0x7fa5a8101290 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.739+0000 7fa59affd640 1 -- 172.21.15.137:0/2851778957 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa5a4018020 con 0x7fa5a8101290 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.739+0000 7fa5b1ee1740 1 Processor -- start 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5b1ee1740 1 -- start start 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa3afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56183f994090 con 0x56183fa2e2f0 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56183f93ba70 con 0x56183fa2d9d0 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56183f990050 con 0x56183fa38bb0 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa38670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57418/0 (socket says 172.21.15.137:57418) 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 -- 172.21.15.137:0/1903944880 learned_addr learned my addr 172.21.15.137:0/1903944880 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa59affd640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5940a2c90 0x7fa5940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 -- 172.21.15.137:0/1903944880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 msgr2=0x56183fa3afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5940a2c90 0x7fa5940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa3afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa3afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 -- 172.21.15.137:0/1903944880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 msgr2=0x56183fa2ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa2ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad8a4640 1 -- 172.21.15.137:0/1903944880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183f948130 con 0x56183fa2e2f0 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa59affd640 1 -- 172.21.15.137:0/2851778957 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fa5a40ce930 con 0x7fa5a8101290 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa2ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.740+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa3afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.741+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa38670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009a70 tx=0x7fa59c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b8565e257c0e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.741+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5940a2c90 0x7fa5940a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005fe0 tx=0x7fa5a002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.741+0000 7fa593fff640 1 -- 172.21.15.137:0/1903944880 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c017070 con 0x56183fa2e2f0 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.741+0000 7fa593fff640 1 -- 172.21.15.137:0/1903944880 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c013470 con 0x56183fa2e2f0 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.741+0000 7fa593fff640 1 -- 172.21.15.137:0/1903944880 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0125d0 con 0x56183fa2e2f0 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1903944880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 msgr2=0x56183fa38670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa38670 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009a70 tx=0x7fa59c009f30 comp rx=0 tx=0).stop 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1903944880 shutdown_connections 2024-09-12T12:12:41.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa3afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa38670 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1903944880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa2ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1903944880 >> 172.21.15.137:0/1903944880 conn(0x56183fa29f60 msgr2=0x56183fa2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1903944880 shutdown_connections 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1903944880 wait complete. 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- start start 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 Processor -- start 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 0x56183fa35700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa35c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56183fa3b780 con 0x56183fa38bb0 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56183f94ef20 con 0x56183fa2e2f0 2024-09-12T12:12:41.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.742+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56183fa3b900 con 0x56183fa2d9d0 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5b1ee1740 10 client.0 _ll_register_callbacks cb 0x56183f918030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa35c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa35c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57428/0 (socket says 172.21.15.137:57428) 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/2635323028 learned_addr learned my addr 172.21.15.137:0/2635323028 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 0x56183fa35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 msgr2=0x56183fa383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/2635323028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 msgr2=0x56183fa35700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 0x56183fa35700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/2635323028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183f7052b0 con 0x56183fa38bb0 2024-09-12T12:12:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa35c40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0035000 tx=0x7fa5a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7e942382655ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5917fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:12:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5917fa640 1 -- 172.21.15.137:0/2635323028 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a004e070 con 0x56183fa38bb0 2024-09-12T12:12:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5917fa640 1 -- 172.21.15.137:0/2635323028 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fa5a0004450 con 0x56183fa38bb0 2024-09-12T12:12:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa59c013470 con 0x56183fa38bb0 2024-09-12T12:12:41.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5917fa640 1 -- 172.21.15.137:0/2635323028 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa5a0043070 con 0x56183fa38bb0 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5b1ee1740 10 client.15009 fetch_fsmap learned FSMap version 23 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.743+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56183fa46f80 con 0x56183fa38bb0 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5917fa640 1 -- 172.21.15.137:0/2635323028 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa59c013470 con 0x56183fa38bb0 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5917fa640 1 -- 172.21.15.137:0/2635323028 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa5a00305c0 con 0x56183fa38bb0 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5b1ee1740 10 client.15009 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5b1ee1740 10 client.15009 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5b1ee1740 20 client.15009 populate_metadata read hostname 'smithi137' 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fa495e0 0x56183fa4aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5b1ee1740 4 client.15009 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 --> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56183f9a1600 con 0x56183fa495e0 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fa495e0 0x56183fa4aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5917fa640 10 client.15009 ms_handle_connect on v2:172.21.15.189:6838/4067701297 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.744+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fa495e0 0x56183fa4aa50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2c8cfdfb06eddce server_cookie=8f6e942e8bccff9c in_seq=0 out_seq=0 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.745+0000 7fa5917fa640 1 -- 172.21.15.137:0/2635323028 <== mds.1 v2:172.21.15.189:6838/4067701297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56183f9a1600 con 0x56183fa495e0 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.745+0000 7fa5917fa640 10 client.15009 handle_command_reply: tid=0 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.745+0000 7fa5b1ee1740 1 client.15009 shutdown 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.745+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 msgr2=0x56183fa35c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.745+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa35c40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0035000 tx=0x7fa5a00095c0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.745+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 shutdown_connections 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.745+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fa495e0 0x56183fa4aa50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa35c40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 0x56183fa35700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/2635323028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5917fa640 0 client.15009 ms_handle_reset on v2:172.21.15.189:6838/4067701297 2024-09-12T12:12:41.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 >> 172.21.15.137:0/2635323028 conn(0x56183fa29f60 msgr2=0x56183fa2fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 shutdown_connections 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/2635323028 wait complete. 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.746+0000 7fa5b1ee1740 20 client.15009 trim_cache size 0 max 16384 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 Processor -- start 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 -- start start 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa45000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 0x56183fa483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa33fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56183fa48d90 con 0x56183fa38bb0 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56183fa49460 con 0x56183fa2e2f0 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56183fa49b30 con 0x56183fa2d9d0 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa33fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa33fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57432/0 (socket says 172.21.15.137:57432) 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/3706208869 learned_addr learned my addr 172.21.15.137:0/3706208869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/3706208869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 msgr2=0x56183fa45000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/3706208869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa45000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/3706208869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 msgr2=0x56183fa483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/3706208869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 0x56183fa483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.763+0000 7fa5ae0a5640 1 -- 172.21.15.137:0/3706208869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56183fa3b890 con 0x56183fa38bb0 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5ae0a5640 1 --2- 172.21.15.137:0/3706208869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa33fd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0031e80 tx=0x7fa5a002f4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1f9b238d89d783 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5937fe640 1 -- 172.21.15.137:0/3706208869 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a004e040 con 0x56183fa38bb0 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5937fe640 1 -- 172.21.15.137:0/3706208869 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0036030 con 0x56183fa38bb0 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5937fe640 1 -- 172.21.15.137:0/3706208869 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a002f610 con 0x56183fa38bb0 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/3706208869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 msgr2=0x56183fa33fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/3706208869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa33fd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0031e80 tx=0x7fa5a002f4c0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/3706208869 shutdown_connections 2024-09-12T12:12:41.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/3706208869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa38bb0 0x56183fa33fd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/3706208869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2e2f0 0x56183fa483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/3706208869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa2d9d0 0x56183fa45000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/3706208869 >> 172.21.15.137:0/3706208869 conn(0x56183fa29f60 msgr2=0x56183fa356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/3706208869 shutdown_connections 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.764+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/3706208869 wait complete. 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 -- start start 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 Processor -- start 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa52dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa53310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa4e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56183fa4a8d0 con 0x56183fa2e2f0 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56183fa4a200 con 0x56183fa2d9d0 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56183fa4afa0 con 0x56183fa38bb0 2024-09-12T12:12:41.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5b1ee1740 10 client.0 _ll_register_callbacks cb 0x7fa5a0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5ad0a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa53310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5ad0a3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa53310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57442/0 (socket says 172.21.15.137:57442) 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.765+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/1107179924 learned_addr learned my addr 172.21.15.137:0/1107179924 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa52dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/1107179924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 msgr2=0x56183fa4e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa4e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/1107179924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 msgr2=0x56183fa52dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa52dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad0a3640 1 -- 172.21.15.137:0/1107179924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56183fa4ea20 con 0x56183fa2e2f0 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5ad0a3640 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa53310 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400bd50 tx=0x7fa5a401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b83dedaaf293ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5917fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:12:41.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5917fa640 1 -- 172.21.15.137:0/1107179924 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40151c0 con 0x56183fa2e2f0 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5917fa640 1 -- 172.21.15.137:0/1107179924 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fa5a4010510 con 0x56183fa2e2f0 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a0036030 con 0x56183fa2e2f0 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5917fa640 1 -- 172.21.15.137:0/1107179924 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa5a40b4150 con 0x56183fa2e2f0 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5b1ee1740 10 client.15015 fetch_fsmap learned FSMap version 23 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56183fab3850 con 0x56183fa2e2f0 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5917fa640 1 -- 172.21.15.137:0/1107179924 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a0036030 con 0x56183fa2e2f0 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5917fa640 1 -- 172.21.15.137:0/1107179924 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa5a4095ce0 con 0x56183fa2e2f0 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5b1ee1740 10 client.15015 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5b1ee1740 10 client.15015 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5b1ee1740 20 client.15015 populate_metadata read hostname 'smithi137' 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.766+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fab8480 0x56183fab7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 4 client.15015 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 --> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] -- command(tid 0: {"prefix": "status"}) -- 0x56183f99bb10 con 0x56183fab8480 2024-09-12T12:12:41.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fab8480 0x56183fab7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5917fa640 10 client.15015 ms_handle_connect on v2:172.21.15.189:6838/4067701297 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5ad8a4640 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fab8480 0x56183fab7190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8382525533cf9e02 server_cookie=fc6bf74836c36984 in_seq=0 out_seq=0 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5917fa640 1 -- 172.21.15.137:0/1107179924 <== mds.1 v2:172.21.15.189:6838/4067701297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56183f99bb10 con 0x56183fab8480 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5917fa640 10 client.15015 handle_command_reply: tid=0 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 client.15015 shutdown 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 msgr2=0x56183fa53310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa53310 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400bd50 tx=0x7fa5a401c000 comp rx=0 tx=0).stop 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 shutdown_connections 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x56183fab8480 0x56183fab7190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56183fa38bb0 0x56183fa4e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56183fa2e2f0 0x56183fa53310 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 --2- 172.21.15.137:0/1107179924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56183fa2d9d0 0x56183fa52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5917fa640 0 client.15015 ms_handle_reset on v2:172.21.15.189:6838/4067701297 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.767+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 >> 172.21.15.137:0/1107179924 conn(0x56183fa29f60 msgr2=0x56183fa33bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 1, 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24475, 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:12:41.823 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 81.232268403999996, 2024-09-12T12:12:41.824 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-09-12T12:12:41.824 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 84, 2024-09-12T12:12:41.824 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T12:12:41.824 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 108.62657655 2024-09-12T12:12:41.824 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:12:41.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 shutdown_connections 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5b1ee1740 1 -- 172.21.15.137:0/1107179924 wait complete. 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5b1ee1740 20 client.15015 trim_cache size 0 max 16384 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5940a2c90 msgr2=0x7fa5940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5940a2c90 0x7fa5940a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0005fe0 tx=0x7fa5a002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8101290 msgr2=0x7fa5a8198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8101290 0x7fa5a8198af0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400b450 tx=0x7fa5a400b850 comp rx=0 tx=0).stop 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 shutdown_connections 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa5940a2c90 0x7fa5940a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a8108180 0x7fa5a819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5a8101290 0x7fa5a8198af0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 --2- 172.21.15.137:0/2851778957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5a81008e0 0x7fa5a81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 >> 172.21.15.137:0/2851778957 conn(0x7fa5a8077210 msgr2=0x7fa5a8105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:41.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 shutdown_connections 2024-09-12T12:12:41.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:41.768+0000 7fa5afb08640 1 -- 172.21.15.137:0/2851778957 wait complete. 2024-09-12T12:12:42.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:42 smithi137 ceph-mon[30950]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T12:12:42.941 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:42 smithi171 ceph-mon[35098]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T12:12:42.941 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:42 smithi189 ceph-mon[35475]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-12T12:12:42.942 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T12:12:43.203 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:43.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.688+0000 7fcf0bec3640 1 -- 172.21.15.137:0/2681911754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041036e0 msgr2=0x7fcf04107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.688+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/2681911754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041036e0 0x7fcf04107b90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009930 tx=0x7fcef802a420 comp rx=0 tx=0).stop 2024-09-12T12:12:43.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.688+0000 7fcf0bec3640 1 -- 172.21.15.137:0/2681911754 shutdown_connections 2024-09-12T12:12:43.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.688+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/2681911754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf041082a0 0x7fcf0410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.688+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/2681911754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041036e0 0x7fcf04107b90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.688+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/2681911754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf04102d30 0x7fcf04103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.688+0000 7fcf0bec3640 1 -- 172.21.15.137:0/2681911754 >> 172.21.15.137:0/2681911754 conn(0x7fcf040fc7d0 msgr2=0x7fcf040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 -- 172.21.15.137:0/2681911754 shutdown_connections 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 -- 172.21.15.137:0/2681911754 wait complete. 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 Processor -- start 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 -- start start 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf04102d30 0x7fcf04198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf041036e0 0x7fcf04198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041082a0 0x7fcf0419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04112140 con 0x7fcf041082a0 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf04111fc0 con 0x7fcf041036e0 2024-09-12T12:12:43.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0bec3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf041122c0 con 0x7fcf04102d30 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0a460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041082a0 0x7fcf0419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0a460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041082a0 0x7fcf0419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57466/0 (socket says 172.21.15.137:57466) 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf0a460640 1 -- 172.21.15.137:0/1386709353 learned_addr learned my addr 172.21.15.137:0/1386709353 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.689+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf04102d30 0x7fcf04198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0945e640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf041036e0 0x7fcf04198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0a460640 1 -- 172.21.15.137:0/1386709353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf04102d30 msgr2=0x7fcf04198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0a460640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf04102d30 0x7fcf04198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0a460640 1 -- 172.21.15.137:0/1386709353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf041036e0 msgr2=0x7fcf04198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0a460640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf041036e0 0x7fcf04198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0a460640 1 -- 172.21.15.137:0/1386709353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0419d6f0 con 0x7fcf041082a0 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0a460640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041082a0 0x7fcf0419d010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009fd0 tx=0x7fcefc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf02ffd640 1 -- 172.21.15.137:0/1386709353 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc011830 con 0x7fcf041082a0 2024-09-12T12:12:43.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf02ffd640 1 -- 172.21.15.137:0/1386709353 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc016070 con 0x7fcf041082a0 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0419d9e0 con 0x7fcf041082a0 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf02ffd640 1 -- 172.21.15.137:0/1386709353 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00f2e0 con 0x7fcf041082a0 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.690+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf041a52c0 con 0x7fcf041082a0 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.692+0000 7fcf02ffd640 1 -- 172.21.15.137:0/1386709353 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcefc0119d0 con 0x7fcf041082a0 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf02ffd640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fceec0a2c10 0x7fceec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf0e29c740 1 Processor -- start 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fceec0a2c10 0x7fceec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf02ffd640 1 -- 172.21.15.137:0/1386709353 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fcefc0ce600 con 0x7fcf041082a0 2024-09-12T12:12:43.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf0e29c740 1 -- start start 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x565477739db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x565477744670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.693+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x565477746fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5654776a0090 con 0x565477744bb0 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565477647a70 con 0x5654777399d0 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56547769c050 con 0x56547773a2f0 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fceec0a2c10 0x7fceec0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcef400a400 tx=0x7fcef400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x565477746fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x565477746fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57472/0 (socket says 172.21.15.137:57472) 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 -- 172.21.15.137:0/4131732207 learned_addr learned my addr 172.21.15.137:0/4131732207 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0945e640 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x565477744670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x565477739db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 -- 172.21.15.137:0/4131732207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 msgr2=0x565477744670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x565477744670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 -- 172.21.15.137:0/4131732207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 msgr2=0x565477739db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x565477739db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 -- 172.21.15.137:0/4131732207 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565477654130 con 0x565477744bb0 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x565477739db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fcf0a460640 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x565477746fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcefc01a000 tx=0x7fcefc09aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5437a8bbd3705754 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fceebfff640 1 -- 172.21.15.137:0/4131732207 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc09d020 con 0x565477744bb0 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fceebfff640 1 -- 172.21.15.137:0/4131732207 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc09ae40 con 0x565477744bb0 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.694+0000 7fceebfff640 1 -- 172.21.15.137:0/4131732207 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc09a290 con 0x565477744bb0 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 -- 172.21.15.137:0/4131732207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 msgr2=0x565477746fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x565477746fa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcefc01a000 tx=0x7fcefc09aa40 comp rx=0 tx=0).stop 2024-09-12T12:12:43.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 -- 172.21.15.137:0/4131732207 shutdown_connections 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x565477746fa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x565477744670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/4131732207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x565477739db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 -- 172.21.15.137:0/4131732207 >> 172.21.15.137:0/4131732207 conn(0x565477735f60 msgr2=0x565477736340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 -- 172.21.15.137:0/4131732207 shutdown_connections 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 -- 172.21.15.137:0/4131732207 wait complete. 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 -- start start 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.695+0000 7fcf0e29c740 1 Processor -- start 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x5654777444b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477741800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 0x565477741d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565477747780 con 0x56547773a2f0 2024-09-12T12:12:43.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56547765af20 con 0x565477744bb0 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565477747900 con 0x5654777399d0 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0e29c740 10 client.0 _ll_register_callbacks cb 0x565477624030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477741800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477741800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57476/0 (socket says 172.21.15.137:57476) 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 -- 172.21.15.137:0/111611591 learned_addr learned my addr 172.21.15.137:0/111611591 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x5654777444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 -- 172.21.15.137:0/111611591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 msgr2=0x5654777444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x5654777444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 -- 172.21.15.137:0/111611591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 msgr2=0x565477741d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 0x565477741d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 -- 172.21.15.137:0/111611591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654774112b0 con 0x56547773a2f0 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x5654777444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:43.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcf0945e640 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477741800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009900 tx=0x7fcef80319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b846060c1958adad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.696+0000 7fcee97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcee97fa640 1 -- 172.21.15.137:0/111611591 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8035040 con 0x56547773a2f0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcee97fa640 1 -- 172.21.15.137:0/111611591 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fcef8004300 con 0x56547773a2f0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc09ae40 con 0x56547773a2f0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcee97fa640 1 -- 172.21.15.137:0/111611591 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcef80035f0 con 0x56547773a2f0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 10 client.15027 fetch_fsmap learned FSMap version 23 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x565477753060 con 0x56547773a2f0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcee97fa640 1 -- 172.21.15.137:0/111611591 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc09ae40 con 0x56547773a2f0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcee97fa640 1 -- 172.21.15.137:0/111611591 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcef803c850 con 0x56547773a2f0 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 10 client.15027 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 10 client.15027 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 20 client.15027 populate_metadata read hostname 'smithi137' 2024-09-12T12:12:43.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565477755750 0x565477756bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 4 client.15027 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.697+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654776ad600 con 0x565477755750 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.698+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565477755750 0x565477756bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.698+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565477755750 0x565477756bc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c71693881a089a04 server_cookie=1134d8f9844eb7ec in_seq=0 out_seq=0 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.698+0000 7fcee97fa640 10 client.15027 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcee97fa640 1 -- 172.21.15.137:0/111611591 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654776ad600 con 0x565477755750 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcee97fa640 10 client.15027 handle_command_reply: tid=0 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 client.15027 shutdown 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 msgr2=0x565477741800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477741800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009900 tx=0x7fcef80319b0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 shutdown_connections 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 0x565477741d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565477755750 0x565477756bc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477741800 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/111611591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x5654777444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcee97fa640 0 client.15027 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 >> 172.21.15.137:0/111611591 conn(0x565477735f60 msgr2=0x56547773bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 shutdown_connections 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 1 -- 172.21.15.137:0/111611591 wait complete. 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.699+0000 7fcf0e29c740 20 client.15027 trim_cache size 0 max 16384 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.716+0000 7fcf0e29c740 1 Processor -- start 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.716+0000 7fcf0e29c740 1 -- start start 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x565477755e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477736340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 0x5654777347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565477754f00 con 0x56547773a2f0 2024-09-12T12:12:43.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5654777555d0 con 0x565477744bb0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565477755ca0 con 0x5654777399d0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477736340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477736340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57486/0 (socket says 172.21.15.137:57486) 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 -- 172.21.15.137:0/3960336607 learned_addr learned my addr 172.21.15.137:0/3960336607 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x565477755e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 -- 172.21.15.137:0/3960336607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 msgr2=0x565477755e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x565477755e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 -- 172.21.15.137:0/3960336607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 msgr2=0x5654777347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 0x5654777347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 -- 172.21.15.137:0/3960336607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654777440d0 con 0x56547773a2f0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fcf0945e640 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477736340 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcef803ce30 tx=0x7fcef8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4978c0c207c5c15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fceeb7fe640 1 -- 172.21.15.137:0/3960336607 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8035070 con 0x56547773a2f0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fceeb7fe640 1 -- 172.21.15.137:0/3960336607 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef8045430 con 0x56547773a2f0 2024-09-12T12:12:43.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.717+0000 7fceeb7fe640 1 -- 172.21.15.137:0/3960336607 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8005280 con 0x56547773a2f0 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 -- 172.21.15.137:0/3960336607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 msgr2=0x565477736340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477736340 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcef803ce30 tx=0x7fcef8005fd0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 -- 172.21.15.137:0/3960336607 shutdown_connections 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565477744bb0 0x5654777347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56547773a2f0 0x565477736340 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/3960336607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5654777399d0 0x565477755e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 -- 172.21.15.137:0/3960336607 >> 172.21.15.137:0/3960336607 conn(0x565477735f60 msgr2=0x5654777408f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 -- 172.21.15.137:0/3960336607 shutdown_connections 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 -- 172.21.15.137:0/3960336607 wait complete. 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.718+0000 7fcf0e29c740 1 -- start start 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 1 Processor -- start 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x56547773da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x56547773dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x56547773e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565477756a40 con 0x565477744bb0 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565477756370 con 0x5654777399d0 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565477757110 con 0x56547773a2f0 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0e29c740 10 client.0 _ll_register_callbacks cb 0x7fcef80314b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x56547773e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x56547773e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57500/0 (socket says 172.21.15.137:57500) 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 -- 172.21.15.137:0/1859196473 learned_addr learned my addr 172.21.15.137:0/1859196473 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0945e640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x56547773dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x56547773da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 -- 172.21.15.137:0/1859196473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 msgr2=0x56547773dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x56547773dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 -- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 msgr2=0x56547773da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x56547773da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf0a460640 1 -- 172.21.15.137:0/1859196473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56547773ec60 con 0x565477744bb0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.719+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x56547773da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0a460640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x56547773e4f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcefc09ab40 tx=0x7fcefc009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34a3204eb0e1b9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcee97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcee97fa640 1 -- 172.21.15.137:0/1859196473 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc09d070 con 0x565477744bb0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcee97fa640 1 -- 172.21.15.137:0/1859196473 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fcefc0974b0 con 0x565477744bb0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef8045430 con 0x565477744bb0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcee97fa640 1 -- 172.21.15.137:0/1859196473 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcefc0a3ab0 con 0x565477744bb0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 10 client.15033 fetch_fsmap learned FSMap version 23 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5654777c4390 con 0x565477744bb0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcee97fa640 1 -- 172.21.15.137:0/1859196473 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef8045430 con 0x565477744bb0 2024-09-12T12:12:43.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcee97fa640 1 -- 172.21.15.137:0/1859196473 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcefc09a420 con 0x565477744bb0 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 10 client.15033 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 10 client.15033 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 20 client.15033 populate_metadata read hostname 'smithi137' 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5654777c8990 0x5654777c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 4 client.15033 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.720+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "status"}) -- 0x5654776a7b10 con 0x5654777c8990 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5654777c8990 0x5654777c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf09c5f640 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5654777c8990 0x5654777c76a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d7f6c4dbb419875 server_cookie=ff4a376845d255b5 in_seq=0 out_seq=0 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcee97fa640 10 client.15033 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcee97fa640 1 -- 172.21.15.137:0/1859196473 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5654776a7b10 con 0x5654777c8990 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcee97fa640 10 client.15033 handle_command_reply: tid=0 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 client.15033 shutdown 2024-09-12T12:12:43.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 msgr2=0x56547773e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x56547773e4f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcefc09ab40 tx=0x7fcefc009330 comp rx=0 tx=0).stop 2024-09-12T12:12:43.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 shutdown_connections 2024-09-12T12:12:43.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5654777c8990 0x5654777c76a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565477744bb0 0x56547773e4f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56547773a2f0 0x56547773dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcee97fa640 0 client.15033 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:12:43.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 --2- 172.21.15.137:0/1859196473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5654777399d0 0x56547773da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 >> 172.21.15.137:0/1859196473 conn(0x565477735f60 msgr2=0x56547773b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 shutdown_connections 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.721+0000 7fcf0e29c740 1 -- 172.21.15.137:0/1859196473 wait complete. 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0e29c740 20 client.15033 trim_cache size 0 max 16384 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 0, 2024-09-12T12:12:43.741 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24476, 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 90.232048039000006, 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 84, 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 101.765973833 2024-09-12T12:12:43.742 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fceec0a2c10 msgr2=0x7fceec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fceec0a2c10 0x7fceec0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcef400a400 tx=0x7fcef400d040 comp rx=0 tx=0).stop 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041082a0 msgr2=0x7fcf0419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041082a0 0x7fcf0419d010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009fd0 tx=0x7fcefc00b8e0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 shutdown_connections 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fceec0a2c10 0x7fceec0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf041082a0 0x7fcf0419d010 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf041036e0 0x7fcf04198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 --2- 172.21.15.137:0/1386709353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf04102d30 0x7fcf04198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:43.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 >> 172.21.15.137:0/1386709353 conn(0x7fcf040fc7d0 msgr2=0x7fcf04105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:43.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 shutdown_connections 2024-09-12T12:12:43.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:43.722+0000 7fcf0bec3640 1 -- 172.21.15.137:0/1386709353 wait complete. 2024-09-12T12:12:44.475 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:44 smithi137 ceph-mon[30950]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:44.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:44 smithi189 ceph-mon[35475]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:44.652 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T12:12:44.653 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T12:12:44.661 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:12:44.662 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T12:12:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:44 smithi171 ceph-mon[35098]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:12:44.907 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:45.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/4115483026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b641026c0 msgr2=0x7f7b64108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:45.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/4115483026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b641026c0 0x7f7b64108f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009ca0 tx=0x7f7b5c02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/4115483026 shutdown_connections 2024-09-12T12:12:45.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/4115483026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b641026c0 0x7f7b64108f50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/4115483026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b64101d00 0x7f7b64102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/4115483026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b64100b00 0x7f7b64100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/4115483026 >> 172.21.15.137:0/4115483026 conn(0x7f7b640fc270 msgr2=0x7f7b640fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/4115483026 shutdown_connections 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/4115483026 wait complete. 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 Processor -- start 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.447+0000 7f7b6bbe8640 1 -- start start 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b64100b00 0x7f7b64196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b64101d00 0x7f7b64197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b641026c0 0x7f7b6419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6410c880 con 0x7f7b64100b00 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6410c700 con 0x7f7b64101d00 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6410ca00 con 0x7f7b641026c0 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b64100b00 0x7f7b64196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b64100b00 0x7f7b64196ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57518/0 (socket says 172.21.15.137:57518) 2024-09-12T12:12:45.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 -- 172.21.15.137:0/1153760865 learned_addr learned my addr 172.21.15.137:0/1153760865 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69183640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b64101d00 0x7f7b64197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6a185640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b641026c0 0x7f7b6419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 -- 172.21.15.137:0/1153760865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b641026c0 msgr2=0x7f7b6419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b641026c0 0x7f7b6419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 -- 172.21.15.137:0/1153760865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b64101d00 msgr2=0x7f7b64197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b64101d00 0x7f7b64197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 -- 172.21.15.137:0/1153760865 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6419ec20 con 0x7f7b64100b00 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69183640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b64101d00 0x7f7b64197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6a185640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b641026c0 0x7f7b6419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b69984640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b64100b00 0x7f7b64196ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000b450 tx=0x7f7b6000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b5affd640 1 -- 172.21.15.137:0/1153760865 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60015020 con 0x7f7b64100b00 2024-09-12T12:12:45.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6419ef10 con 0x7f7b64100b00 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b5affd640 1 -- 172.21.15.137:0/1153760865 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b60011d70 con 0x7f7b64100b00 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b5affd640 1 -- 172.21.15.137:0/1153760865 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000f340 con 0x7f7b64100b00 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.448+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6419f440 con 0x7f7b64100b00 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.449+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b64108040 con 0x7f7b64100b00 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.450+0000 7f7b5affd640 1 -- 172.21.15.137:0/1153760865 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7b60018020 con 0x7f7b64100b00 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.451+0000 7f7b5affd640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7b500a2bb0 0x7f7b500a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.451+0000 7f7b69183640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7b500a2bb0 0x7f7b500a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.451+0000 7f7b5affd640 1 -- 172.21.15.137:0/1153760865 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f7b600ceb30 con 0x7f7b64100b00 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.451+0000 7f7b69183640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7b500a2bb0 0x7f7b500a5070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006fd0 tx=0x7f7b54008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:45.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.455+0000 7f7b5affd640 1 -- 172.21.15.137:0/1153760865 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b6001f020 con 0x7f7b64100b00 2024-09-12T12:12:45.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.641+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7b64062630 con 0x7f7b500a2bb0 2024-09-12T12:12:45.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.642+0000 7f7b5affd640 1 -- 172.21.15.137:0/1153760865 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7b64062630 con 0x7f7b500a2bb0 2024-09-12T12:12:45.642 INFO:teuthology.orchestra.run.smithi137.stdout:Backend: cephadm 2024-09-12T12:12:45.643 INFO:teuthology.orchestra.run.smithi137.stdout:Available: Yes 2024-09-12T12:12:45.643 INFO:teuthology.orchestra.run.smithi137.stdout:Paused: No 2024-09-12T12:12:45.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7b500a2bb0 msgr2=0x7f7b500a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:45.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7b500a2bb0 0x7f7b500a5070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006fd0 tx=0x7f7b54008040 comp rx=0 tx=0).stop 2024-09-12T12:12:45.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b64100b00 msgr2=0x7f7b64196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:45.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b64100b00 0x7f7b64196ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000b450 tx=0x7f7b6000b910 comp rx=0 tx=0).stop 2024-09-12T12:12:45.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 shutdown_connections 2024-09-12T12:12:45.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f7b500a2bb0 0x7f7b500a5070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b641026c0 0x7f7b6419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b64101d00 0x7f7b64197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 --2- 172.21.15.137:0/1153760865 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b64100b00 0x7f7b64196ff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:45.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 >> 172.21.15.137:0/1153760865 conn(0x7f7b640fc270 msgr2=0x7f7b64105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:45.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 shutdown_connections 2024-09-12T12:12:45.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:45.644+0000 7f7b6bbe8640 1 -- 172.21.15.137:0/1153760865 wait complete. 2024-09-12T12:12:46.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:46 smithi137 ceph-mon[30950]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:46.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:46 smithi137 ceph-mon[30950]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T12:12:46.505 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T12:12:46.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:46 smithi189 ceph-mon[35475]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:46.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:46 smithi189 ceph-mon[35475]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T12:12:46.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:46 smithi171 ceph-mon[35098]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:12:46.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:46 smithi171 ceph-mon[35098]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T12:12:46.757 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:47.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 -- 172.21.15.137:0/1127459605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 msgr2=0x7f70d4101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:47.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1127459605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d4101910 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009900 tx=0x7f70c802a410 comp rx=0 tx=0).stop 2024-09-12T12:12:47.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 -- 172.21.15.137:0/1127459605 shutdown_connections 2024-09-12T12:12:47.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1127459605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4101e50 0x7f70d41086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1127459605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d4101910 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1127459605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4100290 0x7f70d4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 -- 172.21.15.137:0/1127459605 >> 172.21.15.137:0/1127459605 conn(0x7f70d40fba40 msgr2=0x7f70d40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 -- 172.21.15.137:0/1127459605 shutdown_connections 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.209+0000 7f70d9f23640 1 -- 172.21.15.137:0/1127459605 wait complete. 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 Processor -- start 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 -- start start 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4100290 0x7f70d4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4101e50 0x7f70d419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f70d410d910 con 0x7f70d4101490 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f70d410d790 con 0x7f70d4101e50 2024-09-12T12:12:47.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d9f23640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70d410da90 con 0x7f70d4100290 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57546/0 (socket says 172.21.15.137:57546) 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.210+0000 7f70d2ffd640 1 -- 172.21.15.137:0/1115479007 learned_addr learned my addr 172.21.15.137:0/1115479007 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d37fe640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4100290 0x7f70d4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d2ffd640 1 -- 172.21.15.137:0/1115479007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4100290 msgr2=0x7f70d4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d3fff640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4101e50 0x7f70d419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d2ffd640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4100290 0x7f70d4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d2ffd640 1 -- 172.21.15.137:0/1115479007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4101e50 msgr2=0x7f70d419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d2ffd640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4101e50 0x7f70d419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d2ffd640 1 -- 172.21.15.137:0/1115479007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d419fbd0 con 0x7f70d4101490 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d2ffd640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d41984e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f70c8031610 tx=0x7f70c8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:47.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d0ff9640 1 -- 172.21.15.137:0/1115479007 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c803f030 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d0ff9640 1 -- 172.21.15.137:0/1115479007 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c8003ce0 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d419fe60 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d0ff9640 1 -- 172.21.15.137:0/1115479007 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8035070 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.211+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d41a0330 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.212+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d41076e0 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.213+0000 7f70d0ff9640 1 -- 172.21.15.137:0/1115479007 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f70c8042030 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.214+0000 7f70d0ff9640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f70bc0a2bb0 0x7f70bc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.214+0000 7f70d37fe640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f70bc0a2bb0 0x7f70bc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.214+0000 7f70d0ff9640 1 -- 172.21.15.137:0/1115479007 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f70c80ef5c0 con 0x7f70d4101490 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.214+0000 7f70d37fe640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f70bc0a2bb0 0x7f70bc0a5070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f70d41012f0 tx=0x7f70c400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:47.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.218+0000 7f70d0ff9640 1 -- 172.21.15.137:0/1115479007 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70c80bc550 con 0x7f70d4101490 2024-09-12T12:12:47.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.395+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f70d4062630 con 0x7f70bc0a2bb0 2024-09-12T12:12:47.396 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:47 smithi137 ceph-mon[30950]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:47.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.409+0000 7f70d0ff9640 1 -- 172.21.15.137:0/1115479007 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f70d4062630 con 0x7f70bc0a2bb0 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.a smithi137 running (108s) 29s ago 108s 14.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 4db1b9d79bd2 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.b smithi171 running (105s) 38s ago 105s 26.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 be8aa567e0a4 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.c smithi189 running (119s) 38s ago 119s 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 794fd1523170 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.d smithi137 running (2m) 29s ago 2m 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 4d9bdea95fb4 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.e smithi171 running (2m) 38s ago 2m 18.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 204a2a4d6585 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.f smithi189 running (2m) 38s ago 2m 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 621c43d9b50d 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.g smithi137 running (112s) 29s ago 111s 15.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 04bed9820499 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.h smithi171 running (117s) 38s ago 116s 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 143f05389199 2024-09-12T12:12:47.410 INFO:teuthology.orchestra.run.smithi137.stdout:mds.i smithi189 running (114s) 38s ago 114s 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 29cdd59e7968 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mds.j smithi137 running (2m) 29s ago 2m 14.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 19c51bf267dd 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mds.k smithi171 running (2m) 38s ago 2m 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 dd5bf0673ac5 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mds.l smithi189 running (2m) 38s ago 2m 18.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 f5d5c33814cc 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.x smithi137 *:9283,8765 running (9m) 29s ago 9m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 9413d1f0ffbb 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.y smithi171 *:8443,8765 running (7m) 38s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 ac545c0fa269 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.z smithi189 *:8443,8765 running (7m) 38s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 39d267a0e552 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mon.a smithi137 running (9m) 29s ago 9m 51.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 f9bc5e1ab701 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mon.b smithi171 running (7m) 38s ago 7m 42.4M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 03ba81652799 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:mon.c smithi189 running (8m) 38s ago 8m 40.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 8560af655c48 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (7m) 29s ago 7m 59.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 cd82463bcfe4 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi171 running (6m) 38s ago 6m 54.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 a822202aefa1 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi189 running (6m) 38s ago 6m 57.1M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 aeabff18f548 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi137 running (6m) 29s ago 6m 44.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 13dc2f41d572 2024-09-12T12:12:47.411 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi171 running (5m) 38s ago 5m 43.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 406d3841e6f1 2024-09-12T12:12:47.412 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi189 running (5m) 38s ago 5m 43.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 2dffe6a129d2 2024-09-12T12:12:47.412 INFO:teuthology.orchestra.run.smithi137.stdout:osd.6 smithi137 running (4m) 29s ago 4m 44.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 ad849c208ae7 2024-09-12T12:12:47.412 INFO:teuthology.orchestra.run.smithi137.stdout:osd.7 smithi171 running (4m) 38s ago 4m 47.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b9c594d0ab73 2024-09-12T12:12:47.412 INFO:teuthology.orchestra.run.smithi137.stdout:osd.8 smithi189 running (3m) 38s ago 3m 42.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 e95f6021b113 2024-09-12T12:12:47.412 INFO:teuthology.orchestra.run.smithi137.stdout:osd.9 smithi137 running (3m) 29s ago 3m 53.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 4bec165e789a 2024-09-12T12:12:47.412 INFO:teuthology.orchestra.run.smithi137.stdout:osd.10 smithi171 running (3m) 38s ago 3m 43.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d48240775ec8 2024-09-12T12:12:47.412 INFO:teuthology.orchestra.run.smithi137.stdout:osd.11 smithi189 running (2m) 38s ago 2m 42.8M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 da7e9f24536c 2024-09-12T12:12:47.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f70bc0a2bb0 msgr2=0x7f70bc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:47.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f70bc0a2bb0 0x7f70bc0a5070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f70d41012f0 tx=0x7f70c400a2d0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 msgr2=0x7f70d41984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:47.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d41984e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f70c8031610 tx=0x7f70c8004430 comp rx=0 tx=0).stop 2024-09-12T12:12:47.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 shutdown_connections 2024-09-12T12:12:47.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f70bc0a2bb0 0x7f70bc0a5070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f70d4101e50 0x7f70d419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70d4101490 0x7f70d41984e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 --2- 172.21.15.137:0/1115479007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70d4100290 0x7f70d4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:47.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.411+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 >> 172.21.15.137:0/1115479007 conn(0x7f70d40fba40 msgr2=0x7f70d41048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:47.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.412+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 shutdown_connections 2024-09-12T12:12:47.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:47.412+0000 7f70d9f23640 1 -- 172.21.15.137:0/1115479007 wait complete. 2024-09-12T12:12:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:47 smithi189 ceph-mon[35475]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:47 smithi171 ceph-mon[35098]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:48.140 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T12:12:48.387 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:48.508 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:48 smithi137 ceph-mon[30950]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T12:12:48.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:48 smithi189 ceph-mon[35475]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T12:12:48.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:48 smithi171 ceph-mon[35098]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-12T12:12:48.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 -- 172.21.15.137:0/2790535577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc107960 msgr2=0x7f16dc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:48.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 --2- 172.21.15.137:0/2790535577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc107960 0x7f16dc109d50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009ca0 tx=0x7f16d402a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:48.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 -- 172.21.15.137:0/2790535577 shutdown_connections 2024-09-12T12:12:48.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 --2- 172.21.15.137:0/2790535577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc107960 0x7f16dc109d50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:48.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 --2- 172.21.15.137:0/2790535577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16dc105030 0x7f16dc107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 --2- 172.21.15.137:0/2790535577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16dc102700 0x7f16dc104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 -- 172.21.15.137:0/2790535577 >> 172.21.15.137:0/2790535577 conn(0x7f16dc0fc280 msgr2=0x7f16dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 -- 172.21.15.137:0/2790535577 shutdown_connections 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.884+0000 7f16e08ab640 1 -- 172.21.15.137:0/2790535577 wait complete. 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 Processor -- start 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 -- start start 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc102700 0x7f16dc102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16dc105030 0x7f16dc1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16dc107960 0x7f16dc100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc10d880 con 0x7f16dc102700 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc10d700 con 0x7f16dc107960 2024-09-12T12:12:48.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16e08ab640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc10da00 con 0x7f16dc105030 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16da59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc102700 0x7f16dc102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16da59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc102700 0x7f16dc102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57558/0 (socket says 172.21.15.137:57558) 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16da59c640 1 -- 172.21.15.137:0/3053094942 learned_addr learned my addr 172.21.15.137:0/3053094942 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16dad9d640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16dc107960 0x7f16dc100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.885+0000 7f16d9d9b640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16dc105030 0x7f16dc1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16da59c640 1 -- 172.21.15.137:0/3053094942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16dc105030 msgr2=0x7f16dc1006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16da59c640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16dc105030 0x7f16dc1006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16da59c640 1 -- 172.21.15.137:0/3053094942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16dc107960 msgr2=0x7f16dc100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16da59c640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16dc107960 0x7f16dc100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16da59c640 1 -- 172.21.15.137:0/3053094942 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16dc1014b0 con 0x7f16dc102700 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16dad9d640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16dc107960 0x7f16dc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:12:48.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16d9d9b640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16dc105030 0x7f16dc1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16da59c640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc102700 0x7f16dc102060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00e9f0 tx=0x7f16cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16cb7fe640 1 -- 172.21.15.137:0/3053094942 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc00cd80 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16cb7fe640 1 -- 172.21.15.137:0/3053094942 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16cc005500 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16dc1a2090 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16cb7fe640 1 -- 172.21.15.137:0/3053094942 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc010690 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.886+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16dc1a2550 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.887+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16dc0fdd70 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.888+0000 7f16cb7fe640 1 -- 172.21.15.137:0/3053094942 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f16cc0046c0 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.889+0000 7f16cb7fe640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f16c40a2b80 0x7f16c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.889+0000 7f16d9d9b640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f16c40a2b80 0x7f16c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.889+0000 7f16cb7fe640 1 -- 172.21.15.137:0/3053094942 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f16cc0cf540 con 0x7f16dc102700 2024-09-12T12:12:48.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.889+0000 7f16d9d9b640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f16c40a2b80 0x7f16c40a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16dc101de0 tx=0x7f16d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:48.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:48.893+0000 7f16cb7fe640 1 -- 172.21.15.137:0/3053094942 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16cc09c4d0 con 0x7f16dc102700 2024-09-12T12:12:49.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.068+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f16dc062630 con 0x7f16c40a2b80 2024-09-12T12:12:49.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.075+0000 7f16cb7fe640 1 -- 172.21.15.137:0/3053094942 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f16dc062630 con 0x7f16c40a2b80 2024-09-12T12:12:49.076 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T12:12:49.076 INFO:teuthology.orchestra.run.smithi137.stdout:mds.all 12/12 40s ago 2m smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi171=b;smithi171=e;smithi171=h;smithi171=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-09-12T12:12:49.076 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 3/3 40s ago 7m smithi137=x;smithi171=y;smithi189=z;count:3 2024-09-12T12:12:49.076 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 40s ago 8m smithi137:172.21.15.137=a;smithi171:172.21.15.171=b;smithi189:172.21.15.189=c;count:3 2024-09-12T12:12:49.076 INFO:teuthology.orchestra.run.smithi137.stdout:osd 12 40s ago - 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f16c40a2b80 msgr2=0x7f16c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f16c40a2b80 0x7f16c40a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f16dc101de0 tx=0x7f16d0005f90 comp rx=0 tx=0).stop 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc102700 msgr2=0x7f16dc102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc102700 0x7f16dc102060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00e9f0 tx=0x7f16cc00edf0 comp rx=0 tx=0).stop 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 shutdown_connections 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f16c40a2b80 0x7f16c40a5040 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16dc107960 0x7f16dc100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16dc105030 0x7f16dc1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:49.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 --2- 172.21.15.137:0/3053094942 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16dc102700 0x7f16dc102060 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:49.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 >> 172.21.15.137:0/3053094942 conn(0x7f16dc0fc280 msgr2=0x7f16dc103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:49.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 shutdown_connections 2024-09-12T12:12:49.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:49.078+0000 7f16e08ab640 1 -- 172.21.15.137:0/3053094942 wait complete. 2024-09-12T12:12:49.369 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:49 smithi137 ceph-mon[30950]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:49 smithi189 ceph-mon[35475]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:49 smithi171 ceph-mon[35098]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:49.840 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T12:12:50.095 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:50.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:50 smithi137 ceph-mon[30950]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:50.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:50 smithi137 ceph-mon[30950]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:50.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:50 smithi189 ceph-mon[35475]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:50.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:50 smithi189 ceph-mon[35475]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:50.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 -- 172.21.15.137:0/4232026034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c0102e00 msgr2=0x7f37c0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/4232026034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c0102e00 0x7f37c0103280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009900 tx=0x7f37b002a410 comp rx=0 tx=0).stop 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 -- 172.21.15.137:0/4232026034 shutdown_connections 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/4232026034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37c01037c0 0x7f37c010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/4232026034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c0102e00 0x7f37c0103280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/4232026034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37c0101c00 0x7f37c0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 -- 172.21.15.137:0/4232026034 >> 172.21.15.137:0/4232026034 conn(0x7f37c00fd390 msgr2=0x7f37c00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 -- 172.21.15.137:0/4232026034 shutdown_connections 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.615+0000 7f37c7dd8640 1 -- 172.21.15.137:0/4232026034 wait complete. 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 Processor -- start 2024-09-12T12:12:50.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 -- start start 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37c0101c00 0x7f37c0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37c0102e00 0x7f37c01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c01037c0 0x7f37c019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010d900 con 0x7f37c01037c0 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010d780 con 0x7f37c0101c00 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c7dd8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37c010da80 con 0x7f37c0102e00 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c6375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c01037c0 0x7f37c019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c6375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c01037c0 0x7f37c019f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38730/0 (socket says 172.21.15.137:38730) 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c6375640 1 -- 172.21.15.137:0/3584367937 learned_addr learned my addr 172.21.15.137:0/3584367937 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c5373640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37c0102e00 0x7f37c01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.616+0000 7f37c5b74640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37c0101c00 0x7f37c0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:50.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c6375640 1 -- 172.21.15.137:0/3584367937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37c0102e00 msgr2=0x7f37c01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c6375640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37c0102e00 0x7f37c01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c6375640 1 -- 172.21.15.137:0/3584367937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37c0101c00 msgr2=0x7f37c0197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c6375640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37c0101c00 0x7f37c0197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c6375640 1 -- 172.21.15.137:0/3584367937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c019fba0 con 0x7f37c01037c0 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c6375640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c01037c0 0x7f37c019f530 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37b800eaa0 tx=0x7f37b800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37b6ffd640 1 -- 172.21.15.137:0/3584367937 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8004d20 con 0x7f37c01037c0 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c019fe90 con 0x7f37c01037c0 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37b6ffd640 1 -- 172.21.15.137:0/3584367937 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b8005020 con 0x7f37c01037c0 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37b6ffd640 1 -- 172.21.15.137:0/3584367937 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80116d0 con 0x7f37c01037c0 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.617+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c01a03c0 con 0x7f37c01037c0 2024-09-12T12:12:50.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.618+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c0109000 con 0x7f37c01037c0 2024-09-12T12:12:50.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.619+0000 7f37b6ffd640 1 -- 172.21.15.137:0/3584367937 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f37b80051d0 con 0x7f37c01037c0 2024-09-12T12:12:50.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.620+0000 7f37b6ffd640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37ac0a2bb0 0x7f37ac0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:50.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.620+0000 7f37c5b74640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37ac0a2bb0 0x7f37ac0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:50.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.620+0000 7f37b6ffd640 1 -- 172.21.15.137:0/3584367937 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f37b80cf560 con 0x7f37c01037c0 2024-09-12T12:12:50.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.620+0000 7f37c5b74640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37ac0a2bb0 0x7f37ac0a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37c0102c60 tx=0x7f37bc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:50.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.623+0000 7f37b6ffd640 1 -- 172.21.15.137:0/3584367937 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37b8016030 con 0x7f37c01037c0 2024-09-12T12:12:50.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:50 smithi171 ceph-mon[35098]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:50.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:50 smithi171 ceph-mon[35098]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.794+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f37c0062630 con 0x7f37ac0a2bb0 2024-09-12T12:12:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.795+0000 7f37b6ffd640 1 -- 172.21.15.137:0/3584367937 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f37c0062630 con 0x7f37ac0a2bb0 2024-09-12T12:12:50.794 INFO:teuthology.orchestra.run.smithi137.stdout:HOST ADDR LABELS STATUS 2024-09-12T12:12:50.795 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137 172.21.15.137 2024-09-12T12:12:50.795 INFO:teuthology.orchestra.run.smithi137.stdout:smithi171 172.21.15.171 2024-09-12T12:12:50.795 INFO:teuthology.orchestra.run.smithi137.stdout:smithi189 172.21.15.189 2024-09-12T12:12:50.795 INFO:teuthology.orchestra.run.smithi137.stdout:3 hosts in cluster 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37ac0a2bb0 msgr2=0x7f37ac0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37ac0a2bb0 0x7f37ac0a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37c0102c60 tx=0x7f37bc009110 comp rx=0 tx=0).stop 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c01037c0 msgr2=0x7f37c019f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c01037c0 0x7f37c019f530 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37b800eaa0 tx=0x7f37b800eea0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 shutdown_connections 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f37ac0a2bb0 0x7f37ac0a5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37c01037c0 0x7f37c019f530 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37c0102e00 0x7f37c01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 --2- 172.21.15.137:0/3584367937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37c0101c00 0x7f37c0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 >> 172.21.15.137:0/3584367937 conn(0x7f37c00fd390 msgr2=0x7f37c01061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 shutdown_connections 2024-09-12T12:12:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:50.798+0000 7f37c7dd8640 1 -- 172.21.15.137:0/3584367937 wait complete. 2024-09-12T12:12:51.513 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T12:12:51.769 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:52.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1946321272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4100ad0 msgr2=0x7ffad4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:52.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1946321272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4100ad0 0x7ffad4100ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009900 tx=0x7ffac402a410 comp rx=0 tx=0).stop 2024-09-12T12:12:52.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1946321272 shutdown_connections 2024-09-12T12:12:52.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1946321272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad4102690 0x7ffad4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1946321272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4101cd0 0x7ffad4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1946321272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4100ad0 0x7ffad4100ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1946321272 >> 172.21.15.137:0/1946321272 conn(0x7ffad40fc280 msgr2=0x7ffad40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1946321272 shutdown_connections 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1946321272 wait complete. 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.290+0000 7ffad8fe2640 1 Processor -- start 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 -- start start 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100ad0 0x7ffad419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4101cd0 0x7ffad419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad4102690 0x7ffad41a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410c7b0 con 0x7ffad4101cd0 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410c630 con 0x7ffad4100ad0 2024-09-12T12:12:52.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffad410c930 con 0x7ffad4102690 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4101cd0 0x7ffad419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4101cd0 0x7ffad419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38752/0 (socket says 172.21.15.137:38752) 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 -- 172.21.15.137:0/1873264619 learned_addr learned my addr 172.21.15.137:0/1873264619 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 -- 172.21.15.137:0/1873264619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad4102690 msgr2=0x7ffad41a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad4102690 0x7ffad41a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 -- 172.21.15.137:0/1873264619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100ad0 msgr2=0x7ffad419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad259c640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100ad0 0x7ffad419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100ad0 0x7ffad419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 -- 172.21.15.137:0/1873264619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad41a1de0 con 0x7ffad4101cd0 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad1d9b640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4101cd0 0x7ffad419a6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffac800b4a0 tx=0x7ffac800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffac37fe640 1 -- 172.21.15.137:0/1873264619 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8014070 con 0x7ffad4101cd0 2024-09-12T12:12:52.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffac37fe640 1 -- 172.21.15.137:0/1873264619 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac8010af0 con 0x7ffad4101cd0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad41a20d0 con 0x7ffad4101cd0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffac37fe640 1 -- 172.21.15.137:0/1873264619 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8017d10 con 0x7ffad4101cd0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.291+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad41a2690 con 0x7ffad4101cd0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.292+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad41a2950 con 0x7ffad4101cd0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.294+0000 7ffac37fe640 1 -- 172.21.15.137:0/1873264619 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffac8017070 con 0x7ffad4101cd0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.294+0000 7ffac37fe640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ffabc0a2bb0 0x7ffabc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.295+0000 7ffad259c640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ffabc0a2bb0 0x7ffabc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.295+0000 7ffac37fe640 1 -- 172.21.15.137:0/1873264619 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7ffac80ce470 con 0x7ffad4101cd0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.295+0000 7ffad259c640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ffabc0a2bb0 0x7ffabc0a5070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffac4031610 tx=0x7ffac4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:52.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.299+0000 7ffac37fe640 1 -- 172.21.15.137:0/1873264619 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffac809b400 con 0x7ffad4101cd0 2024-09-12T12:12:52.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.469+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 --> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ffad41a2b20 con 0x7ffabc0a2bb0 2024-09-12T12:12:52.469 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:52 smithi137 ceph-mon[30950]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:52.469 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:52 smithi137 ceph-mon[30950]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:52.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.471+0000 7ffac37fe640 1 -- 172.21.15.137:0/1873264619 <== mgr.14150 v2:172.21.15.137:6800/1789953576 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ffad41a2b20 con 0x7ffabc0a2bb0 2024-09-12T12:12:52.471 INFO:teuthology.orchestra.run.smithi137.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T12:12:52.471 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T12:12:52.471 INFO:teuthology.orchestra.run.smithi137.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T12:12:52.471 INFO:teuthology.orchestra.run.smithi137.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ffabc0a2bb0 msgr2=0x7ffabc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ffabc0a2bb0 0x7ffabc0a5070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffac4031610 tx=0x7ffac4000bc0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4101cd0 msgr2=0x7ffad419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4101cd0 0x7ffad419a6f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffac800b4a0 tx=0x7ffac800b960 comp rx=0 tx=0).stop 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 shutdown_connections 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ffabc0a2bb0 0x7ffabc0a5070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffad4102690 0x7ffad41a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffad4101cd0 0x7ffad419a6f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 --2- 172.21.15.137:0/1873264619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffad4100ad0 0x7ffad419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:52.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 >> 172.21.15.137:0/1873264619 conn(0x7ffad40fc280 msgr2=0x7ffad4069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 shutdown_connections 2024-09-12T12:12:52.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:52.473+0000 7ffad8fe2640 1 -- 172.21.15.137:0/1873264619 wait complete. 2024-09-12T12:12:52.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:52 smithi189 ceph-mon[35475]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:52.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:52 smithi189 ceph-mon[35475]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:52.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:52 smithi171 ceph-mon[35098]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:52.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:52 smithi171 ceph-mon[35098]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:53.334 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T12:12:53.343 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:12:53.344 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T12:12:53.592 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 -- 172.21.15.137:0/59572304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024102e00 msgr2=0x7ff024103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 --2- 172.21.15.137:0/59572304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024102e00 0x7ff024103280 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff014009900 tx=0x7ff01402a410 comp rx=0 tx=0).stop 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 -- 172.21.15.137:0/59572304 shutdown_connections 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 --2- 172.21.15.137:0/59572304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0241037c0 0x7ff02410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 --2- 172.21.15.137:0/59572304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024102e00 0x7ff024103280 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 --2- 172.21.15.137:0/59572304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff024101c00 0x7ff024102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 -- 172.21.15.137:0/59572304 >> 172.21.15.137:0/59572304 conn(0x7ff0240fd390 msgr2=0x7ff0240ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 -- 172.21.15.137:0/59572304 shutdown_connections 2024-09-12T12:12:54.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.072+0000 7ff028af9640 1 -- 172.21.15.137:0/59572304 wait complete. 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 Processor -- start 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 -- start start 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024101c00 0x7ff024197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024102e00 0x7ff0241984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0241037c0 0x7ff02419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff02410d930 con 0x7ff024101c00 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff02410d7b0 con 0x7ff0241037c0 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff028af9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff02410dab0 con 0x7ff024102e00 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024101c00 0x7ff024197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024101c00 0x7ff024197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38766/0 (socket says 172.21.15.137:38766) 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 -- 172.21.15.137:0/3058421594 learned_addr learned my addr 172.21.15.137:0/3058421594 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff021d9b640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024102e00 0x7ff0241984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 -- 172.21.15.137:0/3058421594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024102e00 msgr2=0x7ff0241984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:54.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024102e00 0x7ff0241984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff022d9d640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0241037c0 0x7ff02419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 -- 172.21.15.137:0/3058421594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0241037c0 msgr2=0x7ff02419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0241037c0 0x7ff02419f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.073+0000 7ff02259c640 1 -- 172.21.15.137:0/3058421594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02419fbb0 con 0x7ff024101c00 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.074+0000 7ff02259c640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024101c00 0x7ff024197f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00b450 tx=0x7ff00c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.074+0000 7ff01b7fe640 1 -- 172.21.15.137:0/3058421594 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c015020 con 0x7ff024101c00 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.074+0000 7ff01b7fe640 1 -- 172.21.15.137:0/3058421594 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff00c011d70 con 0x7ff024101c00 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.074+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02419fea0 con 0x7ff024101c00 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.074+0000 7ff01b7fe640 1 -- 172.21.15.137:0/3058421594 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00f340 con 0x7ff024101c00 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.074+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0241a0370 con 0x7ff024101c00 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.075+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff024108fa0 con 0x7ff024101c00 2024-09-12T12:12:54.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.076+0000 7ff01b7fe640 1 -- 172.21.15.137:0/3058421594 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ff00c018020 con 0x7ff024101c00 2024-09-12T12:12:54.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.077+0000 7ff01b7fe640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0100a2bb0 0x7ff0100a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:54.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.077+0000 7ff021d9b640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0100a2bb0 0x7ff0100a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:54.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.077+0000 7ff01b7fe640 1 -- 172.21.15.137:0/3058421594 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7ff00c0ceae0 con 0x7ff024101c00 2024-09-12T12:12:54.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.077+0000 7ff021d9b640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0100a2bb0 0x7ff0100a5070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff014000c00 tx=0x7ff0140315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:54.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.081+0000 7ff01b7fe640 1 -- 172.21.15.137:0/3058421594 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff00c01f020 con 0x7ff024101c00 2024-09-12T12:12:54.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.284+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff02402d7c0 con 0x7ff024101c00 2024-09-12T12:12:54.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:54 smithi137 ceph-mon[30950]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:54.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:54 smithi137 ceph-mon[30950]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:54.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.285+0000 7ff01b7fe640 1 -- 172.21.15.137:0/3058421594 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3103 (secure 0 0 0) 0x7ff00c00d040 con 0x7ff024101c00 2024-09-12T12:12:54.285 INFO:teuthology.orchestra.run.smithi137.stdout:e23 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout:btime 2024-09-12T12:11:24:550801+0000 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-09-12T12:12:54.286 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 23 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-09-12T12:11:13.196665+0000 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-09-12T12:11:24.095061+0000 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 300 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-09-12T12:12:54.287 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 0 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 3 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1,2 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=24476,1=24475,2=24454} 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-09-12T12:12:54.288 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3] 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 0 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:qdb_cluster leader: 24476 members: 24476,24475,24454 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b{0:24476} state up:active seq 4 addr [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.f{0:24448} state up:standby-replay seq 1 addr [v2:172.21.15.189:6834/2972240277,v1:172.21.15.189:6835/2972240277] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.i{1:24475} state up:active seq 8 addr [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.289 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.h{1:24446} state up:standby-replay seq 1 addr [v2:172.21.15.171:6836/1673181827,v1:172.21.15.171:6837/1673181827] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c{2:24454} state up:active seq 10 addr [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.l{2:24427} state up:standby-replay seq 1 addr [v2:172.21.15.189:6832/4107944214,v1:172.21.15.189:6833/4107944214] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.137:6834/2258495281,v1:172.21.15.137:6835/2258495281] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.137:6836/1845807788,v1:172.21.15.137:6837/1845807788] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.g{-1:14598} state up:standby seq 1 addr [v2:172.21.15.137:6838/3700003405,v1:172.21.15.137:6839/3700003405] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a{-1:14610} state up:standby seq 1 addr [v2:172.21.15.137:6840/2375268827,v1:172.21.15.137:6841/2375268827] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.171:6832/1998961324,v1:172.21.15.171:6833/1998961324] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.290 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.e{-1:24422} state up:standby seq 1 addr [v2:172.21.15.171:6834/971785585,v1:172.21.15.171:6835/971785585] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T12:12:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-09-12T12:12:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0100a2bb0 msgr2=0x7ff0100a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0100a2bb0 0x7ff0100a5070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff014000c00 tx=0x7ff0140315d0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024101c00 msgr2=0x7ff024197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024101c00 0x7ff024197f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00b450 tx=0x7ff00c00b910 comp rx=0 tx=0).stop 2024-09-12T12:12:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 shutdown_connections 2024-09-12T12:12:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff0100a2bb0 0x7ff0100a5070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0241037c0 0x7ff02419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024102e00 0x7ff0241984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 --2- 172.21.15.137:0/3058421594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff024101c00 0x7ff024197f80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 >> 172.21.15.137:0/3058421594 conn(0x7ff0240fd390 msgr2=0x7ff024106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 shutdown_connections 2024-09-12T12:12:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:54.287+0000 7ff028af9640 1 -- 172.21.15.137:0/3058421594 wait complete. 2024-09-12T12:12:54.502 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:54 smithi171 ceph-mon[35098]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:54.502 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:54 smithi171 ceph-mon[35098]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:54.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:54 smithi189 ceph-mon[35475]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:12:54.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:54 smithi189 ceph-mon[35475]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:55.095 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T12:12:55.346 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:55.467 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:55 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3058421594' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T12:12:55.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:55 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3058421594' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T12:12:55.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:55 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3058421594' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 -- 172.21.15.137:0/3030850010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 msgr2=0x7f435810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 --2- 172.21.15.137:0/3030850010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f435810a050 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4350009ca0 tx=0x7f435002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 -- 172.21.15.137:0/3030850010 shutdown_connections 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 --2- 172.21.15.137:0/3030850010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f435810a050 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 --2- 172.21.15.137:0/3030850010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4358102e00 0x7f4358103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 --2- 172.21.15.137:0/3030850010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4358101c00 0x7f4358102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 -- 172.21.15.137:0/3030850010 >> 172.21.15.137:0/3030850010 conn(0x7f43580fd390 msgr2=0x7f43580ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:55.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 -- 172.21.15.137:0/3030850010 shutdown_connections 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 -- 172.21.15.137:0/3030850010 wait complete. 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.891+0000 7f435e071640 1 Processor -- start 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f435e071640 1 -- start start 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f435e071640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4358101c00 0x7f435819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f435e071640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4358102e00 0x7f435819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f435e071640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f43581a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f435e071640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f435810d880 con 0x7f43581037c0 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f435e071640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f435810d700 con 0x7f4358101c00 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f435e071640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f435810da00 con 0x7f4358102e00 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f43581a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f43581a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38784/0 (socket says 172.21.15.137:38784) 2024-09-12T12:12:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 -- 172.21.15.137:0/3859538680 learned_addr learned my addr 172.21.15.137:0/3859538680 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f43577fe640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4358101c00 0x7f435819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 -- 172.21.15.137:0/3859538680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4358102e00 msgr2=0x7f435819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4356ffd640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4358102e00 0x7f435819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4358102e00 0x7f435819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 -- 172.21.15.137:0/3859538680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4358101c00 msgr2=0x7f435819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4358101c00 0x7f435819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 -- 172.21.15.137:0/3859538680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43581a1d20 con 0x7f43581037c0 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f43577fe640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4358101c00 0x7f435819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4356ffd640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4358102e00 0x7f435819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.892+0000 7f4357fff640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f43581a16b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f43500098d0 tx=0x7f435002f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.893+0000 7f4354ff9640 1 -- 172.21.15.137:0/3859538680 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435002fc30 con 0x7f43581037c0 2024-09-12T12:12:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.893+0000 7f4354ff9640 1 -- 172.21.15.137:0/3859538680 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350003e50 con 0x7f43581037c0 2024-09-12T12:12:55.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.893+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43581a1fb0 con 0x7f43581037c0 2024-09-12T12:12:55.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.893+0000 7f4354ff9640 1 -- 172.21.15.137:0/3859538680 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500305c0 con 0x7f43581037c0 2024-09-12T12:12:55.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.893+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43581a24b0 con 0x7f43581037c0 2024-09-12T12:12:55.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.894+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43581a2710 con 0x7f43581037c0 2024-09-12T12:12:55.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.895+0000 7f4354ff9640 1 -- 172.21.15.137:0/3859538680 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4350004b40 con 0x7f43581037c0 2024-09-12T12:12:55.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.896+0000 7f4354ff9640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f43440a2c30 0x7f43440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:55.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.896+0000 7f43577fe640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f43440a2c30 0x7f43440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:55.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.896+0000 7f4354ff9640 1 -- 172.21.15.137:0/3859538680 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f43500f0530 con 0x7f43581037c0 2024-09-12T12:12:55.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.896+0000 7f43577fe640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f43440a2c30 0x7f43440a50f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4358102c60 tx=0x7f4348005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:55.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:55.899+0000 7f4354ff9640 1 -- 172.21.15.137:0/3859538680 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43500433d0 con 0x7f43581037c0 2024-09-12T12:12:56.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.065+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f43581a28e0 con 0x7f43581037c0 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.066+0000 7f4354ff9640 1 -- 172.21.15.137:0/3859538680 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5282 (secure 0 0 0) 0x7f43500bd4c0 con 0x7f43581037c0 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 84 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stdout:fsid dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-09-12T12:03:22.696612+0000 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-09-12T12:12:17.022250+0000 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stdout:crush_version 26 2024-09-12T12:12:56.066 INFO:teuthology.orchestra.run.smithi137.stdout:full_ratio 0.95 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:backfillfull_ratio 0.9 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:nearfull_ratio 0.85 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:require_min_compat_client luminous 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:min_compat_client luminous 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:require_osd_release squid 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:stretch_mode_enabled false 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:max_osd 12 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6802/655478089,v1:172.21.15.137:6803/655478089] [v2:172.21.15.137:6804/655478089,v1:172.21.15.137:6805/655478089] exists,up 45d33c8c-abf3-4322-aeb0-28f4141abb52 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1953204063,v1:172.21.15.171:6801/1953204063] [v2:172.21.15.171:6802/1953204063,v1:172.21.15.171:6803/1953204063] exists,up a1a7f639-1cce-4a38-b5b2-9e6035281396 2024-09-12T12:12:56.067 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/2022208006,v1:172.21.15.189:6801/2022208006] [v2:172.21.15.189:6802/2022208006,v1:172.21.15.189:6803/2022208006] exists,up 845d927b-63b2-4eba-961c-222c975d8779 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6810/1732422087,v1:172.21.15.137:6811/1732422087] [v2:172.21.15.137:6812/1732422087,v1:172.21.15.137:6813/1732422087] exists,up 825bb9bf-6e25-4933-b0fc-4be4950fac0a 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 up in weight 1 up_from 32 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/1838766520,v1:172.21.15.171:6809/1838766520] [v2:172.21.15.171:6810/1838766520,v1:172.21.15.171:6811/1838766520] exists,up 8d266b7f-3149-4edd-b3d3-15c3ba62ce7f 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 up in weight 1 up_from 37 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/3684989511,v1:172.21.15.189:6809/3684989511] [v2:172.21.15.189:6810/3684989511,v1:172.21.15.189:6811/3684989511] exists,up 80eab3c2-6d42-4a43-a151-b8e92c20dda1 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.6 up in weight 1 up_from 44 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6818/2955523379,v1:172.21.15.137:6819/2955523379] [v2:172.21.15.137:6820/2955523379,v1:172.21.15.137:6821/2955523379] exists,up 4971246b-c619-43a1-a798-e0007f23cab0 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.7 up in weight 1 up_from 49 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/665345582,v1:172.21.15.171:6817/665345582] [v2:172.21.15.171:6818/665345582,v1:172.21.15.171:6819/665345582] exists,up e16ef0a1-dcb9-45df-bde2-22fe86d53f55 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.8 up in weight 1 up_from 56 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/1497312216,v1:172.21.15.189:6817/1497312216] [v2:172.21.15.189:6818/1497312216,v1:172.21.15.189:6819/1497312216] exists,up 88d4c0f4-ec90-4c43-a8da-53e18fcd1984 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.9 up in weight 1 up_from 63 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6826/4232440350,v1:172.21.15.137:6827/4232440350] [v2:172.21.15.137:6828/4232440350,v1:172.21.15.137:6829/4232440350] exists,up e6098dac-95f6-45c0-8e0c-2299eb98ca05 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.10 up in weight 1 up_from 68 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/124645600,v1:172.21.15.171:6825/124645600] [v2:172.21.15.171:6826/124645600,v1:172.21.15.171:6827/124645600] exists,up 579cd8cf-bf21-4438-9476-72080c6e193a 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:osd.11 up in weight 1 up_from 75 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/2395385355,v1:172.21.15.189:6825/2395385355] [v2:172.21.15.189:6826/2395385355,v1:172.21.15.189:6827/2395385355] exists,up 26715dfc-c88e-42be-a958-ffdb37376a62 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 2.a [2,11] 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 2.b [8,11] 2024-09-12T12:12:56.068 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.7 [3,9] 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.10 [5,9] 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.1b [4,8] 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.21 [4,6] 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.24 [4,9] 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/228183714 expires 2024-09-13T12:04:15.038962+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6801/1596672143 expires 2024-09-13T12:04:15.038962+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6800/1596672143 expires 2024-09-13T12:04:15.038962+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/2160224100 expires 2024-09-13T12:04:15.038962+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/612748114 expires 2024-09-13T12:04:15.038962+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6801/1104802518 expires 2024-09-13T12:03:52.204874+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/936885966 expires 2024-09-13T12:03:52.204874+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/2994676901 expires 2024-09-13T12:03:52.204874+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6800/1104802518 expires 2024-09-13T12:03:52.204874+0000 2024-09-12T12:12:56.069 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/1305616495 expires 2024-09-13T12:03:52.204874+0000 2024-09-12T12:12:56.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.068+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f43440a2c30 msgr2=0x7f43440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:56.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.068+0000 7f435e071640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f43440a2c30 0x7f43440a50f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4358102c60 tx=0x7f4348005d30 comp rx=0 tx=0).stop 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.068+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 msgr2=0x7f43581a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.068+0000 7f435e071640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f43581a16b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f43500098d0 tx=0x7f435002f840 comp rx=0 tx=0).stop 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.068+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 shutdown_connections 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.068+0000 7f435e071640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f43440a2c30 0x7f43440a50f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.068+0000 7f435e071640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f43581037c0 0x7f43581a16b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.069+0000 7f435e071640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4358102e00 0x7f435819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.069+0000 7f435e071640 1 --2- 172.21.15.137:0/3859538680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4358101c00 0x7f435819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.069+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 >> 172.21.15.137:0/3859538680 conn(0x7f43580fd390 msgr2=0x7f4358074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.069+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 shutdown_connections 2024-09-12T12:12:56.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:56.069+0000 7f435e071640 1 -- 172.21.15.137:0/3859538680 wait complete. 2024-09-12T12:12:56.447 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:56 smithi137 ceph-mon[30950]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:56.447 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:56 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3859538680' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T12:12:56.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:56 smithi189 ceph-mon[35475]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:56.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:56 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3859538680' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T12:12:56.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:56 smithi171 ceph-mon[35098]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:56.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:56 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3859538680' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T12:12:56.902 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T12:12:56.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:12:57.163 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:57.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 -- 172.21.15.137:0/191930231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1810c170 msgr2=0x7f1b1810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/191930231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1810c170 0x7f1b1810e610 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009ca0 tx=0x7f1b1002a950 comp rx=0 tx=0).stop 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 -- 172.21.15.137:0/191930231 shutdown_connections 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/191930231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b1810c170 0x7f1b1810e610 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/191930231 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b18103660 0x7f1b1810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/191930231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b18102d40 0x7f1b18103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 -- 172.21.15.137:0/191930231 >> 172.21.15.137:0/191930231 conn(0x7f1b180fc7a0 msgr2=0x7f1b180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 -- 172.21.15.137:0/191930231 shutdown_connections 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.676+0000 7f1b1ec69640 1 -- 172.21.15.137:0/191930231 wait complete. 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 Processor -- start 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 -- start start 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18102d40 0x7f1b18198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:57.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b18103660 0x7f1b18198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b1810c170 0x7f1b1819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18112030 con 0x7f1b18102d40 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1b18111eb0 con 0x7f1b1810c170 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ec69640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b181121b0 con 0x7f1b18103660 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18102d40 0x7f1b18198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18102d40 0x7f1b18198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38806/0 (socket says 172.21.15.137:38806) 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 -- 172.21.15.137:0/597251358 learned_addr learned my addr 172.21.15.137:0/597251358 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1d206640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b1810c170 0x7f1b1819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 -- 172.21.15.137:0/597251358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b18103660 msgr2=0x7f1b18198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b17fff640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b18103660 0x7f1b18198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b18103660 0x7f1b18198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 -- 172.21.15.137:0/597251358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b1810c170 msgr2=0x7f1b1819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b1810c170 0x7f1b1819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1ca05640 1 -- 172.21.15.137:0/597251358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1819d5f0 con 0x7f1b18102d40 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.677+0000 7f1b1d206640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b1810c170 0x7f1b1819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.678+0000 7f1b1ca05640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18102d40 0x7f1b18198640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b4a0 tx=0x7f1b0c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.678+0000 7f1b15ffb640 1 -- 172.21.15.137:0/597251358 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c014070 con 0x7f1b18102d40 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.678+0000 7f1b15ffb640 1 -- 172.21.15.137:0/597251358 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0c010af0 con 0x7f1b18102d40 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.678+0000 7f1b15ffb640 1 -- 172.21.15.137:0/597251358 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c017d10 con 0x7f1b18102d40 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.678+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1819d8e0 con 0x7f1b18102d40 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.678+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b181a51c0 con 0x7f1b18102d40 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.679+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b181031a0 con 0x7f1b18102d40 2024-09-12T12:12:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.680+0000 7f1b15ffb640 1 -- 172.21.15.137:0/597251358 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1b0c017070 con 0x7f1b18102d40 2024-09-12T12:12:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.681+0000 7f1b15ffb640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1b080a2b80 0x7f1b080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.681+0000 7f1b17fff640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1b080a2b80 0x7f1b080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.681+0000 7f1b15ffb640 1 -- 172.21.15.137:0/597251358 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f1b0c0ce490 con 0x7f1b18102d40 2024-09-12T12:12:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.681+0000 7f1b17fff640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1b080a2b80 0x7f1b080a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1b180fee00 tx=0x7f1b04008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.685+0000 7f1b15ffb640 1 -- 172.21.15.137:0/597251358 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b0c09b420 con 0x7f1b18102d40 2024-09-12T12:12:57.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.885+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b18107210 con 0x7f1b18102d40 2024-09-12T12:12:57.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.886+0000 7f1b15ffb640 1 -- 172.21.15.137:0/597251358 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7f1b0c09ad20 con 0x7f1b18102d40 2024-09-12T12:12:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:57.886 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"btime":"2024-09-12T12:11:24:550801+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:11:24.095061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:12:57.886 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-09-12T12:12:57.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.888+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1b080a2b80 msgr2=0x7f1b080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:57.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.888+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1b080a2b80 0x7f1b080a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1b180fee00 tx=0x7f1b04008040 comp rx=0 tx=0).stop 2024-09-12T12:12:57.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.888+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18102d40 msgr2=0x7f1b18198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:57.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.888+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18102d40 0x7f1b18198640 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00b4a0 tx=0x7f1b0c00b960 comp rx=0 tx=0).stop 2024-09-12T12:12:57.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.888+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 shutdown_connections 2024-09-12T12:12:57.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.889+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f1b080a2b80 0x7f1b080a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.889+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b1810c170 0x7f1b1819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.889+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b18103660 0x7f1b18198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.889+0000 7f1b1ec69640 1 --2- 172.21.15.137:0/597251358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b18102d40 0x7f1b18198640 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:57.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.889+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 >> 172.21.15.137:0/597251358 conn(0x7f1b180fc7a0 msgr2=0x7f1b180fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:57.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.889+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 shutdown_connections 2024-09-12T12:12:57.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:57.889+0000 7f1b1ec69640 1 -- 172.21.15.137:0/597251358 wait complete. 2024-09-12T12:12:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:58 smithi137 ceph-mon[30950]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:58.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:12:58 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/597251358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:58.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:58 smithi189 ceph-mon[35475]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:58.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:12:58 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/597251358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:58.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:58 smithi171 ceph-mon[35098]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-12T12:12:58.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:12:58 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/597251358' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:12:58.731 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:12:58.977 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:12:59.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.492+0000 7f390cb9d640 1 -- 172.21.15.137:0/667556914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102570 msgr2=0x7f39081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:59.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.492+0000 7f390cb9d640 1 --2- 172.21.15.137:0/667556914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102570 0x7f39081029b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009900 tx=0x7f38f002a480 comp rx=0 tx=0).stop 2024-09-12T12:12:59.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 -- 172.21.15.137:0/667556914 shutdown_connections 2024-09-12T12:12:59.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 --2- 172.21.15.137:0/667556914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3908102ef0 0x7f390810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 --2- 172.21.15.137:0/667556914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102570 0x7f39081029b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 --2- 172.21.15.137:0/667556914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908100e20 0x7f3908101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 -- 172.21.15.137:0/667556914 >> 172.21.15.137:0/667556914 conn(0x7f39080fc7d0 msgr2=0x7f39080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 -- 172.21.15.137:0/667556914 shutdown_connections 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 -- 172.21.15.137:0/667556914 wait complete. 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 Processor -- start 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.493+0000 7f390cb9d640 1 -- start start 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908100e20 0x7f3908198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3908102570 0x7f3908198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102ef0 0x7f390819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3908110fc0 con 0x7f3908102ef0 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3908110e40 con 0x7f3908102570 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3908111140 con 0x7f3908100e20 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102ef0 0x7f390819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:59.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102ef0 0x7f390819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38450/0 (socket says 172.21.15.137:38450) 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 -- 172.21.15.137:0/2809908224 learned_addr learned my addr 172.21.15.137:0/2809908224 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3905d9b640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3908102570 0x7f3908198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390659c640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908100e20 0x7f3908198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 -- 172.21.15.137:0/2809908224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908100e20 msgr2=0x7f3908198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908100e20 0x7f3908198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 -- 172.21.15.137:0/2809908224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3908102570 msgr2=0x7f3908198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3908102570 0x7f3908198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 -- 172.21.15.137:0/2809908224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390819d5d0 con 0x7f3908102ef0 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f3906d9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102ef0 0x7f390819cef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009fd0 tx=0x7f38f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f38ff7fe640 1 -- 172.21.15.137:0/2809908224 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f8011830 con 0x7f3908102ef0 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f38ff7fe640 1 -- 172.21.15.137:0/2809908224 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f8016070 con 0x7f3908102ef0 2024-09-12T12:12:59.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390819d8c0 con 0x7f3908102ef0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f38ff7fe640 1 -- 172.21.15.137:0/2809908224 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f800f2e0 con 0x7f3908102ef0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.494+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39081a51a0 con 0x7f3908102ef0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.495+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3908101200 con 0x7f3908102ef0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.497+0000 7f38ff7fe640 1 -- 172.21.15.137:0/2809908224 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f38f80119d0 con 0x7f3908102ef0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.498+0000 7f38ff7fe640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f38f40a2b80 0x7f38f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.498+0000 7f390659c640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f38f40a2b80 0x7f38f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.498+0000 7f38ff7fe640 1 -- 172.21.15.137:0/2809908224 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f38f80cee30 con 0x7f3908102ef0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.498+0000 7f390659c640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f38f40a2b80 0x7f38f40a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3900005fd0 tx=0x7f3900009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:12:59.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.501+0000 7f38ff7fe640 1 -- 172.21.15.137:0/2809908224 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38f809bdc0 con 0x7f3908102ef0 2024-09-12T12:12:59.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.702+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39081a53a0 con 0x7f3908102ef0 2024-09-12T12:12:59.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.703+0000 7f38ff7fe640 1 -- 172.21.15.137:0/2809908224 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11936 (secure 0 0 0) 0x7f38f809b6c0 con 0x7f3908102ef0 2024-09-12T12:12:59.703 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:12:59.703 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"btime":"2024-09-12T12:11:24:550801+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:11:24.095061+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:12:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-09-12T12:12:59.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f38f40a2b80 msgr2=0x7f38f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:59.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f38f40a2b80 0x7f38f40a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3900005fd0 tx=0x7f3900009870 comp rx=0 tx=0).stop 2024-09-12T12:12:59.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102ef0 msgr2=0x7f390819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:12:59.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102ef0 0x7f390819cef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38f8009fd0 tx=0x7f38f800b8e0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 shutdown_connections 2024-09-12T12:12:59.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f38f40a2b80 0x7f38f40a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3908102ef0 0x7f390819cef0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3908102570 0x7f3908198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 --2- 172.21.15.137:0/2809908224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3908100e20 0x7f3908198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:12:59.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 >> 172.21.15.137:0/2809908224 conn(0x7f39080fc7d0 msgr2=0x7f3908069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:12:59.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 shutdown_connections 2024-09-12T12:12:59.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:12:59.705+0000 7f390cb9d640 1 -- 172.21.15.137:0/2809908224 wait complete. 2024-09-12T12:13:00.423 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:00 smithi137 ceph-mon[30950]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T12:13:00.423 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:00 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2809908224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:00.521 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:13:00.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:00 smithi189 ceph-mon[35475]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T12:13:00.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:00 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2809908224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:00.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:00 smithi171 ceph-mon[35098]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-12T12:13:00.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:00 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2809908224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:00.767 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:01.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 -- 172.21.15.137:0/2976643836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 msgr2=0x7f936010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:01.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 --2- 172.21.15.137:0/2976643836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936010e610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9358009ca0 tx=0x7f935802a950 comp rx=0 tx=0).stop 2024-09-12T12:13:01.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 -- 172.21.15.137:0/2976643836 shutdown_connections 2024-09-12T12:13:01.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 --2- 172.21.15.137:0/2976643836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936010e610 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 --2- 172.21.15.137:0/2976643836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9360103700 0x7f936010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 --2- 172.21.15.137:0/2976643836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9360102de0 0x7f93601031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 -- 172.21.15.137:0/2976643836 >> 172.21.15.137:0/2976643836 conn(0x7f93600fc7a0 msgr2=0x7f93600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 -- 172.21.15.137:0/2976643836 shutdown_connections 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.244+0000 7f9366b2b640 1 -- 172.21.15.137:0/2976643836 wait complete. 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 Processor -- start 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 -- start start 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9360102de0 0x7f93601983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9360103700 0x7f9360198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9360111fb0 con 0x7f936010c180 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9360111e30 con 0x7f9360102de0 2024-09-12T12:13:01.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f9366b2b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9360112130 con 0x7f9360103700 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38468/0 (socket says 172.21.15.137:38468) 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 -- 172.21.15.137:0/1060478286 learned_addr learned my addr 172.21.15.137:0/1060478286 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f935ffff640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9360103700 0x7f9360198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 -- 172.21.15.137:0/1060478286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9360103700 msgr2=0x7f9360198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9360103700 0x7f9360198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93648c7640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9360102de0 0x7f93601983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 -- 172.21.15.137:0/1060478286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9360102de0 msgr2=0x7f93601983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9360102de0 0x7f93601983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f93650c8640 1 -- 172.21.15.137:0/1060478286 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936019d3a0 con 0x7f936010c180 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.245+0000 7f935ffff640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9360103700 0x7f9360198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:01.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.246+0000 7f93650c8640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936019ccc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9358006010 tx=0x7f935802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.246+0000 7f935dffb640 1 -- 172.21.15.137:0/1060478286 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358034040 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.246+0000 7f935dffb640 1 -- 172.21.15.137:0/1060478286 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9358003e00 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.246+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936019d630 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.246+0000 7f935dffb640 1 -- 172.21.15.137:0/1060478286 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580456e0 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.246+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93601a4f70 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.247+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93601031c0 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.248+0000 7f935dffb640 1 -- 172.21.15.137:0/1060478286 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9358037020 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.249+0000 7f935dffb640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93500a2b80 0x7f93500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.249+0000 7f93648c7640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93500a2b80 0x7f93500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.249+0000 7f935dffb640 1 -- 172.21.15.137:0/1060478286 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f93580f0180 con 0x7f936010c180 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.249+0000 7f93648c7640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93500a2b80 0x7f93500a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f935400ba40 tx=0x7f9354005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:01.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.253+0000 7f935dffb640 1 -- 172.21.15.137:0/1060478286 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93580bd110 con 0x7f936010c180 2024-09-12T12:13:01.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.421+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93601a51a0 con 0x7f936010c180 2024-09-12T12:13:01.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.422+0000 7f935dffb640 1 -- 172.21.15.137:0/1060478286 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22734 (secure 0 0 0) 0x7f93580bca10 con 0x7f936010c180 2024-09-12T12:13:01.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:01.423 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":84,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:12:17.022250+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93500a2b80 msgr2=0x7f93500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93500a2b80 0x7f93500a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f935400ba40 tx=0x7f9354005cc0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 msgr2=0x7f936019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936019ccc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9358006010 tx=0x7f935802f710 comp rx=0 tx=0).stop 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 shutdown_connections 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f93500a2b80 0x7f93500a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f936010c180 0x7f936019ccc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9360103700 0x7f9360198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 --2- 172.21.15.137:0/1060478286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9360102de0 0x7f93601983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:01.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 >> 172.21.15.137:0/1060478286 conn(0x7f93600fc7a0 msgr2=0x7f93600fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:01.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 shutdown_connections 2024-09-12T12:13:01.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:01.425+0000 7f9366b2b640 1 -- 172.21.15.137:0/1060478286 wait complete. 2024-09-12T12:13:02.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T12:11:13.196665+0000', 'modified': '2024-09-12T12:11:24.095061+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24476, 'mds_1': 24475, 'mds_2': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6833/4107944214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 4107944214}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 4107944214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/1673181827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 1673181827}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 1673181827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/2972240277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 2972240277}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 2972240277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.189:6837/2107760146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 2107760146}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 2107760146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.189:6839/4067701297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 4067701297}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 4067701297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.171:6839/1705495422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1705495422}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1705495422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24476, 24475, 24454]} 2024-09-12T12:13:02.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T12:13:02.141 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T12:13:02.389 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:02.510 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:02 smithi137 ceph-mon[30950]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T12:13:02.510 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:02 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1060478286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:13:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:02 smithi189 ceph-mon[35475]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T12:13:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:02 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1060478286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:13:02.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:02 smithi171 ceph-mon[35098]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-12T12:13:02.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:02 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1060478286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 -- 172.21.15.137:0/2949371948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 msgr2=0x7f2ce4100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 --2- 172.21.15.137:0/2949371948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce4100860 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800aa10 tx=0x7f2cd802b720 comp rx=0 tx=0).stop 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 -- 172.21.15.137:0/2949371948 shutdown_connections 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 --2- 172.21.15.137:0/2949371948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce41055b0 0x7f2ce410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 --2- 172.21.15.137:0/2949371948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ce4100e30 0x7f2ce4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 --2- 172.21.15.137:0/2949371948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce4100860 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 -- 172.21.15.137:0/2949371948 >> 172.21.15.137:0/2949371948 conn(0x7f2ce4077210 msgr2=0x7f2ce40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:02.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 -- 172.21.15.137:0/2949371948 shutdown_connections 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 -- 172.21.15.137:0/2949371948 wait complete. 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 Processor -- start 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.905+0000 7f2ceac39640 1 -- start start 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ceac39640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce40fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ceac39640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4100e30 0x7f2ce40fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ceac39640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ce41055b0 0x7f2ce40fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ceac39640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce410fe30 con 0x7f2ce4100480 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ceac39640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce410fcb0 con 0x7f2ce4100e30 2024-09-12T12:13:02.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ceac39640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce410ffb0 con 0x7f2ce41055b0 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce40fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce40fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38476/0 (socket says 172.21.15.137:38476) 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 -- 172.21.15.137:0/4191560878 learned_addr learned my addr 172.21.15.137:0/4191560878 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ce41055b0 0x7f2ce40fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4100e30 0x7f2ce40fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 -- 172.21.15.137:0/4191560878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ce41055b0 msgr2=0x7f2ce40fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ce41055b0 0x7f2ce40fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 -- 172.21.15.137:0/4191560878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4100e30 msgr2=0x7f2ce40fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4100e30 0x7f2ce40fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 -- 172.21.15.137:0/4191560878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce41a6af0 con 0x7f2ce4100480 2024-09-12T12:13:02.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4100e30 0x7f2ce40fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:02.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce40fc690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8030660 tx=0x7f2cd8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:02.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce1ffb640 1 -- 172.21.15.137:0/4191560878 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd803f070 con 0x7f2ce4100480 2024-09-12T12:13:02.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce41a6d80 con 0x7f2ce4100480 2024-09-12T12:13:02.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce1ffb640 1 -- 172.21.15.137:0/4191560878 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd8032400 con 0x7f2ce4100480 2024-09-12T12:13:02.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.906+0000 7f2ce1ffb640 1 -- 172.21.15.137:0/4191560878 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8036040 con 0x7f2ce4100480 2024-09-12T12:13:02.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.907+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce41a7250 con 0x7f2ce4100480 2024-09-12T12:13:02.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ce1ffb640 1 -- 172.21.15.137:0/4191560878 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2cd8042070 con 0x7f2ce4100480 2024-09-12T12:13:02.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 Processor -- start 2024-09-12T12:13:02.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 -- start start 2024-09-12T12:13:02.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a6ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a78670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a78bb0 0x5555f7a7afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555f79d4090 con 0x5555f7a6e2f0 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555f797ba70 con 0x5555f7a6d9d0 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.909+0000 7f2ced012740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5555f79d0050 con 0x5555f7a78bb0 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a78670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a78670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38490/0 (socket says 172.21.15.137:38490) 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 -- 172.21.15.137:0/2851586396 learned_addr learned my addr 172.21.15.137:0/2851586396 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a6ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce1ffb640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2cd40a2b80 0x7f2cd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a78bb0 0x5555f7a7afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2cd40a2b80 0x7f2cd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce1ffb640 1 -- 172.21.15.137:0/4191560878 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f2cd80ef830 con 0x7f2ce4100480 2024-09-12T12:13:02.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 -- 172.21.15.137:0/2851586396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a78bb0 msgr2=0x5555f7a7afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a78bb0 0x5555f7a7afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 -- 172.21.15.137:0/2851586396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 msgr2=0x5555f7a6ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a6ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 -- 172.21.15.137:0/2851586396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f7988130 con 0x5555f7a6e2f0 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a6ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a78670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80ed2a0 tx=0x7f2cd802b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bb643eea7a959a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2cceffd640 1 -- 172.21.15.137:0/2851586396 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8049070 con 0x5555f7a6e2f0 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2cd40a2b80 0x7f2cd40a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009fd0 tx=0x7f2cdc004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:02.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2cceffd640 1 -- 172.21.15.137:0/2851586396 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd80bc0c0 con 0x5555f7a6e2f0 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.910+0000 7f2cceffd640 1 -- 172.21.15.137:0/2851586396 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80b8730 con 0x5555f7a6e2f0 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 -- 172.21.15.137:0/2851586396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 msgr2=0x5555f7a78670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a78670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80ed2a0 tx=0x7f2cd802b720 comp rx=0 tx=0).stop 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 -- 172.21.15.137:0/2851586396 shutdown_connections 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a78bb0 0x5555f7a7afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a78670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 --2- 172.21.15.137:0/2851586396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a6ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 -- 172.21.15.137:0/2851586396 >> 172.21.15.137:0/2851586396 conn(0x5555f7a69f60 msgr2=0x5555f7a6a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 -- 172.21.15.137:0/2851586396 shutdown_connections 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.911+0000 7f2ced012740 1 -- 172.21.15.137:0/2851586396 wait complete. 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 -- start start 2024-09-12T12:13:02.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 Processor -- start 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6d9d0 0x5555f7a78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a75780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a78bb0 0x5555f7a75cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a7b540 con 0x5555f7a6e2f0 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555f798ef20 con 0x5555f7a78bb0 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a7b6c0 con 0x5555f7a6d9d0 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ced012740 10 client.0 _ll_register_callbacks cb 0x5555f7958030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a75780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a75780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38500/0 (socket says 172.21.15.137:38500) 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2635641803 learned_addr learned my addr 172.21.15.137:0/2635641803 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a78bb0 0x5555f7a75cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6d9d0 0x5555f7a78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6d9d0 msgr2=0x5555f7a78430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6d9d0 0x5555f7a78430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2635641803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a78bb0 msgr2=0x5555f7a75cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.912+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a78bb0 0x5555f7a75cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2635641803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f77452b0 con 0x5555f7a6e2f0 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6d9d0 0x5555f7a78430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a75780 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000f4c0 tx=0x7f2cd000f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf4d5d1c833c840 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2cc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2635641803 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0013850 con 0x5555f7a6e2f0 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2635641803 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f2cd00139f0 con 0x5555f7a6e2f0 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd80bc0c0 con 0x5555f7a6e2f0 2024-09-12T12:13:02.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2635641803 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2cd0022070 con 0x5555f7a6e2f0 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2ced012740 10 client.15105 fetch_fsmap learned FSMap version 23 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5555f7a86de0 con 0x5555f7a6e2f0 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2635641803 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd80bc0c0 con 0x5555f7a6e2f0 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2635641803 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f2cd00113e0 con 0x5555f7a6e2f0 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.913+0000 7f2ced012740 10 client.15105 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2ced012740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2ced012740 20 client.15105 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2ced012740 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a89490 0x5555f7a8a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2ced012740 4 client.15105 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 --> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555f79e1600 con 0x5555f7a89490 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a89490 0x5555f7a8a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2cc3fff640 10 client.15105 ms_handle_connect on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.914+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a89490 0x5555f7a8a900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=527e43cef533e20d server_cookie=3c9f2c007987bb20 in_seq=0 out_seq=0 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2635641803 <== mds.2 v2:172.21.15.189:6836/2107760146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555f79e1600 con 0x5555f7a89490 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2cc3fff640 10 client.15105 handle_command_reply: tid=0 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 client.15105 shutdown 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 msgr2=0x5555f7a75780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a75780 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000f4c0 tx=0x7f2cd000f980 comp rx=0 tx=0).stop 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 shutdown_connections 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a89490 0x5555f7a8a900 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2cc3fff640 0 client.15105 ms_handle_reset on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a78bb0 0x5555f7a75cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a75780 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 --2- 172.21.15.137:0/2635641803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6d9d0 0x5555f7a78430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 >> 172.21.15.137:0/2635641803 conn(0x5555f7a69f60 msgr2=0x5555f7a6fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:02.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 shutdown_connections 2024-09-12T12:13:02.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.915+0000 7f2ced012740 1 -- 172.21.15.137:0/2635641803 wait complete. 2024-09-12T12:13:02.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.916+0000 7f2ced012740 20 client.15105 trim_cache size 0 max 16384 2024-09-12T12:13:02.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 Processor -- start 2024-09-12T12:13:02.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 -- start start 2024-09-12T12:13:02.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a69f60 0x5555f7a84710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a68ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a88c40 con 0x5555f7a6e2f0 2024-09-12T12:13:02.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a89310 con 0x5555f7a6d9d0 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ced012740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a899e0 con 0x5555f7a69f60 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a68ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a68ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38508/0 (socket says 172.21.15.137:38508) 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 -- 172.21.15.137:0/1030142164 learned_addr learned my addr 172.21.15.137:0/1030142164 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a69f60 0x5555f7a84710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 -- 172.21.15.137:0/1030142164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a69f60 msgr2=0x5555f7a84710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a69f60 0x5555f7a84710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 -- 172.21.15.137:0/1030142164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 msgr2=0x5555f7a74050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a74050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce91d6640 1 -- 172.21.15.137:0/1030142164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f7a74730 con 0x5555f7a6e2f0 2024-09-12T12:13:02.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a74050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.933+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a69f60 0x5555f7a84710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a68ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c240 tx=0x7f2cdc0098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b252c1ebe54b696 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2cce7fc640 1 -- 172.21.15.137:0/1030142164 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc02a070 con 0x5555f7a6e2f0 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2cce7fc640 1 -- 172.21.15.137:0/1030142164 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cdc016040 con 0x5555f7a6e2f0 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2cce7fc640 1 -- 172.21.15.137:0/1030142164 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc015050 con 0x5555f7a6e2f0 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 -- 172.21.15.137:0/1030142164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 msgr2=0x5555f7a68ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a68ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc00c240 tx=0x7f2cdc0098d0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 -- 172.21.15.137:0/1030142164 shutdown_connections 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6e2f0 0x5555f7a68ef0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a6d9d0 0x5555f7a74050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 --2- 172.21.15.137:0/1030142164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a69f60 0x5555f7a84710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 -- 172.21.15.137:0/1030142164 >> 172.21.15.137:0/1030142164 conn(0x5555f7a787a0 msgr2=0x5555f7a687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 -- 172.21.15.137:0/1030142164 shutdown_connections 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.934+0000 7f2ced012740 1 -- 172.21.15.137:0/1030142164 wait complete. 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 -- start start 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 Processor -- start 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a69f60 0x5555f7a93420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6d9d0 0x5555f7a8e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6e2f0 0x5555f7a8ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a8a780 con 0x5555f7a6d9d0 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a8a0b0 con 0x5555f7a69f60 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5555f7a8ae50 con 0x5555f7a6e2f0 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ced012740 10 client.0 _ll_register_callbacks cb 0x7f2cd000a510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6d9d0 0x5555f7a8e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6d9d0 0x5555f7a8e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38516/0 (socket says 172.21.15.137:38516) 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.935+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2105185614 learned_addr learned my addr 172.21.15.137:0/2105185614 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6e2f0 0x5555f7a8ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a69f60 0x5555f7a93420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6e2f0 msgr2=0x5555f7a8ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6e2f0 0x5555f7a8ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2105185614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a69f60 msgr2=0x5555f7a93420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a69f60 0x5555f7a93420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce3fff640 1 -- 172.21.15.137:0/2105185614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f7a8f570 con 0x5555f7a6d9d0 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce91d6640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6e2f0 0x5555f7a8ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ce3fff640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6d9d0 0x5555f7a8e750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00119c0 tx=0x7f2cd000ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984c65381de351f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2cc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2105185614 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0011c40 con 0x5555f7a6d9d0 2024-09-12T12:13:02.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2105185614 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f2cd00059c0 con 0x5555f7a6d9d0 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cdc016040 con 0x5555f7a6d9d0 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2105185614 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2cd0005bc0 con 0x5555f7a6d9d0 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ced012740 10 client.15111 fetch_fsmap learned FSMap version 23 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5555f7af3c20 con 0x5555f7a6d9d0 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2105185614 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cdc016040 con 0x5555f7a6d9d0 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.936+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2105185614 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f2cd0006040 con 0x5555f7a6d9d0 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ced012740 10 client.15111 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ced012740 10 client.15111 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ced012740 20 client.15111 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ced012740 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a78bb0 0x5555f7af7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ced012740 4 client.15111 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 --> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] -- command(tid 0: {"prefix": "status"}) -- 0x5555f79dbb10 con 0x5555f7a78bb0 2024-09-12T12:13:02.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a78bb0 0x5555f7af7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ce89d5640 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a78bb0 0x5555f7af7500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b53faeddfafb713 server_cookie=75e2db4529c9b3d in_seq=0 out_seq=0 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2cc3fff640 10 client.15111 ms_handle_connect on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2cc3fff640 1 -- 172.21.15.137:0/2105185614 <== mds.2 v2:172.21.15.189:6836/2107760146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5555f79dbb10 con 0x5555f7a78bb0 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2cc3fff640 10 client.15111 handle_command_reply: tid=0 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.937+0000 7f2ced012740 1 client.15111 shutdown 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6d9d0 msgr2=0x5555f7a8e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6d9d0 0x5555f7a8e750 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00119c0 tx=0x7f2cd000ae10 comp rx=0 tx=0).stop 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 shutdown_connections 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x5555f7a78bb0 0x5555f7af7500 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5555f7a6e2f0 0x5555f7a8ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5555f7a6d9d0 0x5555f7a8e750 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 --2- 172.21.15.137:0/2105185614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5555f7a69f60 0x5555f7a93420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 >> 172.21.15.137:0/2105185614 conn(0x5555f7a787a0 msgr2=0x5555f7a75040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2cc3fff640 0 client.15111 ms_handle_reset on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 shutdown_connections 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 1 -- 172.21.15.137:0/2105185614 wait complete. 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ced012740 20 client.15111 trim_cache size 0 max 16384 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 2, 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24454, 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:13:02.946 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:13:02.947 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:13:02.947 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 101.390660955, 2024-09-12T12:13:02.947 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-09-12T12:13:02.947 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 84, 2024-09-12T12:13:02.947 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T12:13:02.947 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 135.07980625799999 2024-09-12T12:13:02.947 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2cd40a2b80 msgr2=0x7f2cd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2cd40a2b80 0x7f2cd40a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009fd0 tx=0x7f2cdc004a40 comp rx=0 tx=0).stop 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 msgr2=0x7f2ce40fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce40fc690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8030660 tx=0x7f2cd8004340 comp rx=0 tx=0).stop 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 shutdown_connections 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2cd40a2b80 0x7f2cd40a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ce41055b0 0x7f2ce40fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ce4100e30 0x7f2ce40fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 --2- 172.21.15.137:0/4191560878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ce4100480 0x7f2ce40fc690 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.938+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 >> 172.21.15.137:0/4191560878 conn(0x7f2ce4077210 msgr2=0x7f2ce4103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:02.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.939+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 shutdown_connections 2024-09-12T12:13:02.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:02.939+0000 7f2ceac39640 1 -- 172.21.15.137:0/4191560878 wait complete. 2024-09-12T12:13:03.835 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T12:13:04.078 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:04.510 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:04 smithi137 ceph-mon[30950]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:13:04.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:04 smithi189 ceph-mon[35475]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:13:04.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.591+0000 7fe3d76a2640 1 -- 172.21.15.137:0/2167016190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 msgr2=0x7fe3d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.591+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/2167016190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d01029b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009900 tx=0x7fe3cc02a480 comp rx=0 tx=0).stop 2024-09-12T12:13:04.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 -- 172.21.15.137:0/2167016190 shutdown_connections 2024-09-12T12:13:04.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/2167016190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3d0102ef0 0x7fe3d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/2167016190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d01029b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/2167016190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0100e20 0x7fe3d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 -- 172.21.15.137:0/2167016190 >> 172.21.15.137:0/2167016190 conn(0x7fe3d00fc7d0 msgr2=0x7fe3d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 -- 172.21.15.137:0/2167016190 shutdown_connections 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 -- 172.21.15.137:0/2167016190 wait complete. 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 Processor -- start 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.592+0000 7fe3d76a2640 1 -- start start 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d76a2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0100e20 0x7fe3d00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d76a2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d76a2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3d0102ef0 0x7fe3d00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d76a2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0110f80 con 0x7fe3d0102570 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d76a2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0110e00 con 0x7fe3d0102ef0 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d76a2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0111100 con 0x7fe3d0100e20 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d00fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38536/0 (socket says 172.21.15.137:38536) 2024-09-12T12:13:04.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/1961020668 learned_addr learned my addr 172.21.15.137:0/1961020668 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3d0102ef0 0x7fe3d00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/1961020668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0100e20 msgr2=0x7fe3d00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d543e640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0100e20 0x7fe3d00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0100e20 0x7fe3d00fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/1961020668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3d0102ef0 msgr2=0x7fe3d00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3d0102ef0 0x7fe3d00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/1961020668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d01a6910 con 0x7fe3d0102570 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3d0102ef0 0x7fe3d00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d00fd1c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006010 tx=0x7fe3cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3c67fc640 1 -- 172.21.15.137:0/1961020668 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc03f070 con 0x7fe3d0102570 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d01a6ba0 con 0x7fe3d0102570 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3c67fc640 1 -- 172.21.15.137:0/1961020668 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc031c30 con 0x7fe3d0102570 2024-09-12T12:13:04.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.593+0000 7fe3c67fc640 1 -- 172.21.15.137:0/1961020668 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc035030 con 0x7fe3d0102570 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.594+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d01a70d0 con 0x7fe3d0102570 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.596+0000 7fe3d9a7b740 1 Processor -- start 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.596+0000 7fe3c67fc640 1 -- 172.21.15.137:0/1961020668 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe3cc042070 con 0x7fe3d0102570 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.596+0000 7fe3d9a7b740 1 -- start start 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd322f0 0x55899bd3c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd3cbb0 0x55899bd3efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55899bc98090 con 0x55899bd3cbb0 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55899bc3fa70 con 0x55899bd319d0 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55899bc94050 con 0x55899bd322f0 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd3cbb0 0x55899bd3efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd3cbb0 0x55899bd3efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38540/0 (socket says 172.21.15.137:38540) 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/3063671724 learned_addr learned my addr 172.21.15.137:0/3063671724 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:04.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d543e640 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd322f0 0x55899bd3c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3c67fc640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/3063671724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd322f0 msgr2=0x55899bd3c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd322f0 0x55899bd3c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/3063671724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 msgr2=0x55899bd31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/3063671724 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899bc4c130 con 0x55899bd3cbb0 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.597+0000 7fe3d543e640 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd322f0 0x55899bd3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.598+0000 7fe3c67fc640 1 -- 172.21.15.137:0/1961020668 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fe3cc0ef920 con 0x7fe3d0102570 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.598+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd3cbb0 0x55899bd3efa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0983b0 tx=0x7fe3cc086e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61084b334ae3c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.598+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8005cc0 tx=0x7fe3c8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:04.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.598+0000 7fe3bb7fe640 1 -- 172.21.15.137:0/3063671724 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0bbb10 con 0x55899bd3cbb0 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.598+0000 7fe3bb7fe640 1 -- 172.21.15.137:0/3063671724 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc087350 con 0x55899bd3cbb0 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.598+0000 7fe3bb7fe640 1 -- 172.21.15.137:0/3063671724 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0dc570 con 0x55899bd3cbb0 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/3063671724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd3cbb0 msgr2=0x55899bd3efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd3cbb0 0x55899bd3efa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0983b0 tx=0x7fe3cc086e40 comp rx=0 tx=0).stop 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/3063671724 shutdown_connections 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd3cbb0 0x55899bd3efa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd322f0 0x55899bd3c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/3063671724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/3063671724 >> 172.21.15.137:0/3063671724 conn(0x55899bd2df60 msgr2=0x55899bd2e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/3063671724 shutdown_connections 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/3063671724 wait complete. 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 -- start start 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 Processor -- start 2024-09-12T12:13:04.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.599+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd396a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55899bd3f540 con 0x55899bd322f0 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55899bc52f20 con 0x55899bd319d0 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55899bd3f6c0 con 0x55899bd3cbb0 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 10 client.0 _ll_register_callbacks cb 0x55899bc1c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd396a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd396a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38544/0 (socket says 172.21.15.137:38544) 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/690672005 learned_addr learned my addr 172.21.15.137:0/690672005 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d543e640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 msgr2=0x55899bd39be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd39be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/690672005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 msgr2=0x55899bd3c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd3c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/690672005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899ba092b0 con 0x55899bd322f0 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d543e640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd396a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0983b0 tx=0x7fe3cc03b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebd6cc5516be95e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/690672005 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc049040 con 0x55899bd322f0 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/690672005 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fe3cc0ba4b0 con 0x55899bd322f0 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3cc087350 con 0x55899bd322f0 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/690672005 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe3cc0bbb10 con 0x55899bd322f0 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 10 client.15123 fetch_fsmap learned FSMap version 23 2024-09-12T12:13:04.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.600+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55899bd3f540 con 0x55899bd322f0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/690672005 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc087350 con 0x55899bd322f0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/690672005 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe3cc0b8820 con 0x55899bd322f0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3d9a7b740 10 client.15123 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3d9a7b740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3d9a7b740 20 client.15123 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bd4e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3d9a7b740 4 client.15123 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 --> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55899bca5600 con 0x55899bd4cec0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.601+0000 7fe3d543e640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bd4e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.602+0000 7fe3b8ff9640 10 client.15123 ms_handle_connect on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.602+0000 7fe3d543e640 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bd4e330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7602104baaf15b43 server_cookie=492a39d041cb77c5 in_seq=0 out_seq=0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.602+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/690672005 <== mds.1 v2:172.21.15.189:6838/4067701297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55899bca5600 con 0x55899bd4cec0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.602+0000 7fe3b8ff9640 10 client.15123 handle_command_reply: tid=0 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.602+0000 7fe3d9a7b740 1 client.15123 shutdown 2024-09-12T12:13:04.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.602+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 msgr2=0x55899bd396a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.602+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd396a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0983b0 tx=0x7fe3cc03b870 comp rx=0 tx=0).stop 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 shutdown_connections 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bd4e330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3b8ff9640 0 client.15123 ms_handle_reset on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd396a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/690672005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 >> 172.21.15.137:0/690672005 conn(0x55899bd2df60 msgr2=0x55899bd33c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 shutdown_connections 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/690672005 wait complete. 2024-09-12T12:13:04.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.603+0000 7fe3d9a7b740 20 client.15123 trim_cache size 0 max 16384 2024-09-12T12:13:04.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 Processor -- start 2024-09-12T12:13:04.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 -- start start 2024-09-12T12:13:04.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd34de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd33b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd36900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55899bd4c670 con 0x55899bd322f0 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55899bd4cd40 con 0x55899bd319d0 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55899bd4d410 con 0x55899bd3cbb0 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd33b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd33b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38546/0 (socket says 172.21.15.137:38546) 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.634+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2391242968 learned_addr learned my addr 172.21.15.137:0/2391242968 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd36900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d543e640 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd34de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2391242968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 msgr2=0x55899bd36900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd36900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2391242968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 msgr2=0x55899bd34de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd34de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2391242968 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55899bd2dd90 con 0x55899bd322f0 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd36900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd33b30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0ed7c0 tx=0x7fe3cc0efc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a61770ee0593f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3baffd640 1 -- 172.21.15.137:0/2391242968 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc049070 con 0x55899bd322f0 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3baffd640 1 -- 172.21.15.137:0/2391242968 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc0bc1b0 con 0x55899bd322f0 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.635+0000 7fe3baffd640 1 -- 172.21.15.137:0/2391242968 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc08f830 con 0x55899bd322f0 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2391242968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 msgr2=0x55899bd33b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd33b30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0ed7c0 tx=0x7fe3cc0efc20 comp rx=0 tx=0).stop 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2391242968 shutdown_connections 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd3cbb0 0x55899bd36900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd33b30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2391242968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd319d0 0x55899bd34de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2391242968 >> 172.21.15.137:0/2391242968 conn(0x55899bd2df60 msgr2=0x55899bd33420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:04.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2391242968 shutdown_connections 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2391242968 wait complete. 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 -- start start 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.636+0000 7fe3d9a7b740 1 Processor -- start 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd319d0 0x55899bd56960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd56ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d9a7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd3cbb0 0x55899bd4fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55899bd4e1b0 con 0x55899bd322f0 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55899bd4dae0 con 0x55899bd3cbb0 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d9a7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55899bd4e880 con 0x55899bd319d0 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d9a7b740 10 client.0 _ll_register_callbacks cb 0x7fe3c80084f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd56ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd56ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38552/0 (socket says 172.21.15.137:38552) 2024-09-12T12:13:04.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2180622678 learned_addr learned my addr 172.21.15.137:0/2180622678 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd3cbb0 0x55899bd4fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d543e640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd319d0 0x55899bd56960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd319d0 msgr2=0x55899bd56960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd319d0 0x55899bd56960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2180622678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd3cbb0 msgr2=0x55899bd4fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd3cbb0 0x55899bd4fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 -- 172.21.15.137:0/2180622678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55899bd506c0 con 0x55899bd322f0 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d5c3f640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd3cbb0 0x55899bd4fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3d4c3d640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd56ea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc000c00 tx=0x7fe3cc0dc6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5893078edb6e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.637+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/2180622678 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc049070 con 0x55899bd322f0 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/2180622678 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fe3cc0d50c0 con 0x55899bd322f0 2024-09-12T12:13:04.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3cc0bc1b0 con 0x55899bd322f0 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/2180622678 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe3cc0bbcf0 con 0x55899bd322f0 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 10 client.15129 fetch_fsmap learned FSMap version 23 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55899bd506c0 con 0x55899bd322f0 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/2180622678 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc0bc1b0 con 0x55899bd322f0 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/2180622678 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe3cc0c00f0 con 0x55899bd322f0 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 10 client.15129 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 20 client.15129 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:04.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bdba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 4 client.15129 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 --> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] -- command(tid 0: {"prefix": "status"}) -- 0x55899bc9fb10 con 0x55899bd4cec0 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.638+0000 7fe3d543e640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bdba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d543e640 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bdba350 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbfcf30d87819c8b server_cookie=f00ac383e52cd884 in_seq=0 out_seq=0 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3b8ff9640 10 client.15129 ms_handle_connect on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3b8ff9640 1 -- 172.21.15.137:0/2180622678 <== mds.1 v2:172.21.15.189:6838/4067701297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55899bc9fb10 con 0x55899bd4cec0 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3b8ff9640 10 client.15129 handle_command_reply: tid=0 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 client.15129 shutdown 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 msgr2=0x55899bd56ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd56ea0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc000c00 tx=0x7fe3cc0dc6c0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 shutdown_connections 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55899bd4cec0 0x55899bdba350 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3b8ff9640 0 client.15129 ms_handle_reset on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55899bd3cbb0 0x55899bd4fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55899bd322f0 0x55899bd56ea0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 --2- 172.21.15.137:0/2180622678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55899bd319d0 0x55899bd56960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 >> 172.21.15.137:0/2180622678 conn(0x55899bd2df60 msgr2=0x55899bd38660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 shutdown_connections 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 1 -- 172.21.15.137:0/2180622678 wait complete. 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.639+0000 7fe3d9a7b740 20 client.15129 trim_cache size 0 max 16384 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:13:04.649 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 1, 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24475, 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 104.10395524499999, 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 84, 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 131.498263165 2024-09-12T12:13:04.650 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe3bc0a2b80 msgr2=0x7fe3bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8005cc0 tx=0x7fe3c8004a40 comp rx=0 tx=0).stop 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 msgr2=0x7fe3d00fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d00fd1c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006010 tx=0x7fe3cc003c10 comp rx=0 tx=0).stop 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 shutdown_connections 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe3bc0a2b80 0x7fe3bc0a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3d0102ef0 0x7fe3d00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3d0102570 0x7fe3d00fd1c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 --2- 172.21.15.137:0/1961020668 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d0100e20 0x7fe3d00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:04.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 >> 172.21.15.137:0/1961020668 conn(0x7fe3d00fc7d0 msgr2=0x7fe3d006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:04.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 shutdown_connections 2024-09-12T12:13:04.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:04.640+0000 7fe3d76a2640 1 -- 172.21.15.137:0/1961020668 wait complete. 2024-09-12T12:13:04.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:04 smithi171 ceph-mon[35098]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-12T12:13:05.478 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T12:13:05.729 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:06.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.267+0000 7f45df5bf640 1 -- 172.21.15.137:0/185196784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d8103670 msgr2=0x7f45d810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.267+0000 7f45df5bf640 1 --2- 172.21.15.137:0/185196784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d8103670 0x7f45d810bb60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009900 tx=0x7f45c802a480 comp rx=0 tx=0).stop 2024-09-12T12:13:06.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 -- 172.21.15.137:0/185196784 shutdown_connections 2024-09-12T12:13:06.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 --2- 172.21.15.137:0/185196784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d810c180 0x7f45d810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 --2- 172.21.15.137:0/185196784 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d8103670 0x7f45d810bb60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 --2- 172.21.15.137:0/185196784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45d8102d50 0x7f45d8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 -- 172.21.15.137:0/185196784 >> 172.21.15.137:0/185196784 conn(0x7f45d80fc7d0 msgr2=0x7f45d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 -- 172.21.15.137:0/185196784 shutdown_connections 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 -- 172.21.15.137:0/185196784 wait complete. 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 Processor -- start 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.268+0000 7f45df5bf640 1 -- start start 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45df5bf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8102d50 0x7f45d8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45df5bf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45d8103670 0x7f45d8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45df5bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d810c180 0x7f45d819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45df5bf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8111fe0 con 0x7f45d810c180 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45df5bf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8111e60 con 0x7f45d8103670 2024-09-12T12:13:06.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45df5bf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8112160 con 0x7f45d8102d50 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d810c180 0x7f45d819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d810c180 0x7f45d819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38570/0 (socket says 172.21.15.137:38570) 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 -- 172.21.15.137:0/188443312 learned_addr learned my addr 172.21.15.137:0/188443312 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45dd35b640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8102d50 0x7f45d8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45dcb5a640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45d8103670 0x7f45d8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 -- 172.21.15.137:0/188443312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8102d50 msgr2=0x7f45d8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8102d50 0x7f45d8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 -- 172.21.15.137:0/188443312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45d8103670 msgr2=0x7f45d8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45d8103670 0x7f45d8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 -- 172.21.15.137:0/188443312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d819d510 con 0x7f45d810c180 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.269+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d810c180 0x7f45d819ce30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f45d000eaa0 tx=0x7f45d000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.270+0000 7f45ce7fc640 1 -- 172.21.15.137:0/188443312 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004d20 con 0x7f45d810c180 2024-09-12T12:13:06.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.270+0000 7f45ce7fc640 1 -- 172.21.15.137:0/188443312 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0005020 con 0x7f45d810c180 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.270+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d819d800 con 0x7f45d810c180 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.270+0000 7f45ce7fc640 1 -- 172.21.15.137:0/188443312 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d00116d0 con 0x7f45d810c180 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.270+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81a50e0 con 0x7f45d810c180 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.272+0000 7f45ce7fc640 1 -- 172.21.15.137:0/188443312 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f45d00051d0 con 0x7f45d810c180 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.272+0000 7f45e1998740 1 Processor -- start 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45e1998740 1 -- start start 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45ce7fc640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45c40a2c00 0x7f45c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45e1998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a70387a9d0 0x55a70387adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45e1998740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a703885670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45e1998740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a703885bb0 0x55a703887fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45e1998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7037e1090 con 0x55a70387a9d0 2024-09-12T12:13:06.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45e1998740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a703788a70 con 0x55a703885bb0 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45e1998740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7037dd050 con 0x55a70387b2f0 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a70387a9d0 0x55a70387adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a70387a9d0 0x55a70387adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38584/0 (socket says 172.21.15.137:38584) 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 -- 172.21.15.137:0/4264702492 learned_addr learned my addr 172.21.15.137:0/4264702492 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dcb5a640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45c40a2c00 0x7f45c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45ce7fc640 1 -- 172.21.15.137:0/188443312 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f45d00cf0f0 con 0x7f45d810c180 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dcb5a640 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a703885670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 -- 172.21.15.137:0/4264702492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 msgr2=0x55a703885670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a703885670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 -- 172.21.15.137:0/4264702492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a703885bb0 msgr2=0x55a703887fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a703885bb0 0x55a703887fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 -- 172.21.15.137:0/4264702492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a703795130 con 0x55a70387a9d0 2024-09-12T12:13:06.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dd35b640 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a70387a9d0 0x55a70387adb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009a70 tx=0x7f45d4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58667aa7fe9a24c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45dcb5a640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45c40a2c00 0x7f45c40a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009230 tx=0x7f45c8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.273+0000 7f45c37fe640 1 -- 172.21.15.137:0/4264702492 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4017070 con 0x55a70387a9d0 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45c37fe640 1 -- 172.21.15.137:0/4264702492 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d4013470 con 0x55a70387a9d0 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45c37fe640 1 -- 172.21.15.137:0/4264702492 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4012630 con 0x55a70387a9d0 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 -- 172.21.15.137:0/4264702492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a70387a9d0 msgr2=0x55a70387adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a70387a9d0 0x55a70387adb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009a70 tx=0x7f45d4009f30 comp rx=0 tx=0).stop 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 -- 172.21.15.137:0/4264702492 shutdown_connections 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a703885bb0 0x55a703887fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a703885670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 --2- 172.21.15.137:0/4264702492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a70387a9d0 0x55a70387adb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 -- 172.21.15.137:0/4264702492 >> 172.21.15.137:0/4264702492 conn(0x55a703876f60 msgr2=0x55a703877340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 -- 172.21.15.137:0/4264702492 shutdown_connections 2024-09-12T12:13:06.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.274+0000 7f45e1998740 1 -- 172.21.15.137:0/4264702492 wait complete. 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 -- start start 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 Processor -- start 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a703885450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a7038827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703882ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a70388cb90 con 0x55a703885bb0 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a70379bf20 con 0x55a70387a9d0 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a70388cd10 con 0x55a70387b2f0 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45e1998740 10 client.0 _ll_register_callbacks cb 0x55a703765030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:06.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703882ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 0, 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24476, 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 112.810830088, 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 84, 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 124.344754721 2024-09-12T12:13:06.312 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703882ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38590/0 (socket says 172.21.15.137:38590) 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45ddb5c640 1 -- 172.21.15.137:0/428511641 learned_addr learned my addr 172.21.15.137:0/428511641 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45dd35b640 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a703885450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.275+0000 7f45dcb5a640 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a7038827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45ddb5c640 1 -- 172.21.15.137:0/428511641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 msgr2=0x55a7038827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a7038827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45ddb5c640 1 -- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 msgr2=0x55a703885450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a703885450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45ddb5c640 1 -- 172.21.15.137:0/428511641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7035522b0 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703882ce0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ccc20 tx=0x7f45d0065d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75b2fb809c9ac0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45c0ff9640 1 -- 172.21.15.137:0/428511641 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d001f040 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d4013470 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45c0ff9640 1 -- 172.21.15.137:0/428511641 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f45d0005560 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45c0ff9640 1 -- 172.21.15.137:0/428511641 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f45d009b980 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 10 client.15141 fetch_fsmap learned FSMap version 23 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a7038984b0 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45c0ff9640 1 -- 172.21.15.137:0/428511641 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d4013470 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45c0ff9640 1 -- 172.21.15.137:0/428511641 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f45d000f040 con 0x55a703885bb0 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 10 client.15141 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:13:06.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 20 client.15141 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a70389a990 0x55a70389be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 4 client.15141 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.276+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7037ee600 con 0x55a70389a990 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.277+0000 7f45dd35b640 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a70389a990 0x55a70389be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.277+0000 7f45c0ff9640 10 client.15141 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.277+0000 7f45dd35b640 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a70389a990 0x55a70389be00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab8f2467d70cb00 server_cookie=322d135a11ef5cc8 in_seq=0 out_seq=0 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45c0ff9640 1 -- 172.21.15.137:0/428511641 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7037ee600 con 0x55a70389a990 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45c0ff9640 10 client.15141 handle_command_reply: tid=0 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 client.15141 shutdown 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 msgr2=0x55a703882ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703882ce0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ccc20 tx=0x7f45d0065d70 comp rx=0 tx=0).stop 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 shutdown_connections 2024-09-12T12:13:06.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a70389a990 0x55a70389be00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703882ce0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a7038827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 --2- 172.21.15.137:0/428511641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a703885450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45c0ff9640 0 client.15141 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 >> 172.21.15.137:0/428511641 conn(0x55a703876f60 msgr2=0x55a70387cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 shutdown_connections 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 1 -- 172.21.15.137:0/428511641 wait complete. 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.278+0000 7f45e1998740 20 client.15141 trim_cache size 0 max 16384 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 Processor -- start 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 -- start start 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a703877340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a703877880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703881070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a70389a140 con 0x55a703885bb0 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a70389a810 con 0x55a70387a9d0 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45e1998740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a70389aee0 con 0x55a70387b2f0 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703881070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703881070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38604/0 (socket says 172.21.15.137:38604) 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 -- 172.21.15.137:0/634176497 learned_addr learned my addr 172.21.15.137:0/634176497 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 -- 172.21.15.137:0/634176497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 msgr2=0x55a703877880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/634176497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a703877880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 -- 172.21.15.137:0/634176497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 msgr2=0x55a703877340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/634176497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a703877340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 -- 172.21.15.137:0/634176497 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7038780f0 con 0x55a703885bb0 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/634176497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703881070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ccf90 tx=0x7f45d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f04ba0ec2b03d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:06.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45c2ffd640 1 -- 172.21.15.137:0/634176497 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d001f030 con 0x55a703885bb0 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45c2ffd640 1 -- 172.21.15.137:0/634176497 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d00861d0 con 0x55a703885bb0 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.296+0000 7f45c2ffd640 1 -- 172.21.15.137:0/634176497 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0012bd0 con 0x55a703885bb0 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 -- 172.21.15.137:0/634176497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 msgr2=0x55a703881070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 --2- 172.21.15.137:0/634176497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703881070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f45d00ccf90 tx=0x7f45d0000f40 comp rx=0 tx=0).stop 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 -- 172.21.15.137:0/634176497 shutdown_connections 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 --2- 172.21.15.137:0/634176497 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a703881070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 --2- 172.21.15.137:0/634176497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a703877880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 --2- 172.21.15.137:0/634176497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a703877340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 -- 172.21.15.137:0/634176497 >> 172.21.15.137:0/634176497 conn(0x55a703876f60 msgr2=0x55a703882760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 -- 172.21.15.137:0/634176497 shutdown_connections 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 -- 172.21.15.137:0/634176497 wait complete. 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.297+0000 7f45e1998740 1 -- start start 2024-09-12T12:13:06.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 1 Processor -- start 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a70389d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a70389d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a70389db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a70389bc80 con 0x55a703885bb0 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a70389b5b0 con 0x55a70387a9d0 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a70389c350 con 0x55a70387b2f0 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45e1998740 10 client.0 _ll_register_callbacks cb 0x7f45d0004f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a70389db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a70389db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38608/0 (socket says 172.21.15.137:38608) 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 -- 172.21.15.137:0/1264002873 learned_addr learned my addr 172.21.15.137:0/1264002873 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 -- 172.21.15.137:0/1264002873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 msgr2=0x55a70389d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a70389d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 -- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 msgr2=0x55a70389d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:06.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45dd35b640 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a70389d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a70389d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 -- 172.21.15.137:0/1264002873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7039032e0 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45ddb5c640 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a70389db30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45d009b680 tx=0x7f45d0097c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e745d3ed95b98b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.298+0000 7f45c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45c0ff9640 1 -- 172.21.15.137:0/1264002873 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d007e020 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45c0ff9640 1 -- 172.21.15.137:0/1264002873 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f45d009afd0 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d00861d0 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45c0ff9640 1 -- 172.21.15.137:0/1264002873 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f45d00a9050 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 10 client.15147 fetch_fsmap learned FSMap version 23 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a7039045e0 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45c0ff9640 1 -- 172.21.15.137:0/1264002873 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d00861d0 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45c0ff9640 1 -- 172.21.15.137:0/1264002873 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f45d009a840 con 0x55a703885bb0 2024-09-12T12:13:06.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 10 client.15147 fetch_fsmap finished waiting for FSMap version 23 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 20 client.15147 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a703908c10 0x55a703907920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 4 client.15147 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "status"}) -- 0x55a7037e8b10 con 0x55a703908c10 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.299+0000 7f45dd35b640 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a703908c10 0x55a703907920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45c0ff9640 10 client.15147 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45dd35b640 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a703908c10 0x55a703907920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af1ec6795e9f88b server_cookie=6d2d5ad7c49a4106 in_seq=0 out_seq=0 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45c0ff9640 1 -- 172.21.15.137:0/1264002873 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a7037e8b10 con 0x55a703908c10 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45c0ff9640 10 client.15147 handle_command_reply: tid=0 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 client.15147 shutdown 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 msgr2=0x55a70389db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a70389db30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f45d009b680 tx=0x7f45d0097c40 comp rx=0 tx=0).stop 2024-09-12T12:13:06.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 shutdown_connections 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a703908c10 0x55a703907920 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a703885bb0 0x55a70389db30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a70387b2f0 0x55a70389d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 --2- 172.21.15.137:0/1264002873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a70387a9d0 0x55a70389d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45c0ff9640 0 client.15147 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 >> 172.21.15.137:0/1264002873 conn(0x55a703876f60 msgr2=0x55a703882590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 shutdown_connections 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 1 -- 172.21.15.137:0/1264002873 wait complete. 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.300+0000 7f45e1998740 20 client.15147 trim_cache size 0 max 16384 2024-09-12T12:13:06.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45c40a2c00 msgr2=0x7f45c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45c40a2c00 0x7f45c40a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009230 tx=0x7f45c8005b50 comp rx=0 tx=0).stop 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d810c180 msgr2=0x7f45d819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d810c180 0x7f45d819ce30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f45d000eaa0 tx=0x7f45d000eea0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 shutdown_connections 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f45c40a2c00 0x7f45c40a50c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45d810c180 0x7f45d819ce30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45d8103670 0x7f45d8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 --2- 172.21.15.137:0/188443312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45d8102d50 0x7f45d8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 >> 172.21.15.137:0/188443312 conn(0x7f45d80fc7d0 msgr2=0x7f45d80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 shutdown_connections 2024-09-12T12:13:06.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:06.301+0000 7f45df5bf640 1 -- 172.21.15.137:0/188443312 wait complete. 2024-09-12T12:13:06.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:06 smithi137 ceph-mon[30950]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:13:06.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:06 smithi189 ceph-mon[35475]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:13:06.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:06 smithi171 ceph-mon[35098]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-12T12:13:07.103 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T12:13:07.114 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T12:13:07.115 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T12:13:07.115 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T12:13:07.115 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-12T12:13:07.116 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T12:13:07.116 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:13:07.116 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-12T12:13:07.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 -- 172.21.15.137:0/2821028628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 msgr2=0x7f2ae41031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 --2- 172.21.15.137:0/2821028628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41031d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009900 tx=0x7f2ad402a410 comp rx=0 tx=0).stop 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 -- 172.21.15.137:0/2821028628 shutdown_connections 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 --2- 172.21.15.137:0/2821028628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4103710 0x7f2ae4109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 --2- 172.21.15.137:0/2821028628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41031d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 --2- 172.21.15.137:0/2821028628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ae4101b60 0x7f2ae4101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 -- 172.21.15.137:0/2821028628 >> 172.21.15.137:0/2821028628 conn(0x7f2ae40fd2e0 msgr2=0x7f2ae40ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 -- 172.21.15.137:0/2821028628 shutdown_connections 2024-09-12T12:13:07.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.217+0000 7f2aebc53640 1 -- 172.21.15.137:0/2821028628 wait complete. 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 Processor -- start 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 -- start start 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ae4101b60 0x7f2ae4197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4103710 0x7f2ae419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d880 con 0x7f2ae4102d50 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410d700 con 0x7f2ae4101b60 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aebc53640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae410da00 con 0x7f2ae4103710 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38620/0 (socket says 172.21.15.137:38620) 2024-09-12T12:13:07.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 -- 172.21.15.137:0/101517352 learned_addr learned my addr 172.21.15.137:0/101517352 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2aea1f0640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4103710 0x7f2ae419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 -- 172.21.15.137:0/101517352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4103710 msgr2=0x7f2ae419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4103710 0x7f2ae419f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 -- 172.21.15.137:0/101517352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ae4101b60 msgr2=0x7f2ae4197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae99ef640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ae4101b60 0x7f2ae4197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ae4101b60 0x7f2ae4197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 -- 172.21.15.137:0/101517352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae419fa90 con 0x7f2ae4102d50 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae99ef640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ae4101b60 0x7f2ae4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.218+0000 7f2ae91ee640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41983b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009230 tx=0x7f2ad4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.219+0000 7f2adaffd640 1 -- 172.21.15.137:0/101517352 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad403f030 con 0x7f2ae4102d50 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.219+0000 7f2adaffd640 1 -- 172.21.15.137:0/101517352 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ad4003ce0 con 0x7f2ae4102d50 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.219+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae419fd20 con 0x7f2ae4102d50 2024-09-12T12:13:07.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.219+0000 7f2adaffd640 1 -- 172.21.15.137:0/101517352 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad4035070 con 0x7f2ae4102d50 2024-09-12T12:13:07.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.219+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae41a0240 con 0x7f2ae4102d50 2024-09-12T12:13:07.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.220+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae4108ed0 con 0x7f2ae4102d50 2024-09-12T12:13:07.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.221+0000 7f2adaffd640 1 -- 172.21.15.137:0/101517352 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2ad4042030 con 0x7f2ae4102d50 2024-09-12T12:13:07.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.221+0000 7f2adaffd640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2ad00a2c30 0x7f2ad00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:07.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.221+0000 7f2ae99ef640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2ad00a2c30 0x7f2ad00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:07.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.222+0000 7f2adaffd640 1 -- 172.21.15.137:0/101517352 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f2ad40ef700 con 0x7f2ae4102d50 2024-09-12T12:13:07.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.222+0000 7f2ae99ef640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2ad00a2c30 0x7f2ad00a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0006fd0 tx=0x7f2ae0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:07.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.225+0000 7f2adaffd640 1 -- 172.21.15.137:0/101517352 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ad40bc690 con 0x7f2ae4102d50 2024-09-12T12:13:07.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:07.440+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f2ae4062df0 con 0x7f2ae4102d50 2024-09-12T12:13:08.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.270+0000 7f2adaffd640 1 -- 172.21.15.137:0/101517352 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f2ad40bbf90 con 0x7f2ae4102d50 2024-09-12T12:13:08.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.273+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2ad00a2c30 msgr2=0x7f2ad00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:08.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.273+0000 7f2aebc53640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2ad00a2c30 0x7f2ad00a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0006fd0 tx=0x7f2ae0008040 comp rx=0 tx=0).stop 2024-09-12T12:13:08.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.273+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 msgr2=0x7f2ae41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:08.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.273+0000 7f2aebc53640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41983b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4009230 tx=0x7f2ad4004360 comp rx=0 tx=0).stop 2024-09-12T12:13:08.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 shutdown_connections 2024-09-12T12:13:08.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f2ad00a2c30 0x7f2ad00a50f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:08.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ae4103710 0x7f2ae419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:08.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ae4102d50 0x7f2ae41983b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:08.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 --2- 172.21.15.137:0/101517352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2ae4101b60 0x7f2ae4197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:08.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 >> 172.21.15.137:0/101517352 conn(0x7f2ae40fd2e0 msgr2=0x7f2ae41060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:08.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 shutdown_connections 2024-09-12T12:13:08.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:08.276+0000 7f2aebc53640 1 -- 172.21.15.137:0/101517352 wait complete. 2024-09-12T12:13:08.291 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T12:13:08.320 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T12:13:08.320 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T12:13:08.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:08.321 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-09-12T12:13:08.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:08.341 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link delete ceph-brx 2024-09-12T12:13:08.413 INFO:teuthology.orchestra.run.smithi137.stderr:Cannot find device "ceph-brx" 2024-09-12T12:13:08.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:13:08.415 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T12:13:08.415 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T12:13:08.415 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:08.415 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T12:13:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-mon[30950]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:13:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/101517352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T12:13:08.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ignoring --setuser ceph since I am not root 2024-09-12T12:13:08.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ignoring --setgroup ceph since I am not root 2024-09-12T12:13:08.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:08.428+0000 7f4fc6d9b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:13:08.550 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:08.486+0000 7f4fc6d9b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:13:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-mon[35475]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:13:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/101517352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T12:13:08.591 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: ignoring --setuser ceph since I am not root 2024-09-12T12:13:08.592 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: ignoring --setgroup ceph since I am not root 2024-09-12T12:13:08.592 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-mgr[37288]: -- 172.21.15.189:0/1201216498 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cf01d770e0 con 0x55cf01d55400 2024-09-12T12:13:08.592 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:08.426+0000 7fc87e993040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:13:08.592 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:08.483+0000 7fc87e993040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:13:08.631 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-mon[35098]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-12T12:13:08.631 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/101517352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-12T12:13:08.632 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: ignoring --setuser ceph since I am not root 2024-09-12T12:13:08.632 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: ignoring --setgroup ceph since I am not root 2024-09-12T12:13:08.632 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-mgr[36466]: -- 172.21.15.171:0/3408038059 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4fbc050e0 con 0x55e4fbbe2800 2024-09-12T12:13:08.632 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:08.424+0000 7f8fc6b79040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T12:13:08.632 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:08.481+0000 7f8fc6b79040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T12:13:08.632 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:08.630+0000 7f8fc6b79040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:13:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:08.902 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:08.639+0000 7f4fc6d9b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:13:08.903 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:08.712+0000 7f4fc6d9b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:13:08.903 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:08.872+0000 7f4fc6d9b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:13:08.949 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:08.696+0000 7f8fc6b79040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:13:08.950 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:08.844+0000 7f8fc6b79040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:13:08.984 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:08.632+0000 7fc87e993040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T12:13:08.984 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:08.697+0000 7fc87e993040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T12:13:08.984 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:08.842+0000 7fc87e993040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:13:08.984 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:08.983+0000 7fc87e993040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:13:09.228 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:09.015+0000 7f4fc6d9b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 -- 172.21.15.137:0/1417220577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 msgr2=0x7fe470101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 --2- 172.21.15.137:0/1417220577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 0x7fe470101200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe46002a480 comp rx=0 tx=0).stop 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 -- 172.21.15.137:0/1417220577 shutdown_connections 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 --2- 172.21.15.137:0/1417220577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102ef0 0x7fe47010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 --2- 172.21.15.137:0/1417220577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe470102570 0x7fe4701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 --2- 172.21.15.137:0/1417220577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 0x7fe470101200 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 -- 172.21.15.137:0/1417220577 >> 172.21.15.137:0/1417220577 conn(0x7fe4700fc7d0 msgr2=0x7fe4700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:09.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 -- 172.21.15.137:0/1417220577 shutdown_connections 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.227+0000 7fe474cae640 1 -- 172.21.15.137:0/1417220577 wait complete. 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 Processor -- start 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 -- start start 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 0x7fe470103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102570 0x7fe470103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe470102ef0 0x7fe4701041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe470110f80 con 0x7fe470100e20 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe470110e00 con 0x7fe470102ef0 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe474cae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe470111100 con 0x7fe470102570 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe46dd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102570 0x7fe470103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe46e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 0x7fe470103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:09.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe46dd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102570 0x7fe470103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54670/0 (socket says 172.21.15.137:54670) 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe46e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 0x7fe470103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55272/0 (socket says 172.21.15.137:55272) 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe46dd9b640 1 -- 172.21.15.137:0/2993285886 learned_addr learned my addr 172.21.15.137:0/2993285886 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.228+0000 7fe46ed9d640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe470102ef0 0x7fe4701041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe46dd9b640 1 -- 172.21.15.137:0/2993285886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe470102ef0 msgr2=0x7fe4701041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe46dd9b640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe470102ef0 0x7fe4701041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe46dd9b640 1 -- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 msgr2=0x7fe470103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe46dd9b640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 0x7fe470103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe46dd9b640 1 -- 172.21.15.137:0/2993285886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4701a6990 con 0x7fe470102570 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe46dd9b640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102570 0x7fe470103c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe46400e9f0 tx=0x7fe46400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe45f7fe640 1 -- 172.21.15.137:0/2993285886 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46400cd80 con 0x7fe470102570 2024-09-12T12:13:09.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe45f7fe640 1 -- 172.21.15.137:0/2993285886 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe464010910 con 0x7fe470102570 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe45f7fe640 1 -- 172.21.15.137:0/2993285886 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464018990 con 0x7fe470102570 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4701a6c80 con 0x7fe470102570 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.229+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4701a7170 con 0x7fe470102570 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.230+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe470101200 con 0x7fe470102570 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.231+0000 7fe45f7fe640 1 -- 172.21.15.137:0/2993285886 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fe464010ac0 con 0x7fe470102570 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.232+0000 7fe45f7fe640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 0x7fe4580a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.232+0000 7fe46e59c640 1 -- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 msgr2=0x7fe4580a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1789953576 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.232+0000 7fe45f7fe640 1 -- 172.21.15.137:0/2993285886 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fe4640cf8c0 con 0x7fe470102570 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.232+0000 7fe46e59c640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 0x7fe4580a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:13:09.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.235+0000 7fe45f7fe640 1 -- 172.21.15.137:0/2993285886 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe46409c200 con 0x7fe470102570 2024-09-12T12:13:09.272 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:08.987+0000 7f8fc6b79040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T12:13:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.433+0000 7fe46e59c640 1 -- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 msgr2=0x7fe4580a5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1789953576 2024-09-12T12:13:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.433+0000 7fe46e59c640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 0x7fe4580a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:13:09.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.443+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe4701a73e0 con 0x7fe470102570 2024-09-12T12:13:09.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.444+0000 7fe45f7fe640 1 -- 172.21.15.137:0/2993285886 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7fe4640a0500 con 0x7fe470102570 2024-09-12T12:13:09.444 INFO:teuthology.orchestra.run.smithi137.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T12:13:09.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 msgr2=0x7fe4580a5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:13:09.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 0x7fe4580a5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102570 msgr2=0x7fe470103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102570 0x7fe470103c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe46400e9f0 tx=0x7fe46400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 shutdown_connections 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fe4580a2bd0 0x7fe4580a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe470102ef0 0x7fe4701041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe470102570 0x7fe470103c90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 --2- 172.21.15.137:0/2993285886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe470100e20 0x7fe470103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 >> 172.21.15.137:0/2993285886 conn(0x7fe4700fc7d0 msgr2=0x7fe47006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 shutdown_connections 2024-09-12T12:13:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:09.446+0000 7fe474cae640 1 -- 172.21.15.137:0/2993285886 wait complete. 2024-09-12T12:13:09.542 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:09 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/101517352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T12:13:09.542 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:09 smithi189 ceph-mon[35475]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T12:13:09.542 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:09.296+0000 7fc87e993040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:13:09.542 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:09.356+0000 7fc87e993040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:13:09.542 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:09.541+0000 7fc87e993040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:13:09.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:09 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/101517352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T12:13:09.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:09 smithi137 ceph-mon[30950]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T12:13:09.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:09.344+0000 7f4fc6d9b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:13:09.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:09.406+0000 7f4fc6d9b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:13:09.552 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:09 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/101517352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-12T12:13:09.552 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:09 smithi171 ceph-mon[35098]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T12:13:09.552 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:09.303+0000 7f8fc6b79040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T12:13:09.553 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:09.364+0000 7f8fc6b79040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T12:13:09.553 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:09.551+0000 7f8fc6b79040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:13:09.904 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:09.601+0000 7f4fc6d9b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi137.front.sepia.ceph.com 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T12:13:10.324 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:13:10.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:10.325 DEBUG:teuthology.orchestra.run.smithi137:> ip addr 2024-09-12T12:13:10.341 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T12:13:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T12:13:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T12:13:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7e brd ff:ff:ff:ff:ff:ff 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f0 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7f brd ff:ff:ff:ff:ff:ff 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f1 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:36 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:37 brd ff:ff:ff:ff:ff:ff 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: inet 172.21.15.137/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-12T12:13:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-09-12T12:13:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::ec4:7aff:fe88:8137/64 scope link noprefixroute 2024-09-12T12:13:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-09-12T12:13:10.343 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T12:13:10.343 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:13:10.343 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-09-12T12:13:10.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add name ceph-brx type bridge 2024-09-12T12:13:10.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr flush dev ceph-brx 2024-09-12T12:13:10.344 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set ceph-brx up 2024-09-12T12:13:10.344 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T12:13:10.344 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-09-12T12:13:10.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:13:10.523 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:10 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2993285886' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T12:13:10.523 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:10.514+0000 7f4fc6d9b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:13:10.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:10.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:10.544 DEBUG:teuthology.orchestra.run.smithi137:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T12:13:10.570 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-09-12T12:13:10.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:10.572 DEBUG:teuthology.orchestra.run.smithi137:> route 2024-09-12T12:13:10.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:10 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2993285886' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T12:13:10.591 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:10.419+0000 7fc87e993040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:13:10.591 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:10.486+0000 7fc87e993040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:13:10.638 INFO:teuthology.orchestra.run.smithi137.stdout:Kernel IP routing table 2024-09-12T12:13:10.638 INFO:teuthology.orchestra.run.smithi137.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T12:13:10.638 INFO:teuthology.orchestra.run.smithi137.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-12T12:13:10.638 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-12T12:13:10.638 INFO:teuthology.orchestra.run.smithi137.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T12:13:10.639 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:13:10.639 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-09-12T12:13:10.639 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-12T12:13:10.639 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-12T12:13:10.639 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-12T12:13:10.639 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-09-12T12:13:10.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:10 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2993285886' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T12:13:10.700 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:10.442+0000 7f8fc6b79040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T12:13:10.700 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:10.510+0000 7f8fc6b79040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:13:10.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:13:10.799 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:10.590+0000 7f4fc6d9b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T12:13:10.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:10.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:10.941 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-09-12T12:13:10.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:10.958 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list-id 2024-09-12T12:13:11.015 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:13:11.015 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-09-12T12:13:11.015 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:13:11.015 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T12:13:11.015 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-09-12T12:13:11.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:13:11.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:11.134 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T12:13:11.134 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:13:11.134 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-09-12T12:13:11.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T12:13:11.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T12:13:11.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T12:13:11.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T12:13:11.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T12:13:11.135 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-09-12T12:13:11.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:13:11.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:11.267 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T12:13:11.267 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-09-12T12:13:11.267 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set brx.0 up 2024-09-12T12:13:11.267 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T12:13:11.268 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-09-12T12:13:11.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:13:11.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:11.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:13:11.351 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:11.365 INFO:teuthology.orchestra.run.smithi137.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T12:13:11.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T12:13:11.366 DEBUG:teuthology.orchestra.run.smithi137:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:11.423 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T12:13:11.423 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T12:13:11.423 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/3a65211d-c5a8-489e-9665-4910f9a11529 2024-09-12T12:13:11.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:11.423 DEBUG:teuthology.orchestra.run.smithi137:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/3a65211d-c5a8-489e-9665-4910f9a11529 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T12:13:11.449 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.192+0000 7fc87e993040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:13:11.449 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.306+0000 7fc87e993040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:13:11.478 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:11.356+0000 7f4fc6d9b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:13:11.490 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.228+0000 7f8fc6b79040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T12:13:11.490 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.344+0000 7f8fc6b79040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:13:11.530 INFO:teuthology.orchestra.run.smithi137.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T12:13:11.530 INFO:teuthology.orchestra.run.smithi137.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T12:13:11.530 INFO:teuthology.orchestra.run.smithi137.stdout:invalid new device string format 2024-09-12T12:13:11.660 INFO:teuthology.orchestra.run.smithi137.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T12:13:11.660 INFO:teuthology.orchestra.run.smithi137.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T12:13:11.660 INFO:teuthology.orchestra.run.smithi137.stdout:invalid new device string format 2024-09-12T12:13:11.660 INFO:teuthology.orchestra.run.smithi137.stdout:mount.ceph: resolved to: "172.21.15.137:3300,172.21.15.171:3300,172.21.15.189:3300" 2024-09-12T12:13:11.660 INFO:teuthology.orchestra.run.smithi137.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.137:3300,172.21.15.171:3300,172.21.15.189:3300:/volumes/qa/sv_1/3a65211d-c5a8-489e-9665-4910f9a11529 2024-09-12T12:13:11.660 INFO:teuthology.orchestra.run.smithi137.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=dd66268a-70fe-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T12:13:11.661 INFO:teuthology.orchestra.run.smithi137.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T12:13:11.661 INFO:teuthology.orchestra.run.smithi137.stdout: You just mounted a file system that supports labels which does not 2024-09-12T12:13:11.661 INFO:teuthology.orchestra.run.smithi137.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T12:13:11.662 INFO:teuthology.orchestra.run.smithi137.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T12:13:11.662 INFO:teuthology.orchestra.run.smithi137.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T12:13:11.663 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T12:13:11.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:13:11.663 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:11.727 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.448+0000 7fc87e993040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:13:11.727 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.509+0000 7fc87e993040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:13:11.728 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.659+0000 7fc87e993040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:13:11.728 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.727+0000 7fc87e993040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:13:11.730 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:11.478+0000 7f4fc6d9b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T12:13:11.730 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:11.633+0000 7f4fc6d9b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:13:11.730 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:11.702+0000 7f4fc6d9b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:13:11.731 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T12:13:11.774 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.489+0000 7f8fc6b79040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T12:13:11.774 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.550+0000 7f8fc6b79040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T12:13:11.774 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.705+0000 7f8fc6b79040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:13:11.774 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.774+0000 7f8fc6b79040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:13:11.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T12:13:11.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:11.828 INFO:teuthology.orchestra.run.smithi137.stdout:client15180 2024-09-12T12:13:11.829 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:13:11.829 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:13:11.847 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T12:13:11.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T12:13:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:11.945 INFO:teuthology.orchestra.run.smithi137.stdout:client15180 2024-09-12T12:13:11.946 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T12:13:11.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T12:13:11.994 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:11.859+0000 7f4fc6d9b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T12:13:11.994 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:11.930+0000 7f4fc6d9b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T12:13:11.994 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:11.994+0000 7f4fc6d9b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:13:11.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:12.002 INFO:teuthology.orchestra.run.smithi137.stdout:client15180 2024-09-12T12:13:12.002 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/dd66268a-70fe-11ef-bcea-c7b262605968.client15180/status') 2024-09-12T12:13:12.007 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.786+0000 7fc87e993040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:13:12.007 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:11.943+0000 7fc87e993040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:13:12.008 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:12.007+0000 7fc87e993040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:13:12.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T12:13:12.062 INFO:teuthology.orchestra.run.smithi137.stdout:instance: client.15180 (3)192.168.0.1:0/2340003220 2024-09-12T12:13:12.062 INFO:teuthology.orchestra.run.smithi137.stdout:blocklisted: false 2024-09-12T12:13:12.062 INFO:teuthology.orchestra.run.smithi137.stderr:0+1 records in 2024-09-12T12:13:12.062 INFO:teuthology.orchestra.run.smithi137.stderr:0+1 records out 2024-09-12T12:13:12.062 INFO:teuthology.orchestra.run.smithi137.stderr:70 bytes copied, 2.3775e-05 s, 2.9 MB/s 2024-09-12T12:13:12.063 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.835+0000 7f8fc6b79040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T12:13:12.063 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:11.997+0000 7f8fc6b79040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:13:12.063 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:12.061+0000 7f8fc6b79040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:13:12.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T12:13:12.069 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T12:13:12.078 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T12:13:12.079 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:13:12.079 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-12T12:13:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 -- 172.21.15.137:0/3020697063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464102670 msgr2=0x7ff464104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 --2- 172.21.15.137:0/3020697063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464102670 0x7ff464104a60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff454009900 tx=0x7ff45402a410 comp rx=0 tx=0).stop 2024-09-12T12:13:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 -- 172.21.15.137:0/3020697063 shutdown_connections 2024-09-12T12:13:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 --2- 172.21.15.137:0/3020697063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4641078d0 0x7ff464109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 --2- 172.21.15.137:0/3020697063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff464104fa0 0x7ff464107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 --2- 172.21.15.137:0/3020697063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464102670 0x7ff464104a60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 -- 172.21.15.137:0/3020697063 >> 172.21.15.137:0/3020697063 conn(0x7ff4640fc1b0 msgr2=0x7ff4640fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 -- 172.21.15.137:0/3020697063 shutdown_connections 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.182+0000 7ff469284640 1 -- 172.21.15.137:0/3020697063 wait complete. 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 Processor -- start 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 -- start start 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464102670 0x7ff464197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464104fa0 0x7ff464198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4641078d0 0x7ff46419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410d810 con 0x7ff464104fa0 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410d690 con 0x7ff4641078d0 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff469284640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410d990 con 0x7ff464102670 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff4627fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464104fa0 0x7ff464198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464102670 0x7ff464197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff4627fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464104fa0 0x7ff464198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55294/0 (socket says 172.21.15.137:55294) 2024-09-12T12:13:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff4637fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4641078d0 0x7ff46419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.183+0000 7ff4627fc640 1 -- 172.21.15.137:0/17338032 learned_addr learned my addr 172.21.15.137:0/17338032 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff4627fc640 1 -- 172.21.15.137:0/17338032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464102670 msgr2=0x7ff464197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff4627fc640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464102670 0x7ff464197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff4627fc640 1 -- 172.21.15.137:0/17338032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4641078d0 msgr2=0x7ff46419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff4627fc640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4641078d0 0x7ff46419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff4627fc640 1 -- 172.21.15.137:0/17338032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419fa60 con 0x7ff464104fa0 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff4637fe640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4641078d0 0x7ff46419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff4627fc640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464104fa0 0x7ff464198380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e9c0 tx=0x7ff45800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff44bfff640 1 -- 172.21.15.137:0/17338032 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45800cd80 con 0x7ff464104fa0 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff44bfff640 1 -- 172.21.15.137:0/17338032 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff458005500 con 0x7ff464104fa0 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46419fd50 con 0x7ff464104fa0 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff44bfff640 1 -- 172.21.15.137:0/17338032 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458010690 con 0x7ff464104fa0 2024-09-12T12:13:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.184+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a0240 con 0x7ff464104fa0 2024-09-12T12:13:12.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.185+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4640fdca0 con 0x7ff464104fa0 2024-09-12T12:13:12.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.187+0000 7ff44bfff640 1 -- 172.21.15.137:0/17338032 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7ff4580046c0 con 0x7ff464104fa0 2024-09-12T12:13:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.187+0000 7ff44bfff640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 0x7ff44c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.187+0000 7ff462ffd640 1 -- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 msgr2=0x7ff44c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1789953576 2024-09-12T12:13:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.187+0000 7ff44bfff640 1 -- 172.21.15.137:0/17338032 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7ff4580cf600 con 0x7ff464104fa0 2024-09-12T12:13:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.187+0000 7ff462ffd640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 0x7ff44c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:13:12.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.195+0000 7ff44bfff640 1 -- 172.21.15.137:0/17338032 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4580541b0 con 0x7ff464104fa0 2024-09-12T12:13:12.276 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:12.074+0000 7fc87e993040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:13:12.276 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:12.155+0000 7fc87e993040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:13:12.276 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:12.217+0000 7fc87e993040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:13:12.277 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:12.276+0000 7fc87e993040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:13:12.299 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:12.164+0000 7f4fc6d9b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T12:13:12.299 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:12.235+0000 7f4fc6d9b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T12:13:12.334 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:12.128+0000 7f8fc6b79040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:13:12.335 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:12.213+0000 7f8fc6b79040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:13:12.335 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:12.275+0000 7f8fc6b79040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:13:12.335 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:12.334+0000 7f8fc6b79040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:13:12.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.388+0000 7ff462ffd640 1 -- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 msgr2=0x7ff44c0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1789953576 2024-09-12T12:13:12.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.388+0000 7ff462ffd640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 0x7ff44c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T12:13:12.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.404+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7ff46402d7c0 con 0x7ff464104fa0 2024-09-12T12:13:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.456+0000 7ff44bfff640 1 -- 172.21.15.137:0/17338032 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7ff45809c590 con 0x7ff464104fa0 2024-09-12T12:13:12.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 msgr2=0x7ff44c0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 0x7ff44c0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464104fa0 msgr2=0x7ff464198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464104fa0 0x7ff464198380 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e9c0 tx=0x7ff45800ee80 comp rx=0 tx=0).stop 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 shutdown_connections 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7ff44c0a2c80 0x7ff44c0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4641078d0 0x7ff46419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff464104fa0 0x7ff464198380 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 --2- 172.21.15.137:0/17338032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff464102670 0x7ff464197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 >> 172.21.15.137:0/17338032 conn(0x7ff4640fc1b0 msgr2=0x7ff4641030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 shutdown_connections 2024-09-12T12:13:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.459+0000 7ff469284640 1 -- 172.21.15.137:0/17338032 wait complete. 2024-09-12T12:13:12.473 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T12:13:12.484 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T12:13:12.492 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T12:13:12.492 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:13:12.492 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-12T12:13:12.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:12 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/17338032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T12:13:12.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 -- 172.21.15.137:0/2753280918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20104ef0 msgr2=0x7f6f201072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 --2- 172.21.15.137:0/2753280918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20104ef0 0x7f6f201072e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009900 tx=0x7f6f1402a410 comp rx=0 tx=0).stop 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 -- 172.21.15.137:0/2753280918 shutdown_connections 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 --2- 172.21.15.137:0/2753280918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f20107820 0x7f6f20109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 --2- 172.21.15.137:0/2753280918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20104ef0 0x7f6f201072e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 --2- 172.21.15.137:0/2753280918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 0x7f6f201049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 -- 172.21.15.137:0/2753280918 >> 172.21.15.137:0/2753280918 conn(0x7f6f200fc1e0 msgr2=0x7f6f200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 -- 172.21.15.137:0/2753280918 shutdown_connections 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 -- 172.21.15.137:0/2753280918 wait complete. 2024-09-12T12:13:12.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 Processor -- start 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.597+0000 7f6f262b7640 1 -- start start 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f262b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 0x7f6f2019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f262b7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f20104ef0 0x7f6f2019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f262b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20107820 0x7f6f201a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f262b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010d800 con 0x7f6f20107820 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f262b7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010d680 con 0x7f6f201025c0 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f262b7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2010d980 con 0x7f6f20104ef0 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 0x7f6f2019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f24854640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20107820 0x7f6f201a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 0x7f6f2019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:51262/0 (socket says 172.21.15.137:51262) 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f24854640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20107820 0x7f6f201a1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55316/0 (socket says 172.21.15.137:55316) 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 -- 172.21.15.137:0/3447129869 learned_addr learned my addr 172.21.15.137:0/3447129869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1f7fe640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f20104ef0 0x7f6f2019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 -- 172.21.15.137:0/3447129869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f20104ef0 msgr2=0x7f6f2019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f20104ef0 0x7f6f2019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 -- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20107820 msgr2=0x7f6f201a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20107820 0x7f6f201a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 -- 172.21.15.137:0/3447129869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f201a1ca0 con 0x7f6f201025c0 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f24854640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20107820 0x7f6f201a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1ffff640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 0x7f6f2019a080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000e9f0 tx=0x7f6f1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.598+0000 7f6f1d7fa640 1 -- 172.21.15.137:0/3447129869 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000cd80 con 0x7f6f201025c0 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.599+0000 7f6f1d7fa640 1 -- 172.21.15.137:0/3447129869 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f10010910 con 0x7f6f201025c0 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.599+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f201a1f90 con 0x7f6f201025c0 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.599+0000 7f6f1d7fa640 1 -- 172.21.15.137:0/3447129869 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10018990 con 0x7f6f201025c0 2024-09-12T12:13:12.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.599+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201a23f0 con 0x7f6f201025c0 2024-09-12T12:13:12.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.600+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f20109390 con 0x7f6f201025c0 2024-09-12T12:13:12.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.601+0000 7f6f1d7fa640 1 -- 172.21.15.137:0/3447129869 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f6f10010ac0 con 0x7f6f201025c0 2024-09-12T12:13:12.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.601+0000 7f6f1d7fa640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6f080a2c80 0x7f6f080a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.601+0000 7f6f1f7fe640 1 -- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6f080a2c80 msgr2=0x7f6f080a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1789953576 2024-09-12T12:13:12.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.601+0000 7f6f1d7fa640 1 -- 172.21.15.137:0/3447129869 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f6f100cf380 con 0x7f6f201025c0 2024-09-12T12:13:12.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.601+0000 7f6f1f7fe640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6f080a2c80 0x7f6f080a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:13:12.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.605+0000 7f6f1d7fa640 1 -- 172.21.15.137:0/3447129869 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f1009bca0 con 0x7f6f201025c0 2024-09-12T12:13:12.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:12 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/17338032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T12:13:12.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.777+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f6f20062f00 con 0x7f6f201025c0 2024-09-12T12:13:12.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.782+0000 7f6f1d7fa640 1 -- 172.21.15.137:0/3447129869 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7f6f1009ffa0 con 0x7f6f201025c0 2024-09-12T12:13:12.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6f080a2c80 msgr2=0x7f6f080a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:13:12.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6f080a2c80 0x7f6f080a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 msgr2=0x7f6f2019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 0x7f6f2019a080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000e9f0 tx=0x7f6f1000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 shutdown_connections 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f6f080a2c80 0x7f6f080a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f20107820 0x7f6f201a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f20104ef0 0x7f6f2019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 --2- 172.21.15.137:0/3447129869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f201025c0 0x7f6f2019a080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 >> 172.21.15.137:0/3447129869 conn(0x7f6f200fc1e0 msgr2=0x7f6f20101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 shutdown_connections 2024-09-12T12:13:12.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.785+0000 7f6f262b7640 1 -- 172.21.15.137:0/3447129869 wait complete. 2024-09-12T12:13:12.795 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:12 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/17338032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T12:13:12.795 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:12.305+0000 7f4fc6d9b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T12:13:12.795 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:12.390+0000 7f4fc6d9b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T12:13:12.795 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:12.450+0000 7f4fc6d9b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T12:13:12.795 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:12.517+0000 7f4fc6d9b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T12:13:12.796 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-12T12:13:12.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.897+0000 7f6761dfd640 1 -- 172.21.15.137:0/289965446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c101c30 msgr2=0x7f675c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.897+0000 7f6761dfd640 1 --2- 172.21.15.137:0/289965446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c101c30 0x7f675c1020b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6750009900 tx=0x7f675002a410 comp rx=0 tx=0).stop 2024-09-12T12:13:12.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.897+0000 7f6761dfd640 1 -- 172.21.15.137:0/289965446 shutdown_connections 2024-09-12T12:13:12.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.897+0000 7f6761dfd640 1 --2- 172.21.15.137:0/289965446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f675c1025f0 0x7f675c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.897+0000 7f6761dfd640 1 --2- 172.21.15.137:0/289965446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c101c30 0x7f675c1020b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.897+0000 7f6761dfd640 1 --2- 172.21.15.137:0/289965446 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f675c100a40 0x7f675c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.897+0000 7f6761dfd640 1 -- 172.21.15.137:0/289965446 >> 172.21.15.137:0/289965446 conn(0x7f675c0fc1e0 msgr2=0x7f675c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 -- 172.21.15.137:0/289965446 shutdown_connections 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 -- 172.21.15.137:0/289965446 wait complete. 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 Processor -- start 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 -- start start 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c100a40 0x7f675c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f675c101c30 0x7f675c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f675c1025f0 0x7f675c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10d870 con 0x7f675c100a40 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10d6f0 con 0x7f675c1025f0 2024-09-12T12:13:12.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f6761dfd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f675c10d9f0 con 0x7f675c101c30 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.898+0000 7f675b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c100a40 0x7f675c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c100a40 0x7f675c197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55340/0 (socket says 172.21.15.137:55340) 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 -- 172.21.15.137:0/2015623787 learned_addr learned my addr 172.21.15.137:0/2015623787 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675affd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f675c101c30 0x7f675c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675bfff640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f675c1025f0 0x7f675c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 -- 172.21.15.137:0/2015623787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f675c101c30 msgr2=0x7f675c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f675c101c30 0x7f675c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 -- 172.21.15.137:0/2015623787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f675c1025f0 msgr2=0x7f675c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f675c1025f0 0x7f675c19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:12.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 -- 172.21.15.137:0/2015623787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c19fa00 con 0x7f675c100a40 2024-09-12T12:13:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f675b7fe640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c100a40 0x7f675c197de0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f674c00d950 tx=0x7f674c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f6758ff9640 1 -- 172.21.15.137:0/2015623787 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c017020 con 0x7f675c100a40 2024-09-12T12:13:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f6758ff9640 1 -- 172.21.15.137:0/2015623787 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f674c009c30 con 0x7f675c100a40 2024-09-12T12:13:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c19fcf0 con 0x7f675c100a40 2024-09-12T12:13:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f6758ff9640 1 -- 172.21.15.137:0/2015623787 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c00f740 con 0x7f675c100a40 2024-09-12T12:13:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.899+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1a01b0 con 0x7f675c100a40 2024-09-12T12:13:12.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.900+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c107d60 con 0x7f675c100a40 2024-09-12T12:13:12.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.901+0000 7f6758ff9640 1 -- 172.21.15.137:0/2015623787 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f674c01a020 con 0x7f675c100a40 2024-09-12T12:13:12.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.902+0000 7f6758ff9640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67440a2d00 0x7f67440a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:12.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.902+0000 7f675affd640 1 -- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67440a2d00 msgr2=0x7f67440a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1789953576 2024-09-12T12:13:12.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.902+0000 7f6758ff9640 1 -- 172.21.15.137:0/2015623787 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7f674c0ce970 con 0x7f675c100a40 2024-09-12T12:13:12.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.902+0000 7f675affd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67440a2d00 0x7f67440a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:13:12.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:12.907+0000 7f6758ff9640 1 -- 172.21.15.137:0/2015623787 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f674c021070 con 0x7f675c100a40 2024-09-12T12:13:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.078+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f675c062ec0 con 0x7f675c100a40 2024-09-12T12:13:13.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.081+0000 7f6758ff9640 1 -- 172.21.15.137:0/2015623787 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f674c009de0 con 0x7f675c100a40 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67440a2d00 msgr2=0x7f67440a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67440a2d00 0x7f67440a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c100a40 msgr2=0x7f675c197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c100a40 0x7f675c197de0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f674c00d950 tx=0x7f674c00de10 comp rx=0 tx=0).stop 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 shutdown_connections 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7f67440a2d00 0x7f67440a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f675c1025f0 0x7f675c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f675c101c30 0x7f675c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 --2- 172.21.15.137:0/2015623787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f675c100a40 0x7f675c197de0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 >> 172.21.15.137:0/2015623787 conn(0x7f675c0fc1e0 msgr2=0x7f675c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 shutdown_connections 2024-09-12T12:13:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.084+0000 7f6761dfd640 1 -- 172.21.15.137:0/2015623787 wait complete. 2024-09-12T12:13:13.095 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-12T12:13:13.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 -- 172.21.15.137:0/3442174480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920102600 msgr2=0x7fa920108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 --2- 172.21.15.137:0/3442174480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920102600 0x7fa920108e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa918009ca0 tx=0x7fa91802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 -- 172.21.15.137:0/3442174480 shutdown_connections 2024-09-12T12:13:13.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 --2- 172.21.15.137:0/3442174480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920102600 0x7fa920108e90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 --2- 172.21.15.137:0/3442174480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa920101c40 0x7fa9201020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 --2- 172.21.15.137:0/3442174480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa920100a50 0x7fa920100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 -- 172.21.15.137:0/3442174480 >> 172.21.15.137:0/3442174480 conn(0x7fa9200fc1b0 msgr2=0x7fa9200fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:13.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 -- 172.21.15.137:0/3442174480 shutdown_connections 2024-09-12T12:13:13.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.197+0000 7fa925f52640 1 -- 172.21.15.137:0/3442174480 wait complete. 2024-09-12T12:13:13.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.198+0000 7fa925f52640 1 Processor -- start 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.198+0000 7fa925f52640 1 -- start start 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa925f52640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920100a50 0x7fa920196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa925f52640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa920101c40 0x7fa9201970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa925f52640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa920102600 0x7fa92019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa925f52640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c700 con 0x7fa920100a50 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa925f52640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c580 con 0x7fa920102600 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa925f52640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa92010c880 con 0x7fa920101c40 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920100a50 0x7fa920196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa920101c40 0x7fa9201970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:13.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920100a50 0x7fa920196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55368/0 (socket says 172.21.15.137:55368) 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 -- 172.21.15.137:0/1722214074 learned_addr learned my addr 172.21.15.137:0/1722214074 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91ffff640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa920102600 0x7fa92019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 -- 172.21.15.137:0/1722214074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa920101c40 msgr2=0x7fa9201970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa920101c40 0x7fa9201970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 -- 172.21.15.137:0/1722214074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa920102600 msgr2=0x7fa92019e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa920102600 0x7fa92019e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 -- 172.21.15.137:0/1722214074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92019e7c0 con 0x7fa920100a50 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91f7fe640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920100a50 0x7fa920196ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c970 tx=0x7fa91000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910007c10 con 0x7fa920100a50 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910007db0 con 0x7fa920100a50 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91000f940 con 0x7fa920100a50 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.199+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa92019eab0 con 0x7fa920100a50 2024-09-12T12:13:13.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.200+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa92019efb0 con 0x7fa920100a50 2024-09-12T12:13:13.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.201+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa920107ca0 con 0x7fa920100a50 2024-09-12T12:13:13.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.202+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fa91000fae0 con 0x7fa920100a50 2024-09-12T12:13:13.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.203+0000 7fa91cff9640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa9080a2c80 0x7fa9080a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:13.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.203+0000 7fa91effd640 1 -- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa9080a2c80 msgr2=0x7fa9080a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1789953576 2024-09-12T12:13:13.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.203+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8182+0+0 (secure 0 0 0) 0x7fa9100cf3b0 con 0x7fa920100a50 2024-09-12T12:13:13.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.203+0000 7fa91effd640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa9080a2c80 0x7fa9080a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T12:13:13.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.207+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa91009c3b0 con 0x7fa920100a50 2024-09-12T12:13:13.340 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z[37284]: 2024-09-12T12:13:13.071+0000 7fc87e993040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:13:13.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.363+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7fa910098320 con 0x7fa920100a50 2024-09-12T12:13:13.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.363+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa9080a2c80 msgr2=0x7fa9080a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:13:13.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.363+0000 7fa91cff9640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa9080a2c80 0x7fa9080a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.390+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fa92002d7c0 con 0x7fa920100a50 2024-09-12T12:13:13.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.394+0000 7fa91cff9640 1 -- 172.21.15.137:0/1722214074 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7fa910064e20 con 0x7fa920100a50 2024-09-12T12:13:13.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920100a50 msgr2=0x7fa920196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920100a50 0x7fa920196ba0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c970 tx=0x7fa91000ce30 comp rx=0 tx=0).stop 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 shutdown_connections 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:6800/1789953576,v1:172.21.15.137:6801/1789953576] conn(0x7fa9080a2c80 0x7fa9080a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa920102600 0x7fa92019e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa920101c40 0x7fa9201970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 --2- 172.21.15.137:0/1722214074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa920100a50 0x7fa920196ba0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 >> 172.21.15.137:0/1722214074 conn(0x7fa9200fc1b0 msgr2=0x7fa920104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 shutdown_connections 2024-09-12T12:13:13.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.397+0000 7fa925f52640 1 -- 172.21.15.137:0/1722214074 wait complete. 2024-09-12T12:13:13.412 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-12T12:13:13.449 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y[36461]: 2024-09-12T12:13:13.140+0000 7f8fc6b79040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.555+0000 7f2f577f8640 1 -- 172.21.15.137:0/2673789788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109350 msgr2=0x7f2f501097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.555+0000 7f2f577f8640 1 --2- 172.21.15.137:0/2673789788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109350 0x7f2f501097d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009930 tx=0x7f2f4002a470 comp rx=0 tx=0).stop 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.555+0000 7f2f577f8640 1 -- 172.21.15.137:0/2673789788 shutdown_connections 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.555+0000 7f2f577f8640 1 --2- 172.21.15.137:0/2673789788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f50109d10 0x7f2f501105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.555+0000 7f2f577f8640 1 --2- 172.21.15.137:0/2673789788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109350 0x7f2f501097d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.555+0000 7f2f577f8640 1 --2- 172.21.15.137:0/2673789788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 0x7f2f501088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.555+0000 7f2f577f8640 1 -- 172.21.15.137:0/2673789788 >> 172.21.15.137:0/2673789788 conn(0x7f2f5006b6b0 msgr2=0x7f2f50079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:13.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.556+0000 7f2f577f8640 1 -- 172.21.15.137:0/2673789788 shutdown_connections 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.556+0000 7f2f577f8640 1 -- 172.21.15.137:0/2673789788 wait complete. 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.556+0000 7f2f577f8640 1 Processor -- start 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.556+0000 7f2f577f8640 1 -- start start 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f577f8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 0x7f2f5019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f577f8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f50109350 0x7f2f5019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f577f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109d10 0x7f2f501a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f577f8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50113e60 con 0x7f2f50109d10 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f577f8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50113ce0 con 0x7f2f50109350 2024-09-12T12:13:13.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f577f8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50113fe0 con 0x7f2f501084c0 2024-09-12T12:13:13.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 0x7f2f5019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:13.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f54d93640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f50109350 0x7f2f5019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 0x7f2f5019e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54852/0 (socket says 172.21.15.137:54852) 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 -- 172.21.15.137:0/426677114 learned_addr learned my addr 172.21.15.137:0/426677114 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55d95640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109d10 0x7f2f501a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 -- 172.21.15.137:0/426677114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f50109350 msgr2=0x7f2f5019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f50109350 0x7f2f5019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 -- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109d10 msgr2=0x7f2f501a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109d10 0x7f2f501a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 -- 172.21.15.137:0/426677114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f501a6080 con 0x7f2f501084c0 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f55594640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 0x7f2f5019e460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00e9c0 tx=0x7f2f4c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.557+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c00cd80 con 0x7f2f501084c0 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.558+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f4c0035f0 con 0x7f2f501084c0 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.558+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c010690 con 0x7f2f501084c0 2024-09-12T12:13:13.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.558+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f501a6370 con 0x7f2f501084c0 2024-09-12T12:13:13.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.558+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f501a6840 con 0x7f2f501084c0 2024-09-12T12:13:13.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.559+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f5010f550 con 0x7f2f501084c0 2024-09-12T12:13:13.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.561+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f2f4c0037a0 con 0x7f2f501084c0 2024-09-12T12:13:13.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.562+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f2f4c0cf0b0 con 0x7f2f501084c0 2024-09-12T12:13:13.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:13.567+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f4c09c2e0 con 0x7f2f501084c0 2024-09-12T12:13:13.655 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.347+0000 7f4fc6d9b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/17338032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='client.? ' entity='client.admin' 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: Standby manager daemon z restarted 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: Standby manager daemon z started 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2015623787' entity='client.admin' 2024-09-12T12:13:13.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: Standby manager daemon y restarted 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: Standby manager daemon y started 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: Active manager daemon x restarted 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: Activating manager daemon x 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: mgrmap e20: x(active, starting, since 0.0156691s), standbys: y, z 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:13:13.657 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:13:13.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:13:13.659 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: Manager daemon x is now available 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1722214074' entity='client.admin' 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T12:13:13.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T12:13:13.758 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/17338032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T12:13:13.767 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:13:13.767 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='client.? ' entity='client.admin' 2024-09-12T12:13:13.767 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: Standby manager daemon z restarted 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: Standby manager daemon z started 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2015623787' entity='client.admin' 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: Standby manager daemon y restarted 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: Standby manager daemon y started 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T12:13:13.768 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: Active manager daemon x restarted 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: Activating manager daemon x 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: mgrmap e20: x(active, starting, since 0.0156691s), standbys: y, z 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T12:13:13.769 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:13:13.770 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:13:13.771 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T12:13:13.772 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T12:13:13.772 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T12:13:13.772 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: Manager daemon x is now available 2024-09-12T12:13:13.772 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1722214074' entity='client.admin' 2024-09-12T12:13:13.772 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:13:13.772 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T12:13:13.772 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:13 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T12:13:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/17338032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T12:13:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:13:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='client.? ' entity='client.admin' 2024-09-12T12:13:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: Standby manager daemon z restarted 2024-09-12T12:13:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: Standby manager daemon z started 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.189:0/3619843415' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2015623787' entity='client.admin' 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: Standby manager daemon y restarted 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: Standby manager daemon y started 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.? 172.21.15.171:0/2769289326' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: Active manager daemon x restarted 2024-09-12T12:13:13.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: Activating manager daemon x 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: mgrmap e20: x(active, starting, since 0.0156691s), standbys: y, z 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:13:13.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T12:13:13.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: Manager daemon x is now available 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1722214074' entity='client.admin' 2024-09-12T12:13:13.845 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:13:13.846 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T12:13:13.846 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:13 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T12:13:14.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.654+0000 7f4f2cb6a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.654+0000 7f4f2cb6a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.654+0000 7f4f2cb6a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.654+0000 7f4f2cb6a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.654+0000 7f4f2cb6a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.659+0000 7f4f2fb70640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.659+0000 7f4f2fb70640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.659+0000 7f4f2fb70640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.659+0000 7f4f2fb70640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.050 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:13.659+0000 7f4f2fb70640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:13:14.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.366+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f2f4c097930 con 0x7f2f501084c0 2024-09-12T12:13:14.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.366+0000 7f2f467fc640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2f3c0a9930 0x7f2f3c0abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.366+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2f4c00be50 con 0x7f2f3c0a9930 2024-09-12T12:13:14.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.367+0000 7f2f54d93640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2f3c0a9930 0x7f2f3c0abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.371+0000 7f2f54d93640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2f3c0a9930 0x7f2f3c0abd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2f40004b40 tx=0x7f2f40005df0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:14.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.434+0000 7f2f467fc640 1 -- 172.21.15.137:0/426677114 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2f4c00be50 con 0x7f2f3c0a9930 2024-09-12T12:13:14.434 INFO:teuthology.orchestra.run.smithi137.stdout:Schedule set for path / 2024-09-12T12:13:14.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2f3c0a9930 msgr2=0x7f2f3c0abd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2f3c0a9930 0x7f2f3c0abd20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2f40004b40 tx=0x7f2f40005df0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 msgr2=0x7f2f5019e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 0x7f2f5019e460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c00e9c0 tx=0x7f2f4c00ee80 comp rx=0 tx=0).stop 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 shutdown_connections 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2f3c0a9930 0x7f2f3c0abd20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f50109d10 0x7f2f501a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f50109350 0x7f2f5019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 --2- 172.21.15.137:0/426677114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2f501084c0 0x7f2f5019e460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 >> 172.21.15.137:0/426677114 conn(0x7f2f5006b6b0 msgr2=0x7f2f5010c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 shutdown_connections 2024-09-12T12:13:14.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.436+0000 7f2f577f8640 1 -- 172.21.15.137:0/426677114 wait complete. 2024-09-12T12:13:14.448 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-12T12:13:14.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 -- 172.21.15.137:0/1299905886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d01025f0 msgr2=0x7f07d0108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 --2- 172.21.15.137:0/1299905886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d01025f0 0x7f07d0108e80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f07c8009ca0 tx=0x7f07c802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 -- 172.21.15.137:0/1299905886 shutdown_connections 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 --2- 172.21.15.137:0/1299905886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d01025f0 0x7f07d0108e80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 --2- 172.21.15.137:0/1299905886 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d0101c30 0x7f07d01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 --2- 172.21.15.137:0/1299905886 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d0100a40 0x7f07d0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 -- 172.21.15.137:0/1299905886 >> 172.21.15.137:0/1299905886 conn(0x7f07d00fc1e0 msgr2=0x7f07d00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 -- 172.21.15.137:0/1299905886 shutdown_connections 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 -- 172.21.15.137:0/1299905886 wait complete. 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.553+0000 7f07d5eac640 1 Processor -- start 2024-09-12T12:13:14.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 -- start start 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d0100a40 0x7f07d006a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d0101c30 0x7f07d019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d01025f0 0x7f07d019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f07d010c6a0 con 0x7f07d0100a40 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f07d010c520 con 0x7f07d01025f0 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07d010c820 con 0x7f07d0101c30 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d0100a40 0x7f07d006a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d0100a40 0x7f07d006a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55464/0 (socket says 172.21.15.137:55464) 2024-09-12T12:13:14.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 -- 172.21.15.137:0/3400400242 learned_addr learned my addr 172.21.15.137:0/3400400242 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:14.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cffff640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d01025f0 0x7f07d019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07ceffd640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d0101c30 0x7f07d019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 -- 172.21.15.137:0/3400400242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d0101c30 msgr2=0x7f07d019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d0101c30 0x7f07d019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 -- 172.21.15.137:0/3400400242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d01025f0 msgr2=0x7f07d019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d01025f0 0x7f07d019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 -- 172.21.15.137:0/3400400242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d019f9e0 con 0x7f07d0100a40 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07ceffd640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d0101c30 0x7f07d019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07cf7fe640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d0100a40 0x7f07d006a410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f07c000d770 tx=0x7f07c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07ccff9640 1 -- 172.21.15.137:0/3400400242 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0017070 con 0x7f07d0100a40 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d019fc70 con 0x7f07d0100a40 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07ccff9640 1 -- 172.21.15.137:0/3400400242 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c0009c30 con 0x7f07d0100a40 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.554+0000 7f07ccff9640 1 -- 172.21.15.137:0/3400400242 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0003dd0 con 0x7f07d0100a40 2024-09-12T12:13:14.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.555+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d01a01c0 con 0x7f07d0100a40 2024-09-12T12:13:14.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.556+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d0107de0 con 0x7f07d0100a40 2024-09-12T12:13:14.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.557+0000 7f07ccff9640 1 -- 172.21.15.137:0/3400400242 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f07c0003f70 con 0x7f07d0100a40 2024-09-12T12:13:14.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.557+0000 7f07ccff9640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f07b80a2a40 0x7f07b80a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.558+0000 7f07ceffd640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f07b80a2a40 0x7f07b80a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.558+0000 7f07ccff9640 1 -- 172.21.15.137:0/3400400242 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f07c00ce6d0 con 0x7f07d0100a40 2024-09-12T12:13:14.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.558+0000 7f07ceffd640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f07b80a2a40 0x7f07b80a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07d006a180 tx=0x7f07c400b5f0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:14.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.562+0000 7f07ccff9640 1 -- 172.21.15.137:0/3400400242 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07c009b850 con 0x7f07d0100a40 2024-09-12T12:13:14.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.761+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f07d00626d0 con 0x7f07b80a2a40 2024-09-12T12:13:14.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.817+0000 7f07ccff9640 1 -- 172.21.15.137:0/3400400242 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f07d00626d0 con 0x7f07b80a2a40 2024-09-12T12:13:14.818 INFO:teuthology.orchestra.run.smithi137.stdout:Retention added to path / 2024-09-12T12:13:14.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f07b80a2a40 msgr2=0x7f07b80a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f07b80a2a40 0x7f07b80a4f00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07d006a180 tx=0x7f07c400b5f0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d0100a40 msgr2=0x7f07d006a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d0100a40 0x7f07d006a410 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f07c000d770 tx=0x7f07c000dc30 comp rx=0 tx=0).stop 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 shutdown_connections 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f07b80a2a40 0x7f07b80a4f00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d01025f0 0x7f07d019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d0101c30 0x7f07d019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 --2- 172.21.15.137:0/3400400242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07d0100a40 0x7f07d006a410 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 >> 172.21.15.137:0/3400400242 conn(0x7f07d00fc1e0 msgr2=0x7f07d0104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 shutdown_connections 2024-09-12T12:13:14.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.820+0000 7f07d5eac640 1 -- 172.21.15.137:0/3400400242 wait complete. 2024-09-12T12:13:14.831 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 -- 172.21.15.137:0/2860601438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc102d50 msgr2=0x7f0cfc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 --2- 172.21.15.137:0/2860601438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc102d50 0x7f0cfc1031d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009900 tx=0x7f0cec02a410 comp rx=0 tx=0).stop 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 -- 172.21.15.137:0/2860601438 shutdown_connections 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 --2- 172.21.15.137:0/2860601438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cfc103710 0x7f0cfc109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 --2- 172.21.15.137:0/2860601438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc102d50 0x7f0cfc1031d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 --2- 172.21.15.137:0/2860601438 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cfc101b60 0x7f0cfc101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 -- 172.21.15.137:0/2860601438 >> 172.21.15.137:0/2860601438 conn(0x7f0cfc0fd2e0 msgr2=0x7f0cfc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 -- 172.21.15.137:0/2860601438 shutdown_connections 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.934+0000 7f0d0425e640 1 -- 172.21.15.137:0/2860601438 wait complete. 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 Processor -- start 2024-09-12T12:13:14.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 -- start start 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc101b60 0x7f0cfc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cfc102d50 0x7f0cfc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cfc103710 0x7f0cfc19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc10d7e0 con 0x7f0cfc101b60 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc10d660 con 0x7f0cfc103710 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d0425e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc10d960 con 0x7f0cfc102d50 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc101b60 0x7f0cfc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc101b60 0x7f0cfc197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55480/0 (socket says 172.21.15.137:55480) 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 -- 172.21.15.137:0/352834793 learned_addr learned my addr 172.21.15.137:0/352834793 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 -- 172.21.15.137:0/352834793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cfc102d50 msgr2=0x7f0cfc198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d017f9640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cfc102d50 0x7f0cfc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d027fb640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cfc103710 0x7f0cfc19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cfc102d50 0x7f0cfc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 -- 172.21.15.137:0/352834793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cfc103710 msgr2=0x7f0cfc19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cfc103710 0x7f0cfc19f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.935+0000 7f0d01ffa640 1 -- 172.21.15.137:0/352834793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc19f970 con 0x7f0cfc101b60 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.936+0000 7f0d01ffa640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc101b60 0x7f0cfc197d50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800c970 tx=0x7f0cf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.936+0000 7f0cf2ffd640 1 -- 172.21.15.137:0/352834793 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8007c10 con 0x7f0cfc101b60 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.936+0000 7f0cf2ffd640 1 -- 172.21.15.137:0/352834793 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8007db0 con 0x7f0cfc101b60 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.936+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc19fc60 con 0x7f0cfc101b60 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.936+0000 7f0cf2ffd640 1 -- 172.21.15.137:0/352834793 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf800f940 con 0x7f0cfc101b60 2024-09-12T12:13:14.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.936+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1a01b0 con 0x7f0cfc101b60 2024-09-12T12:13:14.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.937+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc108e80 con 0x7f0cfc101b60 2024-09-12T12:13:14.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.938+0000 7f0cf2ffd640 1 -- 172.21.15.137:0/352834793 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f0cf800fae0 con 0x7f0cfc101b60 2024-09-12T12:13:14.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.938+0000 7f0cf2ffd640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ce80a2ac0 0x7f0ce80a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:14.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.938+0000 7f0d017f9640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ce80a2ac0 0x7f0ce80a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:14.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.939+0000 7f0cf2ffd640 1 -- 172.21.15.137:0/352834793 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f0cf80cf0f0 con 0x7f0cfc101b60 2024-09-12T12:13:14.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.939+0000 7f0d017f9640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ce80a2ac0 0x7f0ce80a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0cec031610 tx=0x7f0cec000bc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:14.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:14.942+0000 7f0cf2ffd640 1 -- 172.21.15.137:0/352834793 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cf809c300 con 0x7f0cfc101b60 2024-09-12T12:13:15.121 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:15.007+0000 7f4f1a646640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:12:13:15] ENGINE Error in HTTPServer.serve 2024-09-12T12:13:15.121 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Traceback (most recent call last): 2024-09-12T12:13:15.121 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:13:15.121 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._connections.run(self.expiration_interval) 2024-09-12T12:13:15.121 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:13:15.121 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._run(expiration_interval) 2024-09-12T12:13:15.121 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: s = self.context.wrap_socket( 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: return self.sslsocket_class._create( 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self.do_handshake() 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self._sslobj.do_handshake() 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:13:15.122 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:13:15.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.135+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0cfc062630 con 0x7f0ce80a2ac0 2024-09-12T12:13:15.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.141+0000 7f0cf2ffd640 1 -- 172.21.15.137:0/352834793 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f0cfc062630 con 0x7f0ce80a2ac0 2024-09-12T12:13:15.142 INFO:teuthology.orchestra.run.smithi137.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T12:13:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-12T12:13:15.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ce80a2ac0 msgr2=0x7f0ce80a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:15.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ce80a2ac0 0x7f0ce80a4f80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0cec031610 tx=0x7f0cec000bc0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc101b60 msgr2=0x7f0cfc197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:15.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc101b60 0x7f0cfc197d50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800c970 tx=0x7f0cf800ce30 comp rx=0 tx=0).stop 2024-09-12T12:13:15.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 shutdown_connections 2024-09-12T12:13:15.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ce80a2ac0 0x7f0ce80a4f80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cfc103710 0x7f0cfc19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cfc102d50 0x7f0cfc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 --2- 172.21.15.137:0/352834793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cfc101b60 0x7f0cfc197d50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.143+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 >> 172.21.15.137:0/352834793 conn(0x7f0cfc0fd2e0 msgr2=0x7f0cfc106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:15.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.144+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 shutdown_connections 2024-09-12T12:13:15.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.144+0000 7f0d0425e640 1 -- 172.21.15.137:0/352834793 wait complete. 2024-09-12T12:13:15.154 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T12:13:15.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 -- 172.21.15.137:0/874292912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0101c40 msgr2=0x7f6ee01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:15.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/874292912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0101c40 0x7f6ee01020c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009860 tx=0x7f6ed002a340 comp rx=0 tx=0).stop 2024-09-12T12:13:15.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 -- 172.21.15.137:0/874292912 shutdown_connections 2024-09-12T12:13:15.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/874292912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0102600 0x7f6ee0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/874292912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0101c40 0x7f6ee01020c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/874292912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ee0100a50 0x7f6ee0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 -- 172.21.15.137:0/874292912 >> 172.21.15.137:0/874292912 conn(0x7f6ee00fc1b0 msgr2=0x7f6ee00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 -- 172.21.15.137:0/874292912 shutdown_connections 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 -- 172.21.15.137:0/874292912 wait complete. 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.259+0000 7f6ee7f98640 1 Processor -- start 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee7f98640 1 -- start start 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee7f98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0100a50 0x7f6ee006ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee7f98640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ee0101c40 0x7f6ee019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee7f98640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0102600 0x7f6ee019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee7f98640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010c6b0 con 0x7f6ee0100a50 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee7f98640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010c530 con 0x7f6ee0101c40 2024-09-12T12:13:15.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee7f98640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee010c830 con 0x7f6ee0102600 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0100a50 0x7f6ee006ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0100a50 0x7f6ee006ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55492/0 (socket says 172.21.15.137:55492) 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 -- 172.21.15.137:0/4146709104 learned_addr learned my addr 172.21.15.137:0/4146709104 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee6535640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0102600 0x7f6ee019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5533640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ee0101c40 0x7f6ee019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 -- 172.21.15.137:0/4146709104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0102600 msgr2=0x7f6ee019f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0102600 0x7f6ee019f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 -- 172.21.15.137:0/4146709104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ee0101c40 msgr2=0x7f6ee019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ee0101c40 0x7f6ee019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 -- 172.21.15.137:0/4146709104 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee019fa80 con 0x7f6ee0100a50 2024-09-12T12:13:15.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ee5d34640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0100a50 0x7f6ee006ac30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00e9c0 tx=0x7f6edc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:15.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.260+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00cd80 con 0x7f6ee0100a50 2024-09-12T12:13:15.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.261+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee019fd10 con 0x7f6ee0100a50 2024-09-12T12:13:15.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.261+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6edc0035f0 con 0x7f6ee0100a50 2024-09-12T12:13:15.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.261+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc010690 con 0x7f6ee0100a50 2024-09-12T12:13:15.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.261+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee01a01d0 con 0x7f6ee0100a50 2024-09-12T12:13:15.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.262+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee0107de0 con 0x7f6ee0100a50 2024-09-12T12:13:15.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.264+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f6edc0037a0 con 0x7f6ee0100a50 2024-09-12T12:13:15.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.264+0000 7f6ed6ffd640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ecc0a2a40 0x7f6ecc0a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:15.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.264+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f6edc0cf060 con 0x7f6ee0100a50 2024-09-12T12:13:15.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.264+0000 7f6ee5533640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ecc0a2a40 0x7f6ecc0a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:15.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.265+0000 7f6ee5533640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ecc0a2a40 0x7f6ecc0a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ed002a930 tx=0x7f6ed0030570 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:15.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.268+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6edc09c340 con 0x7f6ee0100a50 2024-09-12T12:13:15.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.379+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 22) ==== 137954+0+0 (secure 0 0 0) 0x7f6edc0982b0 con 0x7f6ee0100a50 2024-09-12T12:13:15.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.505+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6ee0062630 con 0x7f6ecc0a2a40 2024-09-12T12:13:15.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.511+0000 7f6ed6ffd640 1 -- 172.21.15.137:0/4146709104 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6ee0062630 con 0x7f6ecc0a2a40 2024-09-12T12:13:15.512 INFO:teuthology.orchestra.run.smithi137.stdout:/ 1m 6m3h 2024-09-12T12:13:15.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.513+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ecc0a2a40 msgr2=0x7f6ecc0a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.513+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ecc0a2a40 0x7f6ecc0a4f00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6ed002a930 tx=0x7f6ed0030570 comp rx=0 tx=0).stop 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.513+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0100a50 msgr2=0x7f6ee006ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.513+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0100a50 0x7f6ee006ac30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00e9c0 tx=0x7f6edc00ee80 comp rx=0 tx=0).stop 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 shutdown_connections 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ecc0a2a40 0x7f6ecc0a4f00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ee0102600 0x7f6ee019f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ee0101c40 0x7f6ee019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 --2- 172.21.15.137:0/4146709104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ee0100a50 0x7f6ee006ac30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 >> 172.21.15.137:0/4146709104 conn(0x7f6ee00fc1b0 msgr2=0x7f6ee0104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:15.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 shutdown_connections 2024-09-12T12:13:15.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:15.514+0000 7f6ee7f98640 1 -- 172.21.15.137:0/4146709104 wait complete. 2024-09-12T12:13:15.525 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-mon[30950]: mgrmap e21: x(active, since 1.02043s), standbys: y, z 2024-09-12T12:13:15.525 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-mon[30950]: [12/Sep/2024:12:13:14] ENGINE Bus STARTING 2024-09-12T12:13:15.525 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-mon[30950]: [12/Sep/2024:12:13:14] ENGINE Serving on http://172.21.15.137:8765 2024-09-12T12:13:15.525 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-mon[30950]: [12/Sep/2024:12:13:15] ENGINE Serving on https://172.21.15.137:7150 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-mon[30950]: [12/Sep/2024:12:13:15] ENGINE Bus STARTED 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-mon[30950]: [12/Sep/2024:12:13:15] ENGINE Error in HTTPServer.serve 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: Traceback (most recent call last): 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._connections.run(self.expiration_interval) 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._run(expiration_interval) 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:13:15.526 INFO:journalctl@ceph.mon.a.smithi137.stdout: s = self.context.wrap_socket( 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout: return self.sslsocket_class._create( 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout: self.do_handshake() 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout: self._sslobj.do_handshake() 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:13:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:15 smithi137 ceph-mon[30950]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:13:15.528 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-12T12:13:15.556 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-12T12:13:15.584 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-12T12:13:15.595 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:13:15.595 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-12T12:13:15.599 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T12:13:15.632 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T12:13:15.632 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:13:15.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:15 smithi171 ceph-mon[35098]: mgrmap e21: x(active, since 1.02043s), standbys: y, z 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:15 smithi171 ceph-mon[35098]: [12/Sep/2024:12:13:14] ENGINE Bus STARTING 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:15 smithi171 ceph-mon[35098]: [12/Sep/2024:12:13:14] ENGINE Serving on http://172.21.15.137:8765 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:15 smithi171 ceph-mon[35098]: [12/Sep/2024:12:13:15] ENGINE Serving on https://172.21.15.137:7150 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:15 smithi171 ceph-mon[35098]: [12/Sep/2024:12:13:15] ENGINE Bus STARTED 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:15 smithi171 ceph-mon[35098]: [12/Sep/2024:12:13:15] ENGINE Error in HTTPServer.serve 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout: Traceback (most recent call last): 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout: self._connections.run(self.expiration_interval) 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout: self._run(expiration_interval) 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:13:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: s = self.context.wrap_socket( 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: return self.sslsocket_class._create( 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: self.do_handshake() 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: self._sslobj.do_handshake() 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:13:15.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:15 smithi171 ceph-mon[35098]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:13:15.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:15 smithi189 ceph-mon[35475]: mgrmap e21: x(active, since 1.02043s), standbys: y, z 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:15 smithi189 ceph-mon[35475]: [12/Sep/2024:12:13:14] ENGINE Bus STARTING 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:15 smithi189 ceph-mon[35475]: [12/Sep/2024:12:13:14] ENGINE Serving on http://172.21.15.137:8765 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:15 smithi189 ceph-mon[35475]: [12/Sep/2024:12:13:15] ENGINE Serving on https://172.21.15.137:7150 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:15 smithi189 ceph-mon[35475]: [12/Sep/2024:12:13:15] ENGINE Bus STARTED 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:15 smithi189 ceph-mon[35475]: [12/Sep/2024:12:13:15] ENGINE Error in HTTPServer.serve 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: Traceback (most recent call last): 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: self._connections.run(self.expiration_interval) 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: self._run(expiration_interval) 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T12:13:15.841 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: s = self.context.wrap_socket( 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: return self.sslsocket_class._create( 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: self.do_handshake() 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: self._sslobj.do_handshake() 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T12:13:15.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:15 smithi189 ceph-mon[35475]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:13:15.915 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:16.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2570133179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 msgr2=0x7f8ca010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2570133179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 0x7f8ca010da40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009ca0 tx=0x7f8c9802a950 comp rx=0 tx=0).stop 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2570133179 shutdown_connections 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2570133179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 0x7f8ca010da40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2570133179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0102570 0x7f8ca01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2570133179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca0100e20 0x7f8ca0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2570133179 >> 172.21.15.137:0/2570133179 conn(0x7f8ca00fc7d0 msgr2=0x7f8ca00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2570133179 shutdown_connections 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.453+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2570133179 wait complete. 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.454+0000 7f8ca4c10640 1 Processor -- start 2024-09-12T12:13:16.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.454+0000 7f8ca4c10640 1 -- start start 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8ca4c10640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0100e20 0x7f8ca01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8ca4c10640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca0102570 0x7f8ca01997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8ca4c10640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 0x7f8ca01a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8ca4c10640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca0110f80 con 0x7f8ca0102ef0 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8ca4c10640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca0110e00 con 0x7f8ca0100e20 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8ca4c10640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca0111100 con 0x7f8ca0102570 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0100e20 0x7f8ca01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 0x7f8ca01a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0100e20 0x7f8ca01992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:51518/0 (socket says 172.21.15.137:51518) 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 -- 172.21.15.137:0/2724744591 learned_addr learned my addr 172.21.15.137:0/2724744591 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9dd9b640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca0102570 0x7f8ca01997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:16.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 -- 172.21.15.137:0/2724744591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca0102570 msgr2=0x7f8ca01997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca0102570 0x7f8ca01997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 -- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 msgr2=0x7f8ca01a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 0x7f8ca01a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.455+0000 7f8c9e59c640 1 -- 172.21.15.137:0/2724744591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca01a4870 con 0x7f8ca0100e20 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.456+0000 7f8c9e59c640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0100e20 0x7f8ca01992b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d950 tx=0x7f8c9000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.456+0000 7f8c8f7fe640 1 -- 172.21.15.137:0/2724744591 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90017030 con 0x7f8ca0100e20 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.456+0000 7f8c8f7fe640 1 -- 172.21.15.137:0/2724744591 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c90004020 con 0x7f8ca0100e20 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.456+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca01a4a40 con 0x7f8ca0100e20 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.456+0000 7f8c8f7fe640 1 -- 172.21.15.137:0/2724744591 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c900047a0 con 0x7f8ca0100e20 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.456+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca01a4f50 con 0x7f8ca0100e20 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.457+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca01021b0 con 0x7f8ca0100e20 2024-09-12T12:13:16.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.458+0000 7f8c8f7fe640 1 -- 172.21.15.137:0/2724744591 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 22) ==== 137954+0+0 (secure 0 0 0) 0x7f8c900041d0 con 0x7f8ca0100e20 2024-09-12T12:13:16.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.458+0000 7f8c8f7fe640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8c880a2cf0 0x7f8c880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:16.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.458+0000 7f8c9dd9b640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8c880a2cf0 0x7f8c880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:16.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.459+0000 7f8c8f7fe640 1 -- 172.21.15.137:0/2724744591 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f8c900cedd0 con 0x7f8ca0100e20 2024-09-12T12:13:16.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.459+0000 7f8c9dd9b640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8c880a2cf0 0x7f8c880a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ca006aac0 tx=0x7f8c9400b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:16.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.463+0000 7f8c8f7fe640 1 -- 172.21.15.137:0/2724744591 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c9009b9e0 con 0x7f8ca0100e20 2024-09-12T12:13:16.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.632+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ca0106f30 con 0x7f8c880a2cf0 2024-09-12T12:13:16.633 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:13:16.635 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:16 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:13:16.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.639+0000 7f8c8f7fe640 1 -- 172.21.15.137:0/2724744591 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257978 (secure 0 0 0) 0x7f8ca0106f30 con 0x7f8c880a2cf0 2024-09-12T12:13:16.643 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:16.644 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:13:16.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.642+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8c880a2cf0 msgr2=0x7f8c880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:16.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.642+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8c880a2cf0 0x7f8c880a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ca006aac0 tx=0x7f8c9400b390 comp rx=0 tx=0).stop 2024-09-12T12:13:16.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.642+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0100e20 msgr2=0x7f8ca01992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:16.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.642+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0100e20 0x7f8ca01992b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d950 tx=0x7f8c9000de10 comp rx=0 tx=0).stop 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 shutdown_connections 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8c880a2cf0 0x7f8c880a51b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ca0102ef0 0x7f8ca01a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ca0102570 0x7f8ca01997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 --2- 172.21.15.137:0/2724744591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0100e20 0x7f8ca01992b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 >> 172.21.15.137:0/2724744591 conn(0x7f8ca00fc7d0 msgr2=0x7f8ca0106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 shutdown_connections 2024-09-12T12:13:16.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:16.643+0000 7f8ca4c10640 1 -- 172.21.15.137:0/2724744591 wait complete. 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:13:16.702 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:16 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:13:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:16.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:16.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:13:16.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:16 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:13:17.482 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-12T12:13:15.372804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186976,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":929,"num_read_kb":621,"num_write":471,"num_write_kb":4375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360740,"kb_used_data":11460,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581084,"statfs":{"total":1151940427776,"available":1151571030016,"internally_reserved":0,"allocated":11735040,"data_stored":8982234,"data_compressed":215188,"data_compressed_allocated":278528,"data_compressed_original":3407872,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994730"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508775+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:13:13.508775+0000","last_peered":"2024-09-12T12:13:13.508775+0000","last_clean":"2024-09-12T12:13:13.508775+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:13:13.508775+0000","last_undegraded":"2024-09-12T12:13:13.508775+0000","last_fullsized":"2024-09-12T12:13:13.508775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511045+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:13:13.511045+0000","last_peered":"2024-09-12T12:13:13.511045+0000","last_clean":"2024-09-12T12:13:13.511045+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:13:13.511045+0000","last_undegraded":"2024-09-12T12:13:13.511045+0000","last_fullsized":"2024-09-12T12:13:13.511045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.510847+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:13:13.510847+0000","last_peered":"2024-09-12T12:13:13.510847+0000","last_clean":"2024-09-12T12:13:13.510847+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:13:13.510847+0000","last_undegraded":"2024-09-12T12:13:13.510847+0000","last_fullsized":"2024-09-12T12:13:13.510847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509302+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:13:13.509302+0000","last_peered":"2024-09-12T12:13:13.509302+0000","last_clean":"2024-09-12T12:13:13.509302+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:13:13.509302+0000","last_undegraded":"2024-09-12T12:13:13.509302+0000","last_fullsized":"2024-09-12T12:13:13.509302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456528+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:13:13.456528+0000","last_peered":"2024-09-12T12:13:13.456528+0000","last_clean":"2024-09-12T12:13:13.456528+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:13:13.456528+0000","last_undegraded":"2024-09-12T12:13:13.456528+0000","last_fullsized":"2024-09-12T12:13:13.456528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470195+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:13:13.470195+0000","last_peered":"2024-09-12T12:13:13.470195+0000","last_clean":"2024-09-12T12:13:13.470195+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:13:13.470195+0000","last_undegraded":"2024-09-12T12:13:13.470195+0000","last_fullsized":"2024-09-12T12:13:13.470195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508440+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:13:13.508440+0000","last_peered":"2024-09-12T12:13:13.508440+0000","last_clean":"2024-09-12T12:13:13.508440+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:13:13.508440+0000","last_undegraded":"2024-09-12T12:13:13.508440+0000","last_fullsized":"2024-09-12T12:13:13.508440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456974+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:13:13.456974+0000","last_peered":"2024-09-12T12:13:13.456974+0000","last_clean":"2024-09-12T12:13:13.456974+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:13:13.456974+0000","last_undegraded":"2024-09-12T12:13:13.456974+0000","last_fullsized":"2024-09-12T12:13:13.456974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360554+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:13:13.360554+0000","last_peered":"2024-09-12T12:13:13.360554+0000","last_clean":"2024-09-12T12:13:13.360554+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:13:13.360554+0000","last_undegraded":"2024-09-12T12:13:13.360554+0000","last_fullsized":"2024-09-12T12:13:13.360554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470544+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:13:13.470544+0000","last_peered":"2024-09-12T12:13:13.470544+0000","last_clean":"2024-09-12T12:13:13.470544+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:13:13.470544+0000","last_undegraded":"2024-09-12T12:13:13.470544+0000","last_fullsized":"2024-09-12T12:13:13.470544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508664+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:13:13.508664+0000","last_peered":"2024-09-12T12:13:13.508664+0000","last_clean":"2024-09-12T12:13:13.508664+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:13:13.508664+0000","last_undegraded":"2024-09-12T12:13:13.508664+0000","last_fullsized":"2024-09-12T12:13:13.508664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509342+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:13:13.509342+0000","last_peered":"2024-09-12T12:13:13.509342+0000","last_clean":"2024-09-12T12:13:13.509342+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:13:13.509342+0000","last_undegraded":"2024-09-12T12:13:13.509342+0000","last_fullsized":"2024-09-12T12:13:13.509342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357828+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:13:13.357828+0000","last_peered":"2024-09-12T12:13:13.357828+0000","last_clean":"2024-09-12T12:13:13.357828+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:13:13.357828+0000","last_undegraded":"2024-09-12T12:13:13.357828+0000","last_fullsized":"2024-09-12T12:13:13.357828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509985+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:13:13.509985+0000","last_peered":"2024-09-12T12:13:13.509985+0000","last_clean":"2024-09-12T12:13:13.509985+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:13:13.509985+0000","last_undegraded":"2024-09-12T12:13:13.509985+0000","last_fullsized":"2024-09-12T12:13:13.509985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509667+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:13:13.509667+0000","last_peered":"2024-09-12T12:13:13.509667+0000","last_clean":"2024-09-12T12:13:13.509667+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:13:13.509667+0000","last_undegraded":"2024-09-12T12:13:13.509667+0000","last_fullsized":"2024-09-12T12:13:13.509667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360066+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:13:13.360066+0000","last_peered":"2024-09-12T12:13:13.360066+0000","last_clean":"2024-09-12T12:13:13.360066+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:13:13.360066+0000","last_undegraded":"2024-09-12T12:13:13.360066+0000","last_fullsized":"2024-09-12T12:13:13.360066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508812+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:13:13.508812+0000","last_peered":"2024-09-12T12:13:13.508812+0000","last_clean":"2024-09-12T12:13:13.508812+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:13:13.508812+0000","last_undegraded":"2024-09-12T12:13:13.508812+0000","last_fullsized":"2024-09-12T12:13:13.508812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454295+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:13:13.454295+0000","last_peered":"2024-09-12T12:13:13.454295+0000","last_clean":"2024-09-12T12:13:13.454295+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:13:13.454295+0000","last_undegraded":"2024-09-12T12:13:13.454295+0000","last_fullsized":"2024-09-12T12:13:13.454295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359546+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:13:13.359546+0000","last_peered":"2024-09-12T12:13:13.359546+0000","last_clean":"2024-09-12T12:13:13.359546+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:13:13.359546+0000","last_undegraded":"2024-09-12T12:13:13.359546+0000","last_fullsized":"2024-09-12T12:13:13.359546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508574+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:13:13.508574+0000","last_peered":"2024-09-12T12:13:13.508574+0000","last_clean":"2024-09-12T12:13:13.508574+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:13:13.508574+0000","last_undegraded":"2024-09-12T12:13:13.508574+0000","last_fullsized":"2024-09-12T12:13:13.508574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357918+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:13:13.357918+0000","last_peered":"2024-09-12T12:13:13.357918+0000","last_clean":"2024-09-12T12:13:13.357918+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:13:13.357918+0000","last_undegraded":"2024-09-12T12:13:13.357918+0000","last_fullsized":"2024-09-12T12:13:13.357918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360208+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:13:13.360208+0000","last_peered":"2024-09-12T12:13:13.360208+0000","last_clean":"2024-09-12T12:13:13.360208+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:13:13.360208+0000","last_undegraded":"2024-09-12T12:13:13.360208+0000","last_fullsized":"2024-09-12T12:13:13.360208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457029+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:13:13.457029+0000","last_peered":"2024-09-12T12:13:13.457029+0000","last_clean":"2024-09-12T12:13:13.457029+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:13:13.457029+0000","last_undegraded":"2024-09-12T12:13:13.457029+0000","last_fullsized":"2024-09-12T12:13:13.457029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464351+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:13:13.464351+0000","last_peered":"2024-09-12T12:13:13.464351+0000","last_clean":"2024-09-12T12:13:13.464351+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:13:13.464351+0000","last_undegraded":"2024-09-12T12:13:13.464351+0000","last_fullsized":"2024-09-12T12:13:13.464351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458079+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:13:13.458079+0000","last_peered":"2024-09-12T12:13:13.458079+0000","last_clean":"2024-09-12T12:13:13.458079+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:13:13.458079+0000","last_undegraded":"2024-09-12T12:13:13.458079+0000","last_fullsized":"2024-09-12T12:13:13.458079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359666+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:13:13.359666+0000","last_peered":"2024-09-12T12:13:13.359666+0000","last_clean":"2024-09-12T12:13:13.359666+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:13:13.359666+0000","last_undegraded":"2024-09-12T12:13:13.359666+0000","last_fullsized":"2024-09-12T12:13:13.359666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508765+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:13:13.508765+0000","last_peered":"2024-09-12T12:13:13.508765+0000","last_clean":"2024-09-12T12:13:13.508765+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:13:13.508765+0000","last_undegraded":"2024-09-12T12:13:13.508765+0000","last_fullsized":"2024-09-12T12:13:13.508765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357834+0000","last_change":"2024-09-12T12:12:17.201138+0000","last_active":"2024-09-12T12:13:13.357834+0000","last_peered":"2024-09-12T12:13:13.357834+0000","last_clean":"2024-09-12T12:13:13.357834+0000","last_became_active":"2024-09-12T12:12:17.200768+0000","last_became_peered":"2024-09-12T12:12:17.200768+0000","last_unstale":"2024-09-12T12:13:13.357834+0000","last_undegraded":"2024-09-12T12:13:13.357834+0000","last_fullsized":"2024-09-12T12:13:13.357834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:48.749285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.510116+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:13:13.510116+0000","last_peered":"2024-09-12T12:13:13.510116+0000","last_clean":"2024-09-12T12:13:13.510116+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:13:13.510116+0000","last_undegraded":"2024-09-12T12:13:13.510116+0000","last_fullsized":"2024-09-12T12:13:13.510116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359239+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:13:13.359239+0000","last_peered":"2024-09-12T12:13:13.359239+0000","last_clean":"2024-09-12T12:13:13.359239+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:13:13.359239+0000","last_undegraded":"2024-09-12T12:13:13.359239+0000","last_fullsized":"2024-09-12T12:13:13.359239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359396+0000","last_change":"2024-09-12T12:12:17.030334+0000","last_active":"2024-09-12T12:13:13.359396+0000","last_peered":"2024-09-12T12:13:13.359396+0000","last_clean":"2024-09-12T12:13:13.359396+0000","last_became_active":"2024-09-12T12:12:17.029947+0000","last_became_peered":"2024-09-12T12:12:17.029947+0000","last_unstale":"2024-09-12T12:13:13.359396+0000","last_undegraded":"2024-09-12T12:13:13.359396+0000","last_fullsized":"2024-09-12T12:13:13.359396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:10.299557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508480+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:13:13.508480+0000","last_peered":"2024-09-12T12:13:13.508480+0000","last_clean":"2024-09-12T12:13:13.508480+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:13:13.508480+0000","last_undegraded":"2024-09-12T12:13:13.508480+0000","last_fullsized":"2024-09-12T12:13:13.508480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456459+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:13:13.456459+0000","last_peered":"2024-09-12T12:13:13.456459+0000","last_clean":"2024-09-12T12:13:13.456459+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:13:13.456459+0000","last_undegraded":"2024-09-12T12:13:13.456459+0000","last_fullsized":"2024-09-12T12:13:13.456459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.619396+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:13:13.619396+0000","last_peered":"2024-09-12T12:13:13.619396+0000","last_clean":"2024-09-12T12:13:13.619396+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:13:13.619396+0000","last_undegraded":"2024-09-12T12:13:13.619396+0000","last_fullsized":"2024-09-12T12:13:13.619396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464566+0000","last_change":"2024-09-12T12:12:17.037302+0000","last_active":"2024-09-12T12:13:13.464566+0000","last_peered":"2024-09-12T12:13:13.464566+0000","last_clean":"2024-09-12T12:13:13.464566+0000","last_became_active":"2024-09-12T12:12:17.036907+0000","last_became_peered":"2024-09-12T12:12:17.036907+0000","last_unstale":"2024-09-12T12:13:13.464566+0000","last_undegraded":"2024-09-12T12:13:13.464566+0000","last_fullsized":"2024-09-12T12:13:13.464566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:06:37.061690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470256+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:13:13.470256+0000","last_peered":"2024-09-12T12:13:13.470256+0000","last_clean":"2024-09-12T12:13:13.470256+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:13:13.470256+0000","last_undegraded":"2024-09-12T12:13:13.470256+0000","last_fullsized":"2024-09-12T12:13:13.470256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357661+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:13:13.357661+0000","last_peered":"2024-09-12T12:13:13.357661+0000","last_clean":"2024-09-12T12:13:13.357661+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:13:13.357661+0000","last_undegraded":"2024-09-12T12:13:13.357661+0000","last_fullsized":"2024-09-12T12:13:13.357661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457130+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:13:13.457130+0000","last_peered":"2024-09-12T12:13:13.457130+0000","last_clean":"2024-09-12T12:13:13.457130+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:13:13.457130+0000","last_undegraded":"2024-09-12T12:13:13.457130+0000","last_fullsized":"2024-09-12T12:13:13.457130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464646+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:13:13.464646+0000","last_peered":"2024-09-12T12:13:13.464646+0000","last_clean":"2024-09-12T12:13:13.464646+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:13:13.464646+0000","last_undegraded":"2024-09-12T12:13:13.464646+0000","last_fullsized":"2024-09-12T12:13:13.464646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454075+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:13:13.454075+0000","last_peered":"2024-09-12T12:13:13.454075+0000","last_clean":"2024-09-12T12:13:13.454075+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:13:13.454075+0000","last_undegraded":"2024-09-12T12:13:13.454075+0000","last_fullsized":"2024-09-12T12:13:13.454075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.618935+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:13:13.618935+0000","last_peered":"2024-09-12T12:13:13.618935+0000","last_clean":"2024-09-12T12:13:13.618935+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:13:13.618935+0000","last_undegraded":"2024-09-12T12:13:13.618935+0000","last_fullsized":"2024-09-12T12:13:13.618935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360344+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:13:13.360344+0000","last_peered":"2024-09-12T12:13:13.360344+0000","last_clean":"2024-09-12T12:13:13.360344+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:13:13.360344+0000","last_undegraded":"2024-09-12T12:13:13.360344+0000","last_fullsized":"2024-09-12T12:13:13.360344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508916+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:13:13.508916+0000","last_peered":"2024-09-12T12:13:13.508916+0000","last_clean":"2024-09-12T12:13:13.508916+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:13:13.508916+0000","last_undegraded":"2024-09-12T12:13:13.508916+0000","last_fullsized":"2024-09-12T12:13:13.508916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511649+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:13:13.511649+0000","last_peered":"2024-09-12T12:13:13.511649+0000","last_clean":"2024-09-12T12:13:13.511649+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:13:13.511649+0000","last_undegraded":"2024-09-12T12:13:13.511649+0000","last_fullsized":"2024-09-12T12:13:13.511649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464697+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:13:13.464697+0000","last_peered":"2024-09-12T12:13:13.464697+0000","last_clean":"2024-09-12T12:13:13.464697+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:13:13.464697+0000","last_undegraded":"2024-09-12T12:13:13.464697+0000","last_fullsized":"2024-09-12T12:13:13.464697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360194+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:13:13.360194+0000","last_peered":"2024-09-12T12:13:13.360194+0000","last_clean":"2024-09-12T12:13:13.360194+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:13:13.360194+0000","last_undegraded":"2024-09-12T12:13:13.360194+0000","last_fullsized":"2024-09-12T12:13:13.360194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":294,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.662153+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:13:13.662153+0000","last_peered":"2024-09-12T12:13:13.662153+0000","last_clean":"2024-09-12T12:13:13.662153+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:13:13.662153+0000","last_undegraded":"2024-09-12T12:13:13.662153+0000","last_fullsized":"2024-09-12T12:13:13.662153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":180,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456736+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:13:13.456736+0000","last_peered":"2024-09-12T12:13:13.456736+0000","last_clean":"2024-09-12T12:13:13.456736+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:13:13.456736+0000","last_undegraded":"2024-09-12T12:13:13.456736+0000","last_fullsized":"2024-09-12T12:13:13.456736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359428+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:13:13.359428+0000","last_peered":"2024-09-12T12:13:13.359428+0000","last_clean":"2024-09-12T12:13:13.359428+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:13:13.359428+0000","last_undegraded":"2024-09-12T12:13:13.359428+0000","last_fullsized":"2024-09-12T12:13:13.359428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357722+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:13:13.357722+0000","last_peered":"2024-09-12T12:13:13.357722+0000","last_clean":"2024-09-12T12:13:13.357722+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:13:13.357722+0000","last_undegraded":"2024-09-12T12:13:13.357722+0000","last_fullsized":"2024-09-12T12:13:13.357722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464553+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:13:13.464553+0000","last_peered":"2024-09-12T12:13:13.464553+0000","last_clean":"2024-09-12T12:13:13.464553+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:13:13.464553+0000","last_undegraded":"2024-09-12T12:13:13.464553+0000","last_fullsized":"2024-09-12T12:13:13.464553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508794+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:13:13.508794+0000","last_peered":"2024-09-12T12:13:13.508794+0000","last_clean":"2024-09-12T12:13:13.508794+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:13:13.508794+0000","last_undegraded":"2024-09-12T12:13:13.508794+0000","last_fullsized":"2024-09-12T12:13:13.508794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":151,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.664394+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:13:13.664394+0000","last_peered":"2024-09-12T12:13:13.664394+0000","last_clean":"2024-09-12T12:13:13.664394+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:13:13.664394+0000","last_undegraded":"2024-09-12T12:13:13.664394+0000","last_fullsized":"2024-09-12T12:13:13.664394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508424+0000","last_change":"2024-09-12T12:12:17.031317+0000","last_active":"2024-09-12T12:13:13.508424+0000","last_peered":"2024-09-12T12:13:13.508424+0000","last_clean":"2024-09-12T12:13:13.508424+0000","last_became_active":"2024-09-12T12:12:17.030842+0000","last_became_peered":"2024-09-12T12:12:17.030842+0000","last_unstale":"2024-09-12T12:13:13.508424+0000","last_undegraded":"2024-09-12T12:13:13.508424+0000","last_fullsized":"2024-09-12T12:13:13.508424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:11.616682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456456+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:13:13.456456+0000","last_peered":"2024-09-12T12:13:13.456456+0000","last_clean":"2024-09-12T12:13:13.456456+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:13:13.456456+0000","last_undegraded":"2024-09-12T12:13:13.456456+0000","last_fullsized":"2024-09-12T12:13:13.456456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464470+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:13:13.464470+0000","last_peered":"2024-09-12T12:13:13.464470+0000","last_clean":"2024-09-12T12:13:13.464470+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:13:13.464470+0000","last_undegraded":"2024-09-12T12:13:13.464470+0000","last_fullsized":"2024-09-12T12:13:13.464470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456848+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:13:13.456848+0000","last_peered":"2024-09-12T12:13:13.456848+0000","last_clean":"2024-09-12T12:13:13.456848+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:13:13.456848+0000","last_undegraded":"2024-09-12T12:13:13.456848+0000","last_fullsized":"2024-09-12T12:13:13.456848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508769+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:13:13.508769+0000","last_peered":"2024-09-12T12:13:13.508769+0000","last_clean":"2024-09-12T12:13:13.508769+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:13:13.508769+0000","last_undegraded":"2024-09-12T12:13:13.508769+0000","last_fullsized":"2024-09-12T12:13:13.508769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.358004+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:13:13.358004+0000","last_peered":"2024-09-12T12:13:13.358004+0000","last_clean":"2024-09-12T12:13:13.358004+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:13:13.358004+0000","last_undegraded":"2024-09-12T12:13:13.358004+0000","last_fullsized":"2024-09-12T12:13:13.358004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508965+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:13:13.508965+0000","last_peered":"2024-09-12T12:13:13.508965+0000","last_clean":"2024-09-12T12:13:13.508965+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:13:13.508965+0000","last_undegraded":"2024-09-12T12:13:13.508965+0000","last_fullsized":"2024-09-12T12:13:13.508965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.595315+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:13:13.595315+0000","last_peered":"2024-09-12T12:13:13.595315+0000","last_clean":"2024-09-12T12:13:13.595315+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:13:13.595315+0000","last_undegraded":"2024-09-12T12:13:13.595315+0000","last_fullsized":"2024-09-12T12:13:13.595315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508595+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:13:13.508595+0000","last_peered":"2024-09-12T12:13:13.508595+0000","last_clean":"2024-09-12T12:13:13.508595+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:13:13.508595+0000","last_undegraded":"2024-09-12T12:13:13.508595+0000","last_fullsized":"2024-09-12T12:13:13.508595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359543+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:13:13.359543+0000","last_peered":"2024-09-12T12:13:13.359543+0000","last_clean":"2024-09-12T12:13:13.359543+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:13:13.359543+0000","last_undegraded":"2024-09-12T12:13:13.359543+0000","last_fullsized":"2024-09-12T12:13:13.359543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508748+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:13:13.508748+0000","last_peered":"2024-09-12T12:13:13.508748+0000","last_clean":"2024-09-12T12:13:13.508748+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:13:13.508748+0000","last_undegraded":"2024-09-12T12:13:13.508748+0000","last_fullsized":"2024-09-12T12:13:13.508748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508599+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:13:13.508599+0000","last_peered":"2024-09-12T12:13:13.508599+0000","last_clean":"2024-09-12T12:13:13.508599+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:13:13.508599+0000","last_undegraded":"2024-09-12T12:13:13.508599+0000","last_fullsized":"2024-09-12T12:13:13.508599+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509280+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:13:13.509280+0000","last_peered":"2024-09-12T12:13:13.509280+0000","last_clean":"2024-09-12T12:13:13.509280+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:13:13.509280+0000","last_undegraded":"2024-09-12T12:13:13.509280+0000","last_fullsized":"2024-09-12T12:13:13.509280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457450+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:13:13.457450+0000","last_peered":"2024-09-12T12:13:13.457450+0000","last_clean":"2024-09-12T12:13:13.457450+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:13:13.457450+0000","last_undegraded":"2024-09-12T12:13:13.457450+0000","last_fullsized":"2024-09-12T12:13:13.457450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508614+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:13:13.508614+0000","last_peered":"2024-09-12T12:13:13.508614+0000","last_clean":"2024-09-12T12:13:13.508614+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:13:13.508614+0000","last_undegraded":"2024-09-12T12:13:13.508614+0000","last_fullsized":"2024-09-12T12:13:13.508614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357588+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:13:13.357588+0000","last_peered":"2024-09-12T12:13:13.357588+0000","last_clean":"2024-09-12T12:13:13.357588+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:13:13.357588+0000","last_undegraded":"2024-09-12T12:13:13.357588+0000","last_fullsized":"2024-09-12T12:13:13.357588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.619847+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:13:13.619847+0000","last_peered":"2024-09-12T12:13:13.619847+0000","last_clean":"2024-09-12T12:13:13.619847+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:13:13.619847+0000","last_undegraded":"2024-09-12T12:13:13.619847+0000","last_fullsized":"2024-09-12T12:13:13.619847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464403+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:13:13.464403+0000","last_peered":"2024-09-12T12:13:13.464403+0000","last_clean":"2024-09-12T12:13:13.464403+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:13:13.464403+0000","last_undegraded":"2024-09-12T12:13:13.464403+0000","last_fullsized":"2024-09-12T12:13:13.464403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508626+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:13:13.508626+0000","last_peered":"2024-09-12T12:13:13.508626+0000","last_clean":"2024-09-12T12:13:13.508626+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:13:13.508626+0000","last_undegraded":"2024-09-12T12:13:13.508626+0000","last_fullsized":"2024-09-12T12:13:13.508626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470385+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:13:13.470385+0000","last_peered":"2024-09-12T12:13:13.470385+0000","last_clean":"2024-09-12T12:13:13.470385+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:13:13.470385+0000","last_undegraded":"2024-09-12T12:13:13.470385+0000","last_fullsized":"2024-09-12T12:13:13.470385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454372+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:13:13.454372+0000","last_peered":"2024-09-12T12:13:13.454372+0000","last_clean":"2024-09-12T12:13:13.454372+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:13:13.454372+0000","last_undegraded":"2024-09-12T12:13:13.454372+0000","last_fullsized":"2024-09-12T12:13:13.454372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470318+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:13:13.470318+0000","last_peered":"2024-09-12T12:13:13.470318+0000","last_clean":"2024-09-12T12:13:13.470318+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:13:13.470318+0000","last_undegraded":"2024-09-12T12:13:13.470318+0000","last_fullsized":"2024-09-12T12:13:13.470318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456864+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:13:13.456864+0000","last_peered":"2024-09-12T12:13:13.456864+0000","last_clean":"2024-09-12T12:13:13.456864+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:13:13.456864+0000","last_undegraded":"2024-09-12T12:13:13.456864+0000","last_fullsized":"2024-09-12T12:13:13.456864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357909+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:13:13.357909+0000","last_peered":"2024-09-12T12:13:13.357909+0000","last_clean":"2024-09-12T12:13:13.357909+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:13:13.357909+0000","last_undegraded":"2024-09-12T12:13:13.357909+0000","last_fullsized":"2024-09-12T12:13:13.357909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'47","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.598676+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:13:13.598676+0000","last_peered":"2024-09-12T12:13:13.598676+0000","last_clean":"2024-09-12T12:13:13.598676+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:13:13.598676+0000","last_undegraded":"2024-09-12T12:13:13.598676+0000","last_fullsized":"2024-09-12T12:13:13.598676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454442+0000","last_change":"2024-09-12T12:12:17.032314+0000","last_active":"2024-09-12T12:13:13.454442+0000","last_peered":"2024-09-12T12:13:13.454442+0000","last_clean":"2024-09-12T12:13:13.454442+0000","last_became_active":"2024-09-12T12:12:17.032022+0000","last_became_peered":"2024-09-12T12:12:17.032022+0000","last_unstale":"2024-09-12T12:13:13.454442+0000","last_undegraded":"2024-09-12T12:13:13.454442+0000","last_fullsized":"2024-09-12T12:13:13.454442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:55.111688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509435+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:13:13.509435+0000","last_peered":"2024-09-12T12:13:13.509435+0000","last_clean":"2024-09-12T12:13:13.509435+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:13:13.509435+0000","last_undegraded":"2024-09-12T12:13:13.509435+0000","last_fullsized":"2024-09-12T12:13:13.509435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511183+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:13:13.511183+0000","last_peered":"2024-09-12T12:13:13.511183+0000","last_clean":"2024-09-12T12:13:13.511183+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:13:13.511183+0000","last_undegraded":"2024-09-12T12:13:13.511183+0000","last_fullsized":"2024-09-12T12:13:13.511183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360305+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:13:13.360305+0000","last_peered":"2024-09-12T12:13:13.360305+0000","last_clean":"2024-09-12T12:13:13.360305+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:13:13.360305+0000","last_undegraded":"2024-09-12T12:13:13.360305+0000","last_fullsized":"2024-09-12T12:13:13.360305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456450+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:13:13.456450+0000","last_peered":"2024-09-12T12:13:13.456450+0000","last_clean":"2024-09-12T12:13:13.456450+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:13:13.456450+0000","last_undegraded":"2024-09-12T12:13:13.456450+0000","last_fullsized":"2024-09-12T12:13:13.456450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457246+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:13:13.457246+0000","last_peered":"2024-09-12T12:13:13.457246+0000","last_clean":"2024-09-12T12:13:13.457246+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:13:13.457246+0000","last_undegraded":"2024-09-12T12:13:13.457246+0000","last_fullsized":"2024-09-12T12:13:13.457246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508548+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:13:13.508548+0000","last_peered":"2024-09-12T12:13:13.508548+0000","last_clean":"2024-09-12T12:13:13.508548+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:13:13.508548+0000","last_undegraded":"2024-09-12T12:13:13.508548+0000","last_fullsized":"2024-09-12T12:13:13.508548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508524+0000","last_change":"2024-09-12T12:12:17.038499+0000","last_active":"2024-09-12T12:13:13.508524+0000","last_peered":"2024-09-12T12:13:13.508524+0000","last_clean":"2024-09-12T12:13:13.508524+0000","last_became_active":"2024-09-12T12:12:17.034622+0000","last_became_peered":"2024-09-12T12:12:17.034622+0000","last_unstale":"2024-09-12T12:13:13.508524+0000","last_undegraded":"2024-09-12T12:13:13.508524+0000","last_fullsized":"2024-09-12T12:13:13.508524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:30.684068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457597+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:13:13.457597+0000","last_peered":"2024-09-12T12:13:13.457597+0000","last_clean":"2024-09-12T12:13:13.457597+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:13:13.457597+0000","last_undegraded":"2024-09-12T12:13:13.457597+0000","last_fullsized":"2024-09-12T12:13:13.457597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508525+0000","last_change":"2024-09-12T12:12:17.041616+0000","last_active":"2024-09-12T12:13:13.508525+0000","last_peered":"2024-09-12T12:13:13.508525+0000","last_clean":"2024-09-12T12:13:13.508525+0000","last_became_active":"2024-09-12T12:12:17.035895+0000","last_became_peered":"2024-09-12T12:12:17.035895+0000","last_unstale":"2024-09-12T12:13:13.508525+0000","last_undegraded":"2024-09-12T12:13:13.508525+0000","last_fullsized":"2024-09-12T12:13:13.508525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:11.722682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508538+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:13:13.508538+0000","last_peered":"2024-09-12T12:13:13.508538+0000","last_clean":"2024-09-12T12:13:13.508538+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:13:13.508538+0000","last_undegraded":"2024-09-12T12:13:13.508538+0000","last_fullsized":"2024-09-12T12:13:13.508538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464560+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:13:13.464560+0000","last_peered":"2024-09-12T12:13:13.464560+0000","last_clean":"2024-09-12T12:13:13.464560+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:13:13.464560+0000","last_undegraded":"2024-09-12T12:13:13.464560+0000","last_fullsized":"2024-09-12T12:13:13.464560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456728+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:13:13.456728+0000","last_peered":"2024-09-12T12:13:13.456728+0000","last_clean":"2024-09-12T12:13:13.456728+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:13:13.456728+0000","last_undegraded":"2024-09-12T12:13:13.456728+0000","last_fullsized":"2024-09-12T12:13:13.456728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360132+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:13:13.360132+0000","last_peered":"2024-09-12T12:13:13.360132+0000","last_clean":"2024-09-12T12:13:13.360132+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:13:13.360132+0000","last_undegraded":"2024-09-12T12:13:13.360132+0000","last_fullsized":"2024-09-12T12:13:13.360132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456757+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:13:13.456757+0000","last_peered":"2024-09-12T12:13:13.456757+0000","last_clean":"2024-09-12T12:13:13.456757+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:13:13.456757+0000","last_undegraded":"2024-09-12T12:13:13.456757+0000","last_fullsized":"2024-09-12T12:13:13.456757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511549+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:13:13.511549+0000","last_peered":"2024-09-12T12:13:13.511549+0000","last_clean":"2024-09-12T12:13:13.511549+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:13:13.511549+0000","last_undegraded":"2024-09-12T12:13:13.511549+0000","last_fullsized":"2024-09-12T12:13:13.511549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458550+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:13:13.458550+0000","last_peered":"2024-09-12T12:13:13.458550+0000","last_clean":"2024-09-12T12:13:13.458550+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:13:13.458550+0000","last_undegraded":"2024-09-12T12:13:13.458550+0000","last_fullsized":"2024-09-12T12:13:13.458550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458269+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:13:13.458269+0000","last_peered":"2024-09-12T12:13:13.458269+0000","last_clean":"2024-09-12T12:13:13.458269+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:13:13.458269+0000","last_undegraded":"2024-09-12T12:13:13.458269+0000","last_fullsized":"2024-09-12T12:13:13.458269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454620+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:13:13.454620+0000","last_peered":"2024-09-12T12:13:13.454620+0000","last_clean":"2024-09-12T12:13:13.454620+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:13:13.454620+0000","last_undegraded":"2024-09-12T12:13:13.454620+0000","last_fullsized":"2024-09-12T12:13:13.454620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596374,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":793,"num_read_kb":502,"num_write":288,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1789122,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":453099,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1243735,"data_compressed":44842,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":846551,"data_compressed":22049,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":846471,"data_compressed":22049,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1046815,"data_compressed":38686,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":452724,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1176471,"data_compressed":44842,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":454789,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":977851,"data_compressed":28205,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1015808,"data_stored":520187,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":518444,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200733,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:13:17.487 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:13:17.768 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:17.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:17.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:13:17.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:13:17.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:17.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:17.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:17.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:17.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:13:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi137:/etc/ceph/ceph.conf 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi189:/etc/ceph/ceph.conf 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:18.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-12T12:13:18.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.093 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.278+0000 7f7587447640 1 -- 172.21.15.137:0/1866779625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 msgr2=0x7f758010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:18.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.278+0000 7f7587447640 1 --2- 172.21.15.137:0/1866779625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f758010d940 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7578009cd0 tx=0x7f757802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.278+0000 7f7587447640 1 -- 172.21.15.137:0/1866779625 shutdown_connections 2024-09-12T12:13:18.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.278+0000 7f7587447640 1 --2- 172.21.15.137:0/1866779625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f758010d940 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.278+0000 7f7587447640 1 --2- 172.21.15.137:0/1866779625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7580100a70 0x7f7580100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.278+0000 7f7587447640 1 --2- 172.21.15.137:0/1866779625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580106a70 0x7f7580106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.278+0000 7f7587447640 1 -- 172.21.15.137:0/1866779625 >> 172.21.15.137:0/1866779625 conn(0x7f75800fc790 msgr2=0x7f75800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.279+0000 7f7587447640 1 -- 172.21.15.137:0/1866779625 shutdown_connections 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.279+0000 7f7587447640 1 -- 172.21.15.137:0/1866779625 wait complete. 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.279+0000 7f7587447640 1 Processor -- start 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.279+0000 7f7587447640 1 -- start start 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.279+0000 7f7587447640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7580100a70 0x7f75801992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f7587447640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f75801997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f7587447640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580106a70 0x7f75801a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:18.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f7587447640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7580110e70 con 0x7f7580101410 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f7587447640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7580110cf0 con 0x7f7580100a70 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f7587447640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7580110ff0 con 0x7f7580106a70 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75849e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f75801997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75849e2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f75801997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55556/0 (socket says 172.21.15.137:55556) 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75849e2640 1 -- 172.21.15.137:0/3229896296 learned_addr learned my addr 172.21.15.137:0/3229896296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75851e3640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7580100a70 0x7f75801992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75859e4640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580106a70 0x7f75801a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75851e3640 1 -- 172.21.15.137:0/3229896296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580106a70 msgr2=0x7f75801a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75851e3640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580106a70 0x7f75801a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75851e3640 1 -- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 msgr2=0x7f75801997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75851e3640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f75801997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75851e3640 1 -- 172.21.15.137:0/3229896296 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75801a4860 con 0x7f7580100a70 2024-09-12T12:13:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75849e2640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f75801997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75851e3640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7580100a70 0x7f75801992a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9f0 tx=0x7f757c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.280+0000 7f75767fc640 1 -- 172.21.15.137:0/3229896296 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c00ccc0 con 0x7f7580100a70 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.281+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75801a4a30 con 0x7f7580100a70 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.281+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801a4f40 con 0x7f7580100a70 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.281+0000 7f75767fc640 1 -- 172.21.15.137:0/3229896296 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757c005020 con 0x7f7580100a70 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.281+0000 7f75767fc640 1 -- 172.21.15.137:0/3229896296 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c0049e0 con 0x7f7580100a70 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.281+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75801021b0 con 0x7f7580100a70 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.283+0000 7f75767fc640 1 -- 172.21.15.137:0/3229896296 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f757c005560 con 0x7f7580100a70 2024-09-12T12:13:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.284+0000 7f75767fc640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f756c0a2d80 0x7f756c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:18.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.284+0000 7f75849e2640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f756c0a2d80 0x7f756c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:18.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.284+0000 7f75767fc640 1 -- 172.21.15.137:0/3229896296 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f757c0cf550 con 0x7f7580100a70 2024-09-12T12:13:18.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.284+0000 7f75849e2640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f756c0a2d80 0x7f756c0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f758006b2e0 tx=0x7f7570005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:18.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.286+0000 7f75767fc640 1 -- 172.21.15.137:0/3229896296 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f757c0051d0 con 0x7f7580100a70 2024-09-12T12:13:18.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.452+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7580062630 con 0x7f756c0a2d80 2024-09-12T12:13:18.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.460+0000 7f75767fc640 1 -- 172.21.15.137:0/3229896296 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257978 (secure 0 0 0) 0x7f7580062630 con 0x7f756c0a2d80 2024-09-12T12:13:18.463 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:18.464 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.464+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f756c0a2d80 msgr2=0x7f756c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.464+0000 7f7587447640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f756c0a2d80 0x7f756c0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f758006b2e0 tx=0x7f7570005f90 comp rx=0 tx=0).stop 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.464+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7580100a70 msgr2=0x7f75801992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.464+0000 7f7587447640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7580100a70 0x7f75801992a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e9f0 tx=0x7f757c00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 shutdown_connections 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f756c0a2d80 0x7f756c0a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7580106a70 0x7f75801a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7580101410 0x7f75801997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 --2- 172.21.15.137:0/3229896296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7580100a70 0x7f75801992a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:18.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 >> 172.21.15.137:0/3229896296 conn(0x7f75800fc790 msgr2=0x7f75800fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:18.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 shutdown_connections 2024-09-12T12:13:18.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:18.465+0000 7f7587447640 1 -- 172.21.15.137:0/3229896296 wait complete. 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:13:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:13:18.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:13:18.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:13:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: Updating smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: Updating smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: Updating smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:13:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:13:19.222 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-12T12:13:17.373233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186976,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":929,"num_read_kb":621,"num_write":471,"num_write_kb":4375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":865,"ondisk_log_size":865,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360740,"kb_used_data":11460,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581084,"statfs":{"total":1151940427776,"available":1151571030016,"internally_reserved":0,"allocated":11735040,"data_stored":8982234,"data_compressed":215188,"data_compressed_allocated":278528,"data_compressed_original":3407872,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.995159"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508775+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:13:13.508775+0000","last_peered":"2024-09-12T12:13:13.508775+0000","last_clean":"2024-09-12T12:13:13.508775+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:13:13.508775+0000","last_undegraded":"2024-09-12T12:13:13.508775+0000","last_fullsized":"2024-09-12T12:13:13.508775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511045+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:13:13.511045+0000","last_peered":"2024-09-12T12:13:13.511045+0000","last_clean":"2024-09-12T12:13:13.511045+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:13:13.511045+0000","last_undegraded":"2024-09-12T12:13:13.511045+0000","last_fullsized":"2024-09-12T12:13:13.511045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.510847+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:13:13.510847+0000","last_peered":"2024-09-12T12:13:13.510847+0000","last_clean":"2024-09-12T12:13:13.510847+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:13:13.510847+0000","last_undegraded":"2024-09-12T12:13:13.510847+0000","last_fullsized":"2024-09-12T12:13:13.510847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509302+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:13:13.509302+0000","last_peered":"2024-09-12T12:13:13.509302+0000","last_clean":"2024-09-12T12:13:13.509302+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:13:13.509302+0000","last_undegraded":"2024-09-12T12:13:13.509302+0000","last_fullsized":"2024-09-12T12:13:13.509302+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456528+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:13:13.456528+0000","last_peered":"2024-09-12T12:13:13.456528+0000","last_clean":"2024-09-12T12:13:13.456528+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:13:13.456528+0000","last_undegraded":"2024-09-12T12:13:13.456528+0000","last_fullsized":"2024-09-12T12:13:13.456528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470195+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:13:13.470195+0000","last_peered":"2024-09-12T12:13:13.470195+0000","last_clean":"2024-09-12T12:13:13.470195+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:13:13.470195+0000","last_undegraded":"2024-09-12T12:13:13.470195+0000","last_fullsized":"2024-09-12T12:13:13.470195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508440+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:13:13.508440+0000","last_peered":"2024-09-12T12:13:13.508440+0000","last_clean":"2024-09-12T12:13:13.508440+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:13:13.508440+0000","last_undegraded":"2024-09-12T12:13:13.508440+0000","last_fullsized":"2024-09-12T12:13:13.508440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456974+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:13:13.456974+0000","last_peered":"2024-09-12T12:13:13.456974+0000","last_clean":"2024-09-12T12:13:13.456974+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:13:13.456974+0000","last_undegraded":"2024-09-12T12:13:13.456974+0000","last_fullsized":"2024-09-12T12:13:13.456974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360554+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:13:13.360554+0000","last_peered":"2024-09-12T12:13:13.360554+0000","last_clean":"2024-09-12T12:13:13.360554+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:13:13.360554+0000","last_undegraded":"2024-09-12T12:13:13.360554+0000","last_fullsized":"2024-09-12T12:13:13.360554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470544+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:13:13.470544+0000","last_peered":"2024-09-12T12:13:13.470544+0000","last_clean":"2024-09-12T12:13:13.470544+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:13:13.470544+0000","last_undegraded":"2024-09-12T12:13:13.470544+0000","last_fullsized":"2024-09-12T12:13:13.470544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508664+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:13:13.508664+0000","last_peered":"2024-09-12T12:13:13.508664+0000","last_clean":"2024-09-12T12:13:13.508664+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:13:13.508664+0000","last_undegraded":"2024-09-12T12:13:13.508664+0000","last_fullsized":"2024-09-12T12:13:13.508664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509342+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:13:13.509342+0000","last_peered":"2024-09-12T12:13:13.509342+0000","last_clean":"2024-09-12T12:13:13.509342+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:13:13.509342+0000","last_undegraded":"2024-09-12T12:13:13.509342+0000","last_fullsized":"2024-09-12T12:13:13.509342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357828+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:13:13.357828+0000","last_peered":"2024-09-12T12:13:13.357828+0000","last_clean":"2024-09-12T12:13:13.357828+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:13:13.357828+0000","last_undegraded":"2024-09-12T12:13:13.357828+0000","last_fullsized":"2024-09-12T12:13:13.357828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509985+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:13:13.509985+0000","last_peered":"2024-09-12T12:13:13.509985+0000","last_clean":"2024-09-12T12:13:13.509985+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:13:13.509985+0000","last_undegraded":"2024-09-12T12:13:13.509985+0000","last_fullsized":"2024-09-12T12:13:13.509985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509667+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:13:13.509667+0000","last_peered":"2024-09-12T12:13:13.509667+0000","last_clean":"2024-09-12T12:13:13.509667+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:13:13.509667+0000","last_undegraded":"2024-09-12T12:13:13.509667+0000","last_fullsized":"2024-09-12T12:13:13.509667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360066+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:13:13.360066+0000","last_peered":"2024-09-12T12:13:13.360066+0000","last_clean":"2024-09-12T12:13:13.360066+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:13:13.360066+0000","last_undegraded":"2024-09-12T12:13:13.360066+0000","last_fullsized":"2024-09-12T12:13:13.360066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508812+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:13:13.508812+0000","last_peered":"2024-09-12T12:13:13.508812+0000","last_clean":"2024-09-12T12:13:13.508812+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:13:13.508812+0000","last_undegraded":"2024-09-12T12:13:13.508812+0000","last_fullsized":"2024-09-12T12:13:13.508812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454295+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:13:13.454295+0000","last_peered":"2024-09-12T12:13:13.454295+0000","last_clean":"2024-09-12T12:13:13.454295+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:13:13.454295+0000","last_undegraded":"2024-09-12T12:13:13.454295+0000","last_fullsized":"2024-09-12T12:13:13.454295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359546+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:13:13.359546+0000","last_peered":"2024-09-12T12:13:13.359546+0000","last_clean":"2024-09-12T12:13:13.359546+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:13:13.359546+0000","last_undegraded":"2024-09-12T12:13:13.359546+0000","last_fullsized":"2024-09-12T12:13:13.359546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508574+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:13:13.508574+0000","last_peered":"2024-09-12T12:13:13.508574+0000","last_clean":"2024-09-12T12:13:13.508574+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:13:13.508574+0000","last_undegraded":"2024-09-12T12:13:13.508574+0000","last_fullsized":"2024-09-12T12:13:13.508574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357918+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:13:13.357918+0000","last_peered":"2024-09-12T12:13:13.357918+0000","last_clean":"2024-09-12T12:13:13.357918+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:13:13.357918+0000","last_undegraded":"2024-09-12T12:13:13.357918+0000","last_fullsized":"2024-09-12T12:13:13.357918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360208+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:13:13.360208+0000","last_peered":"2024-09-12T12:13:13.360208+0000","last_clean":"2024-09-12T12:13:13.360208+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:13:13.360208+0000","last_undegraded":"2024-09-12T12:13:13.360208+0000","last_fullsized":"2024-09-12T12:13:13.360208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457029+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:13:13.457029+0000","last_peered":"2024-09-12T12:13:13.457029+0000","last_clean":"2024-09-12T12:13:13.457029+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:13:13.457029+0000","last_undegraded":"2024-09-12T12:13:13.457029+0000","last_fullsized":"2024-09-12T12:13:13.457029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464351+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:13:13.464351+0000","last_peered":"2024-09-12T12:13:13.464351+0000","last_clean":"2024-09-12T12:13:13.464351+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:13:13.464351+0000","last_undegraded":"2024-09-12T12:13:13.464351+0000","last_fullsized":"2024-09-12T12:13:13.464351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458079+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:13:13.458079+0000","last_peered":"2024-09-12T12:13:13.458079+0000","last_clean":"2024-09-12T12:13:13.458079+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:13:13.458079+0000","last_undegraded":"2024-09-12T12:13:13.458079+0000","last_fullsized":"2024-09-12T12:13:13.458079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359666+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:13:13.359666+0000","last_peered":"2024-09-12T12:13:13.359666+0000","last_clean":"2024-09-12T12:13:13.359666+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:13:13.359666+0000","last_undegraded":"2024-09-12T12:13:13.359666+0000","last_fullsized":"2024-09-12T12:13:13.359666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508765+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:13:13.508765+0000","last_peered":"2024-09-12T12:13:13.508765+0000","last_clean":"2024-09-12T12:13:13.508765+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:13:13.508765+0000","last_undegraded":"2024-09-12T12:13:13.508765+0000","last_fullsized":"2024-09-12T12:13:13.508765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357834+0000","last_change":"2024-09-12T12:12:17.201138+0000","last_active":"2024-09-12T12:13:13.357834+0000","last_peered":"2024-09-12T12:13:13.357834+0000","last_clean":"2024-09-12T12:13:13.357834+0000","last_became_active":"2024-09-12T12:12:17.200768+0000","last_became_peered":"2024-09-12T12:12:17.200768+0000","last_unstale":"2024-09-12T12:13:13.357834+0000","last_undegraded":"2024-09-12T12:13:13.357834+0000","last_fullsized":"2024-09-12T12:13:13.357834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:48.749285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.510116+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:13:13.510116+0000","last_peered":"2024-09-12T12:13:13.510116+0000","last_clean":"2024-09-12T12:13:13.510116+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:13:13.510116+0000","last_undegraded":"2024-09-12T12:13:13.510116+0000","last_fullsized":"2024-09-12T12:13:13.510116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359239+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:13:13.359239+0000","last_peered":"2024-09-12T12:13:13.359239+0000","last_clean":"2024-09-12T12:13:13.359239+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:13:13.359239+0000","last_undegraded":"2024-09-12T12:13:13.359239+0000","last_fullsized":"2024-09-12T12:13:13.359239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359396+0000","last_change":"2024-09-12T12:12:17.030334+0000","last_active":"2024-09-12T12:13:13.359396+0000","last_peered":"2024-09-12T12:13:13.359396+0000","last_clean":"2024-09-12T12:13:13.359396+0000","last_became_active":"2024-09-12T12:12:17.029947+0000","last_became_peered":"2024-09-12T12:12:17.029947+0000","last_unstale":"2024-09-12T12:13:13.359396+0000","last_undegraded":"2024-09-12T12:13:13.359396+0000","last_fullsized":"2024-09-12T12:13:13.359396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:10.299557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508480+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:13:13.508480+0000","last_peered":"2024-09-12T12:13:13.508480+0000","last_clean":"2024-09-12T12:13:13.508480+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:13:13.508480+0000","last_undegraded":"2024-09-12T12:13:13.508480+0000","last_fullsized":"2024-09-12T12:13:13.508480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456459+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:13:13.456459+0000","last_peered":"2024-09-12T12:13:13.456459+0000","last_clean":"2024-09-12T12:13:13.456459+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:13:13.456459+0000","last_undegraded":"2024-09-12T12:13:13.456459+0000","last_fullsized":"2024-09-12T12:13:13.456459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.619396+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:13:13.619396+0000","last_peered":"2024-09-12T12:13:13.619396+0000","last_clean":"2024-09-12T12:13:13.619396+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:13:13.619396+0000","last_undegraded":"2024-09-12T12:13:13.619396+0000","last_fullsized":"2024-09-12T12:13:13.619396+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464566+0000","last_change":"2024-09-12T12:12:17.037302+0000","last_active":"2024-09-12T12:13:13.464566+0000","last_peered":"2024-09-12T12:13:13.464566+0000","last_clean":"2024-09-12T12:13:13.464566+0000","last_became_active":"2024-09-12T12:12:17.036907+0000","last_became_peered":"2024-09-12T12:12:17.036907+0000","last_unstale":"2024-09-12T12:13:13.464566+0000","last_undegraded":"2024-09-12T12:13:13.464566+0000","last_fullsized":"2024-09-12T12:13:13.464566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:06:37.061690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470256+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:13:13.470256+0000","last_peered":"2024-09-12T12:13:13.470256+0000","last_clean":"2024-09-12T12:13:13.470256+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:13:13.470256+0000","last_undegraded":"2024-09-12T12:13:13.470256+0000","last_fullsized":"2024-09-12T12:13:13.470256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357661+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:13:13.357661+0000","last_peered":"2024-09-12T12:13:13.357661+0000","last_clean":"2024-09-12T12:13:13.357661+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:13:13.357661+0000","last_undegraded":"2024-09-12T12:13:13.357661+0000","last_fullsized":"2024-09-12T12:13:13.357661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457130+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:13:13.457130+0000","last_peered":"2024-09-12T12:13:13.457130+0000","last_clean":"2024-09-12T12:13:13.457130+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:13:13.457130+0000","last_undegraded":"2024-09-12T12:13:13.457130+0000","last_fullsized":"2024-09-12T12:13:13.457130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464646+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:13:13.464646+0000","last_peered":"2024-09-12T12:13:13.464646+0000","last_clean":"2024-09-12T12:13:13.464646+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:13:13.464646+0000","last_undegraded":"2024-09-12T12:13:13.464646+0000","last_fullsized":"2024-09-12T12:13:13.464646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454075+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:13:13.454075+0000","last_peered":"2024-09-12T12:13:13.454075+0000","last_clean":"2024-09-12T12:13:13.454075+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:13:13.454075+0000","last_undegraded":"2024-09-12T12:13:13.454075+0000","last_fullsized":"2024-09-12T12:13:13.454075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.618935+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:13:13.618935+0000","last_peered":"2024-09-12T12:13:13.618935+0000","last_clean":"2024-09-12T12:13:13.618935+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:13:13.618935+0000","last_undegraded":"2024-09-12T12:13:13.618935+0000","last_fullsized":"2024-09-12T12:13:13.618935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360344+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:13:13.360344+0000","last_peered":"2024-09-12T12:13:13.360344+0000","last_clean":"2024-09-12T12:13:13.360344+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:13:13.360344+0000","last_undegraded":"2024-09-12T12:13:13.360344+0000","last_fullsized":"2024-09-12T12:13:13.360344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508916+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:13:13.508916+0000","last_peered":"2024-09-12T12:13:13.508916+0000","last_clean":"2024-09-12T12:13:13.508916+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:13:13.508916+0000","last_undegraded":"2024-09-12T12:13:13.508916+0000","last_fullsized":"2024-09-12T12:13:13.508916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511649+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:13:13.511649+0000","last_peered":"2024-09-12T12:13:13.511649+0000","last_clean":"2024-09-12T12:13:13.511649+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:13:13.511649+0000","last_undegraded":"2024-09-12T12:13:13.511649+0000","last_fullsized":"2024-09-12T12:13:13.511649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464697+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:13:13.464697+0000","last_peered":"2024-09-12T12:13:13.464697+0000","last_clean":"2024-09-12T12:13:13.464697+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:13:13.464697+0000","last_undegraded":"2024-09-12T12:13:13.464697+0000","last_fullsized":"2024-09-12T12:13:13.464697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360194+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:13:13.360194+0000","last_peered":"2024-09-12T12:13:13.360194+0000","last_clean":"2024-09-12T12:13:13.360194+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:13:13.360194+0000","last_undegraded":"2024-09-12T12:13:13.360194+0000","last_fullsized":"2024-09-12T12:13:13.360194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":294,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.662153+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:13:13.662153+0000","last_peered":"2024-09-12T12:13:13.662153+0000","last_clean":"2024-09-12T12:13:13.662153+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:13:13.662153+0000","last_undegraded":"2024-09-12T12:13:13.662153+0000","last_fullsized":"2024-09-12T12:13:13.662153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":180,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456736+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:13:13.456736+0000","last_peered":"2024-09-12T12:13:13.456736+0000","last_clean":"2024-09-12T12:13:13.456736+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:13:13.456736+0000","last_undegraded":"2024-09-12T12:13:13.456736+0000","last_fullsized":"2024-09-12T12:13:13.456736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359428+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:13:13.359428+0000","last_peered":"2024-09-12T12:13:13.359428+0000","last_clean":"2024-09-12T12:13:13.359428+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:13:13.359428+0000","last_undegraded":"2024-09-12T12:13:13.359428+0000","last_fullsized":"2024-09-12T12:13:13.359428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357722+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:13:13.357722+0000","last_peered":"2024-09-12T12:13:13.357722+0000","last_clean":"2024-09-12T12:13:13.357722+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:13:13.357722+0000","last_undegraded":"2024-09-12T12:13:13.357722+0000","last_fullsized":"2024-09-12T12:13:13.357722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464553+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:13:13.464553+0000","last_peered":"2024-09-12T12:13:13.464553+0000","last_clean":"2024-09-12T12:13:13.464553+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:13:13.464553+0000","last_undegraded":"2024-09-12T12:13:13.464553+0000","last_fullsized":"2024-09-12T12:13:13.464553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508794+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:13:13.508794+0000","last_peered":"2024-09-12T12:13:13.508794+0000","last_clean":"2024-09-12T12:13:13.508794+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:13:13.508794+0000","last_undegraded":"2024-09-12T12:13:13.508794+0000","last_fullsized":"2024-09-12T12:13:13.508794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":151,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.664394+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:13:13.664394+0000","last_peered":"2024-09-12T12:13:13.664394+0000","last_clean":"2024-09-12T12:13:13.664394+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:13:13.664394+0000","last_undegraded":"2024-09-12T12:13:13.664394+0000","last_fullsized":"2024-09-12T12:13:13.664394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":6,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508424+0000","last_change":"2024-09-12T12:12:17.031317+0000","last_active":"2024-09-12T12:13:13.508424+0000","last_peered":"2024-09-12T12:13:13.508424+0000","last_clean":"2024-09-12T12:13:13.508424+0000","last_became_active":"2024-09-12T12:12:17.030842+0000","last_became_peered":"2024-09-12T12:12:17.030842+0000","last_unstale":"2024-09-12T12:13:13.508424+0000","last_undegraded":"2024-09-12T12:13:13.508424+0000","last_fullsized":"2024-09-12T12:13:13.508424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:11.616682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456456+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:13:13.456456+0000","last_peered":"2024-09-12T12:13:13.456456+0000","last_clean":"2024-09-12T12:13:13.456456+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:13:13.456456+0000","last_undegraded":"2024-09-12T12:13:13.456456+0000","last_fullsized":"2024-09-12T12:13:13.456456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464470+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:13:13.464470+0000","last_peered":"2024-09-12T12:13:13.464470+0000","last_clean":"2024-09-12T12:13:13.464470+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:13:13.464470+0000","last_undegraded":"2024-09-12T12:13:13.464470+0000","last_fullsized":"2024-09-12T12:13:13.464470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456848+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:13:13.456848+0000","last_peered":"2024-09-12T12:13:13.456848+0000","last_clean":"2024-09-12T12:13:13.456848+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:13:13.456848+0000","last_undegraded":"2024-09-12T12:13:13.456848+0000","last_fullsized":"2024-09-12T12:13:13.456848+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508769+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:13:13.508769+0000","last_peered":"2024-09-12T12:13:13.508769+0000","last_clean":"2024-09-12T12:13:13.508769+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:13:13.508769+0000","last_undegraded":"2024-09-12T12:13:13.508769+0000","last_fullsized":"2024-09-12T12:13:13.508769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.358004+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:13:13.358004+0000","last_peered":"2024-09-12T12:13:13.358004+0000","last_clean":"2024-09-12T12:13:13.358004+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:13:13.358004+0000","last_undegraded":"2024-09-12T12:13:13.358004+0000","last_fullsized":"2024-09-12T12:13:13.358004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508965+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:13:13.508965+0000","last_peered":"2024-09-12T12:13:13.508965+0000","last_clean":"2024-09-12T12:13:13.508965+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:13:13.508965+0000","last_undegraded":"2024-09-12T12:13:13.508965+0000","last_fullsized":"2024-09-12T12:13:13.508965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.595315+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:13:13.595315+0000","last_peered":"2024-09-12T12:13:13.595315+0000","last_clean":"2024-09-12T12:13:13.595315+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:13:13.595315+0000","last_undegraded":"2024-09-12T12:13:13.595315+0000","last_fullsized":"2024-09-12T12:13:13.595315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508595+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:13:13.508595+0000","last_peered":"2024-09-12T12:13:13.508595+0000","last_clean":"2024-09-12T12:13:13.508595+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:13:13.508595+0000","last_undegraded":"2024-09-12T12:13:13.508595+0000","last_fullsized":"2024-09-12T12:13:13.508595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359543+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:13:13.359543+0000","last_peered":"2024-09-12T12:13:13.359543+0000","last_clean":"2024-09-12T12:13:13.359543+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:13:13.359543+0000","last_undegraded":"2024-09-12T12:13:13.359543+0000","last_fullsized":"2024-09-12T12:13:13.359543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508748+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:13:13.508748+0000","last_peered":"2024-09-12T12:13:13.508748+0000","last_clean":"2024-09-12T12:13:13.508748+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:13:13.508748+0000","last_undegraded":"2024-09-12T12:13:13.508748+0000","last_fullsized":"2024-09-12T12:13:13.508748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508599+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:13:13.508599+0000","last_peered":"2024-09-12T12:13:13.508599+0000","last_clean":"2024-09-12T12:13:13.508599+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:13:13.508599+0000","last_undegraded":"2024-09-12T12:13:13.508599+0000","last_fullsized":"2024-09-12T12:13:13.508599+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509280+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:13:13.509280+0000","last_peered":"2024-09-12T12:13:13.509280+0000","last_clean":"2024-09-12T12:13:13.509280+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:13:13.509280+0000","last_undegraded":"2024-09-12T12:13:13.509280+0000","last_fullsized":"2024-09-12T12:13:13.509280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457450+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:13:13.457450+0000","last_peered":"2024-09-12T12:13:13.457450+0000","last_clean":"2024-09-12T12:13:13.457450+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:13:13.457450+0000","last_undegraded":"2024-09-12T12:13:13.457450+0000","last_fullsized":"2024-09-12T12:13:13.457450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508614+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:13:13.508614+0000","last_peered":"2024-09-12T12:13:13.508614+0000","last_clean":"2024-09-12T12:13:13.508614+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:13:13.508614+0000","last_undegraded":"2024-09-12T12:13:13.508614+0000","last_fullsized":"2024-09-12T12:13:13.508614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357588+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:13:13.357588+0000","last_peered":"2024-09-12T12:13:13.357588+0000","last_clean":"2024-09-12T12:13:13.357588+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:13:13.357588+0000","last_undegraded":"2024-09-12T12:13:13.357588+0000","last_fullsized":"2024-09-12T12:13:13.357588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.619847+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:13:13.619847+0000","last_peered":"2024-09-12T12:13:13.619847+0000","last_clean":"2024-09-12T12:13:13.619847+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:13:13.619847+0000","last_undegraded":"2024-09-12T12:13:13.619847+0000","last_fullsized":"2024-09-12T12:13:13.619847+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464403+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:13:13.464403+0000","last_peered":"2024-09-12T12:13:13.464403+0000","last_clean":"2024-09-12T12:13:13.464403+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:13:13.464403+0000","last_undegraded":"2024-09-12T12:13:13.464403+0000","last_fullsized":"2024-09-12T12:13:13.464403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508626+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:13:13.508626+0000","last_peered":"2024-09-12T12:13:13.508626+0000","last_clean":"2024-09-12T12:13:13.508626+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:13:13.508626+0000","last_undegraded":"2024-09-12T12:13:13.508626+0000","last_fullsized":"2024-09-12T12:13:13.508626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470385+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:13:13.470385+0000","last_peered":"2024-09-12T12:13:13.470385+0000","last_clean":"2024-09-12T12:13:13.470385+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:13:13.470385+0000","last_undegraded":"2024-09-12T12:13:13.470385+0000","last_fullsized":"2024-09-12T12:13:13.470385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454372+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:13:13.454372+0000","last_peered":"2024-09-12T12:13:13.454372+0000","last_clean":"2024-09-12T12:13:13.454372+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:13:13.454372+0000","last_undegraded":"2024-09-12T12:13:13.454372+0000","last_fullsized":"2024-09-12T12:13:13.454372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470318+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:13:13.470318+0000","last_peered":"2024-09-12T12:13:13.470318+0000","last_clean":"2024-09-12T12:13:13.470318+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:13:13.470318+0000","last_undegraded":"2024-09-12T12:13:13.470318+0000","last_fullsized":"2024-09-12T12:13:13.470318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456864+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:13:13.456864+0000","last_peered":"2024-09-12T12:13:13.456864+0000","last_clean":"2024-09-12T12:13:13.456864+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:13:13.456864+0000","last_undegraded":"2024-09-12T12:13:13.456864+0000","last_fullsized":"2024-09-12T12:13:13.456864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357909+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:13:13.357909+0000","last_peered":"2024-09-12T12:13:13.357909+0000","last_clean":"2024-09-12T12:13:13.357909+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:13:13.357909+0000","last_undegraded":"2024-09-12T12:13:13.357909+0000","last_fullsized":"2024-09-12T12:13:13.357909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'47","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.598676+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:13:13.598676+0000","last_peered":"2024-09-12T12:13:13.598676+0000","last_clean":"2024-09-12T12:13:13.598676+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:13:13.598676+0000","last_undegraded":"2024-09-12T12:13:13.598676+0000","last_fullsized":"2024-09-12T12:13:13.598676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454442+0000","last_change":"2024-09-12T12:12:17.032314+0000","last_active":"2024-09-12T12:13:13.454442+0000","last_peered":"2024-09-12T12:13:13.454442+0000","last_clean":"2024-09-12T12:13:13.454442+0000","last_became_active":"2024-09-12T12:12:17.032022+0000","last_became_peered":"2024-09-12T12:12:17.032022+0000","last_unstale":"2024-09-12T12:13:13.454442+0000","last_undegraded":"2024-09-12T12:13:13.454442+0000","last_fullsized":"2024-09-12T12:13:13.454442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:55.111688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509435+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:13:13.509435+0000","last_peered":"2024-09-12T12:13:13.509435+0000","last_clean":"2024-09-12T12:13:13.509435+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:13:13.509435+0000","last_undegraded":"2024-09-12T12:13:13.509435+0000","last_fullsized":"2024-09-12T12:13:13.509435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511183+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:13:13.511183+0000","last_peered":"2024-09-12T12:13:13.511183+0000","last_clean":"2024-09-12T12:13:13.511183+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:13:13.511183+0000","last_undegraded":"2024-09-12T12:13:13.511183+0000","last_fullsized":"2024-09-12T12:13:13.511183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360305+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:13:13.360305+0000","last_peered":"2024-09-12T12:13:13.360305+0000","last_clean":"2024-09-12T12:13:13.360305+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:13:13.360305+0000","last_undegraded":"2024-09-12T12:13:13.360305+0000","last_fullsized":"2024-09-12T12:13:13.360305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456450+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:13:13.456450+0000","last_peered":"2024-09-12T12:13:13.456450+0000","last_clean":"2024-09-12T12:13:13.456450+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:13:13.456450+0000","last_undegraded":"2024-09-12T12:13:13.456450+0000","last_fullsized":"2024-09-12T12:13:13.456450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457246+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:13:13.457246+0000","last_peered":"2024-09-12T12:13:13.457246+0000","last_clean":"2024-09-12T12:13:13.457246+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:13:13.457246+0000","last_undegraded":"2024-09-12T12:13:13.457246+0000","last_fullsized":"2024-09-12T12:13:13.457246+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508548+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:13:13.508548+0000","last_peered":"2024-09-12T12:13:13.508548+0000","last_clean":"2024-09-12T12:13:13.508548+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:13:13.508548+0000","last_undegraded":"2024-09-12T12:13:13.508548+0000","last_fullsized":"2024-09-12T12:13:13.508548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508524+0000","last_change":"2024-09-12T12:12:17.038499+0000","last_active":"2024-09-12T12:13:13.508524+0000","last_peered":"2024-09-12T12:13:13.508524+0000","last_clean":"2024-09-12T12:13:13.508524+0000","last_became_active":"2024-09-12T12:12:17.034622+0000","last_became_peered":"2024-09-12T12:12:17.034622+0000","last_unstale":"2024-09-12T12:13:13.508524+0000","last_undegraded":"2024-09-12T12:13:13.508524+0000","last_fullsized":"2024-09-12T12:13:13.508524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:30.684068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457597+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:13:13.457597+0000","last_peered":"2024-09-12T12:13:13.457597+0000","last_clean":"2024-09-12T12:13:13.457597+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:13:13.457597+0000","last_undegraded":"2024-09-12T12:13:13.457597+0000","last_fullsized":"2024-09-12T12:13:13.457597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508525+0000","last_change":"2024-09-12T12:12:17.041616+0000","last_active":"2024-09-12T12:13:13.508525+0000","last_peered":"2024-09-12T12:13:13.508525+0000","last_clean":"2024-09-12T12:13:13.508525+0000","last_became_active":"2024-09-12T12:12:17.035895+0000","last_became_peered":"2024-09-12T12:12:17.035895+0000","last_unstale":"2024-09-12T12:13:13.508525+0000","last_undegraded":"2024-09-12T12:13:13.508525+0000","last_fullsized":"2024-09-12T12:13:13.508525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:11.722682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508538+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:13:13.508538+0000","last_peered":"2024-09-12T12:13:13.508538+0000","last_clean":"2024-09-12T12:13:13.508538+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:13:13.508538+0000","last_undegraded":"2024-09-12T12:13:13.508538+0000","last_fullsized":"2024-09-12T12:13:13.508538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464560+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:13:13.464560+0000","last_peered":"2024-09-12T12:13:13.464560+0000","last_clean":"2024-09-12T12:13:13.464560+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:13:13.464560+0000","last_undegraded":"2024-09-12T12:13:13.464560+0000","last_fullsized":"2024-09-12T12:13:13.464560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456728+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:13:13.456728+0000","last_peered":"2024-09-12T12:13:13.456728+0000","last_clean":"2024-09-12T12:13:13.456728+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:13:13.456728+0000","last_undegraded":"2024-09-12T12:13:13.456728+0000","last_fullsized":"2024-09-12T12:13:13.456728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360132+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:13:13.360132+0000","last_peered":"2024-09-12T12:13:13.360132+0000","last_clean":"2024-09-12T12:13:13.360132+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:13:13.360132+0000","last_undegraded":"2024-09-12T12:13:13.360132+0000","last_fullsized":"2024-09-12T12:13:13.360132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456757+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:13:13.456757+0000","last_peered":"2024-09-12T12:13:13.456757+0000","last_clean":"2024-09-12T12:13:13.456757+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:13:13.456757+0000","last_undegraded":"2024-09-12T12:13:13.456757+0000","last_fullsized":"2024-09-12T12:13:13.456757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511549+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:13:13.511549+0000","last_peered":"2024-09-12T12:13:13.511549+0000","last_clean":"2024-09-12T12:13:13.511549+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:13:13.511549+0000","last_undegraded":"2024-09-12T12:13:13.511549+0000","last_fullsized":"2024-09-12T12:13:13.511549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458550+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:13:13.458550+0000","last_peered":"2024-09-12T12:13:13.458550+0000","last_clean":"2024-09-12T12:13:13.458550+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:13:13.458550+0000","last_undegraded":"2024-09-12T12:13:13.458550+0000","last_fullsized":"2024-09-12T12:13:13.458550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458269+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:13:13.458269+0000","last_peered":"2024-09-12T12:13:13.458269+0000","last_clean":"2024-09-12T12:13:13.458269+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:13:13.458269+0000","last_undegraded":"2024-09-12T12:13:13.458269+0000","last_fullsized":"2024-09-12T12:13:13.458269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454620+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:13:13.454620+0000","last_peered":"2024-09-12T12:13:13.454620+0000","last_clean":"2024-09-12T12:13:13.454620+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:13:13.454620+0000","last_undegraded":"2024-09-12T12:13:13.454620+0000","last_fullsized":"2024-09-12T12:13:13.454620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596374,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":793,"num_read_kb":502,"num_write":288,"num_write_kb":1435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1789122,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":453099,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1243735,"data_compressed":44842,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":846551,"data_compressed":22049,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":962560,"data_stored":846471,"data_compressed":22049,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1046815,"data_compressed":38686,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561087,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":452724,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1176471,"data_compressed":44842,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":454789,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":977851,"data_compressed":28205,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1015808,"data_stored":520187,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":518444,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":933888,"data_stored":445097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":133469,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200733,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:13:19.229 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T12:13:19.229 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T12:13:19.484 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:19.797 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:19 smithi137 ceph-mon[30950]: from='client.24857 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:19.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:19 smithi171 ceph-mon[35098]: from='client.24857 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:20.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 -- 172.21.15.137:0/3873408130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51641027a0 msgr2=0x7f51641067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:20.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 --2- 172.21.15.137:0/3873408130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51641027a0 0x7f51641067f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5158009900 tx=0x7f515802a410 comp rx=0 tx=0).stop 2024-09-12T12:13:20.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 -- 172.21.15.137:0/3873408130 shutdown_connections 2024-09-12T12:13:20.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 --2- 172.21.15.137:0/3873408130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5164106f20 0x7f516410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 --2- 172.21.15.137:0/3873408130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51641027a0 0x7f51641067f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 --2- 172.21.15.137:0/3873408130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5164101df0 0x7f51641021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 -- 172.21.15.137:0/3873408130 >> 172.21.15.137:0/3873408130 conn(0x7f51640fd8f0 msgr2=0x7f51640ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 -- 172.21.15.137:0/3873408130 shutdown_connections 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 -- 172.21.15.137:0/3873408130 wait complete. 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 Processor -- start 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.036+0000 7f516c539640 1 -- start start 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516c539640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5164101df0 0x7f5164198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516c539640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51641027a0 0x7f51641989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516c539640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164106f20 0x7f516419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516c539640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5164111ff0 con 0x7f5164106f20 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516c539640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5164111e70 con 0x7f51641027a0 2024-09-12T12:13:20.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516c539640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5164112170 con 0x7f5164101df0 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164106f20 0x7f516419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164106f20 0x7f516419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51566/0 (socket says 172.21.15.137:51566) 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 -- 172.21.15.137:0/3232946288 learned_addr learned my addr 172.21.15.137:0/3232946288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516a2d5640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5164101df0 0x7f5164198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f5169ad4640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51641027a0 0x7f51641989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 -- 172.21.15.137:0/3232946288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5164101df0 msgr2=0x7f5164198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5164101df0 0x7f5164198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 -- 172.21.15.137:0/3232946288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51641027a0 msgr2=0x7f51641989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51641027a0 0x7f51641989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 -- 172.21.15.137:0/3232946288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516419d440 con 0x7f5164106f20 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f516aad6640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164106f20 0x7f516419cd60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f515c009fd0 tx=0x7f515c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:20.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.037+0000 7f51637fe640 1 -- 172.21.15.137:0/3232946288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c011830 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.038+0000 7f51637fe640 1 -- 172.21.15.137:0/3232946288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515c016070 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.038+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516419d730 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.038+0000 7f51637fe640 1 -- 172.21.15.137:0/3232946288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c00f2e0 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.038+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641a5010 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.039+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51641021d0 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.040+0000 7f51637fe640 1 -- 172.21.15.137:0/3232946288 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f515c0119d0 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.041+0000 7f51637fe640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f514c0a2d80 0x7f514c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.041+0000 7f516a2d5640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f514c0a2d80 0x7f514c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.041+0000 7f51637fe640 1 -- 172.21.15.137:0/3232946288 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f515c0ceb40 con 0x7f5164106f20 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.041+0000 7f516a2d5640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f514c0a2d80 0x7f514c0a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5154005fd0 tx=0x7f5154009680 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:20.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.045+0000 7f51637fe640 1 -- 172.21.15.137:0/3232946288 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f515c09b750 con 0x7f5164106f20 2024-09-12T12:13:20.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:19 smithi189 ceph-mon[35475]: from='client.24857 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:20.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.211+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5164062630 con 0x7f514c0a2d80 2024-09-12T12:13:20.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.217+0000 7f51637fe640 1 -- 172.21.15.137:0/3232946288 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258007 (secure 0 0 0) 0x7f5164062630 con 0x7f514c0a2d80 2024-09-12T12:13:20.220 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:20.221 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-09-12T12:13:20.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f514c0a2d80 msgr2=0x7f514c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:20.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f514c0a2d80 0x7f514c0a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5154005fd0 tx=0x7f5154009680 comp rx=0 tx=0).stop 2024-09-12T12:13:20.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164106f20 msgr2=0x7f516419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164106f20 0x7f516419cd60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f515c009fd0 tx=0x7f515c00b8e0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 shutdown_connections 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f514c0a2d80 0x7f514c0a5240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5164106f20 0x7f516419cd60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f51641027a0 0x7f51641989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 --2- 172.21.15.137:0/3232946288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5164101df0 0x7f5164198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 >> 172.21.15.137:0/3232946288 conn(0x7f51640fd8f0 msgr2=0x7f51640fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:20.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 shutdown_connections 2024-09-12T12:13:20.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:20.220+0000 7f516c539640 1 -- 172.21.15.137:0/3232946288 wait complete. 2024-09-12T12:13:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:20 smithi171 ceph-mon[35098]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-09-12T12:13:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:20 smithi171 ceph-mon[35098]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:20.970 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:20 smithi137 ceph-mon[30950]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-09-12T12:13:20.970 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:20 smithi137 ceph-mon[30950]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:20.998 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-12T12:13:19.373527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1383608,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1070,"num_read_kb":862,"num_write":520,"num_write_kb":4842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":903,"ondisk_log_size":903,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360812,"kb_used_data":11520,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581012,"statfs":{"total":1151940427776,"available":1151570956288,"internally_reserved":0,"allocated":11796480,"data_stored":9384396,"data_compressed":236469,"data_compressed_allocated":307200,"data_compressed_original":3784704,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":241,"num_write":49,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995454"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508775+0000","last_change":"2024-09-12T12:11:12.504215+0000","last_active":"2024-09-12T12:13:13.508775+0000","last_peered":"2024-09-12T12:13:13.508775+0000","last_clean":"2024-09-12T12:13:13.508775+0000","last_became_active":"2024-09-12T12:11:12.503833+0000","last_became_peered":"2024-09-12T12:11:12.503833+0000","last_unstale":"2024-09-12T12:13:13.508775+0000","last_undegraded":"2024-09-12T12:13:13.508775+0000","last_fullsized":"2024-09-12T12:13:13.508775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:41:00.484142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511099+0000","last_change":"2024-09-12T12:11:12.506003+0000","last_active":"2024-09-12T12:13:13.511099+0000","last_peered":"2024-09-12T12:13:13.511099+0000","last_clean":"2024-09-12T12:13:13.511099+0000","last_became_active":"2024-09-12T12:11:12.504199+0000","last_became_peered":"2024-09-12T12:11:12.504199+0000","last_unstale":"2024-09-12T12:13:13.511099+0000","last_undegraded":"2024-09-12T12:13:13.511099+0000","last_fullsized":"2024-09-12T12:13:13.511099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:34:30.089027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.510847+0000","last_change":"2024-09-12T12:11:12.503498+0000","last_active":"2024-09-12T12:13:13.510847+0000","last_peered":"2024-09-12T12:13:13.510847+0000","last_clean":"2024-09-12T12:13:13.510847+0000","last_became_active":"2024-09-12T12:11:12.501440+0000","last_became_peered":"2024-09-12T12:11:12.501440+0000","last_unstale":"2024-09-12T12:13:13.510847+0000","last_undegraded":"2024-09-12T12:13:13.510847+0000","last_fullsized":"2024-09-12T12:13:13.510847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:17:21.520533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509362+0000","last_change":"2024-09-12T12:11:12.509745+0000","last_active":"2024-09-12T12:13:13.509362+0000","last_peered":"2024-09-12T12:13:13.509362+0000","last_clean":"2024-09-12T12:13:13.509362+0000","last_became_active":"2024-09-12T12:11:12.509389+0000","last_became_peered":"2024-09-12T12:11:12.509389+0000","last_unstale":"2024-09-12T12:13:13.509362+0000","last_undegraded":"2024-09-12T12:13:13.509362+0000","last_fullsized":"2024-09-12T12:13:13.509362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:22:22.802740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456528+0000","last_change":"2024-09-12T12:11:12.494279+0000","last_active":"2024-09-12T12:13:13.456528+0000","last_peered":"2024-09-12T12:13:13.456528+0000","last_clean":"2024-09-12T12:13:13.456528+0000","last_became_active":"2024-09-12T12:11:12.493950+0000","last_became_peered":"2024-09-12T12:11:12.493950+0000","last_unstale":"2024-09-12T12:13:13.456528+0000","last_undegraded":"2024-09-12T12:13:13.456528+0000","last_fullsized":"2024-09-12T12:13:13.456528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:09:24.704470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470195+0000","last_change":"2024-09-12T12:11:12.500785+0000","last_active":"2024-09-12T12:13:13.470195+0000","last_peered":"2024-09-12T12:13:13.470195+0000","last_clean":"2024-09-12T12:13:13.470195+0000","last_became_active":"2024-09-12T12:11:12.500443+0000","last_became_peered":"2024-09-12T12:11:12.500443+0000","last_unstale":"2024-09-12T12:13:13.470195+0000","last_undegraded":"2024-09-12T12:13:13.470195+0000","last_fullsized":"2024-09-12T12:13:13.470195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:46:25.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508440+0000","last_change":"2024-09-12T12:11:12.496840+0000","last_active":"2024-09-12T12:13:13.508440+0000","last_peered":"2024-09-12T12:13:13.508440+0000","last_clean":"2024-09-12T12:13:13.508440+0000","last_became_active":"2024-09-12T12:11:12.496469+0000","last_became_peered":"2024-09-12T12:11:12.496469+0000","last_unstale":"2024-09-12T12:13:13.508440+0000","last_undegraded":"2024-09-12T12:13:13.508440+0000","last_fullsized":"2024-09-12T12:13:13.508440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:29:54.715557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456974+0000","last_change":"2024-09-12T12:11:12.499069+0000","last_active":"2024-09-12T12:13:13.456974+0000","last_peered":"2024-09-12T12:13:13.456974+0000","last_clean":"2024-09-12T12:13:13.456974+0000","last_became_active":"2024-09-12T12:11:12.498551+0000","last_became_peered":"2024-09-12T12:11:12.498551+0000","last_unstale":"2024-09-12T12:13:13.456974+0000","last_undegraded":"2024-09-12T12:13:13.456974+0000","last_fullsized":"2024-09-12T12:13:13.456974+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:42.402309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360554+0000","last_change":"2024-09-12T12:11:12.497936+0000","last_active":"2024-09-12T12:13:13.360554+0000","last_peered":"2024-09-12T12:13:13.360554+0000","last_clean":"2024-09-12T12:13:13.360554+0000","last_became_active":"2024-09-12T12:11:12.497247+0000","last_became_peered":"2024-09-12T12:11:12.497247+0000","last_unstale":"2024-09-12T12:13:13.360554+0000","last_undegraded":"2024-09-12T12:13:13.360554+0000","last_fullsized":"2024-09-12T12:13:13.360554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:44:56.355132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470544+0000","last_change":"2024-09-12T12:11:12.501551+0000","last_active":"2024-09-12T12:13:13.470544+0000","last_peered":"2024-09-12T12:13:13.470544+0000","last_clean":"2024-09-12T12:13:13.470544+0000","last_became_active":"2024-09-12T12:11:12.501251+0000","last_became_peered":"2024-09-12T12:11:12.501251+0000","last_unstale":"2024-09-12T12:13:13.470544+0000","last_undegraded":"2024-09-12T12:13:13.470544+0000","last_fullsized":"2024-09-12T12:13:13.470544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:38:06.766937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508664+0000","last_change":"2024-09-12T12:11:12.504973+0000","last_active":"2024-09-12T12:13:13.508664+0000","last_peered":"2024-09-12T12:13:13.508664+0000","last_clean":"2024-09-12T12:13:13.508664+0000","last_became_active":"2024-09-12T12:11:12.504372+0000","last_became_peered":"2024-09-12T12:11:12.504372+0000","last_unstale":"2024-09-12T12:13:13.508664+0000","last_undegraded":"2024-09-12T12:13:13.508664+0000","last_fullsized":"2024-09-12T12:13:13.508664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:18.815221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509342+0000","last_change":"2024-09-12T12:11:12.510146+0000","last_active":"2024-09-12T12:13:13.509342+0000","last_peered":"2024-09-12T12:13:13.509342+0000","last_clean":"2024-09-12T12:13:13.509342+0000","last_became_active":"2024-09-12T12:11:12.509783+0000","last_became_peered":"2024-09-12T12:11:12.509783+0000","last_unstale":"2024-09-12T12:13:13.509342+0000","last_undegraded":"2024-09-12T12:13:13.509342+0000","last_fullsized":"2024-09-12T12:13:13.509342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:50:34.020297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357828+0000","last_change":"2024-09-12T12:11:12.495841+0000","last_active":"2024-09-12T12:13:13.357828+0000","last_peered":"2024-09-12T12:13:13.357828+0000","last_clean":"2024-09-12T12:13:13.357828+0000","last_became_active":"2024-09-12T12:11:12.495560+0000","last_became_peered":"2024-09-12T12:11:12.495560+0000","last_unstale":"2024-09-12T12:13:13.357828+0000","last_undegraded":"2024-09-12T12:13:13.357828+0000","last_fullsized":"2024-09-12T12:13:13.357828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:29:26.355509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509985+0000","last_change":"2024-09-12T12:11:12.500192+0000","last_active":"2024-09-12T12:13:13.509985+0000","last_peered":"2024-09-12T12:13:13.509985+0000","last_clean":"2024-09-12T12:13:13.509985+0000","last_became_active":"2024-09-12T12:11:12.499835+0000","last_became_peered":"2024-09-12T12:11:12.499835+0000","last_unstale":"2024-09-12T12:13:13.509985+0000","last_undegraded":"2024-09-12T12:13:13.509985+0000","last_fullsized":"2024-09-12T12:13:13.509985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:22:12.528330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509667+0000","last_change":"2024-09-12T12:11:12.510158+0000","last_active":"2024-09-12T12:13:13.509667+0000","last_peered":"2024-09-12T12:13:13.509667+0000","last_clean":"2024-09-12T12:13:13.509667+0000","last_became_active":"2024-09-12T12:11:12.509769+0000","last_became_peered":"2024-09-12T12:11:12.509769+0000","last_unstale":"2024-09-12T12:13:13.509667+0000","last_undegraded":"2024-09-12T12:13:13.509667+0000","last_fullsized":"2024-09-12T12:13:13.509667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:38:43.242180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360066+0000","last_change":"2024-09-12T12:11:12.502794+0000","last_active":"2024-09-12T12:13:13.360066+0000","last_peered":"2024-09-12T12:13:13.360066+0000","last_clean":"2024-09-12T12:13:13.360066+0000","last_became_active":"2024-09-12T12:11:12.502455+0000","last_became_peered":"2024-09-12T12:11:12.502455+0000","last_unstale":"2024-09-12T12:13:13.360066+0000","last_undegraded":"2024-09-12T12:13:13.360066+0000","last_fullsized":"2024-09-12T12:13:13.360066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:27:40.621735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508812+0000","last_change":"2024-09-12T12:11:12.500739+0000","last_active":"2024-09-12T12:13:13.508812+0000","last_peered":"2024-09-12T12:13:13.508812+0000","last_clean":"2024-09-12T12:13:13.508812+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:13:13.508812+0000","last_undegraded":"2024-09-12T12:13:13.508812+0000","last_fullsized":"2024-09-12T12:13:13.508812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:33:05.974198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454295+0000","last_change":"2024-09-12T12:11:12.493539+0000","last_active":"2024-09-12T12:13:13.454295+0000","last_peered":"2024-09-12T12:13:13.454295+0000","last_clean":"2024-09-12T12:13:13.454295+0000","last_became_active":"2024-09-12T12:11:12.493217+0000","last_became_peered":"2024-09-12T12:11:12.493217+0000","last_unstale":"2024-09-12T12:13:13.454295+0000","last_undegraded":"2024-09-12T12:13:13.454295+0000","last_fullsized":"2024-09-12T12:13:13.454295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:22:18.856027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359546+0000","last_change":"2024-09-12T12:11:12.505581+0000","last_active":"2024-09-12T12:13:13.359546+0000","last_peered":"2024-09-12T12:13:13.359546+0000","last_clean":"2024-09-12T12:13:13.359546+0000","last_became_active":"2024-09-12T12:11:12.505269+0000","last_became_peered":"2024-09-12T12:11:12.505269+0000","last_unstale":"2024-09-12T12:13:13.359546+0000","last_undegraded":"2024-09-12T12:13:13.359546+0000","last_fullsized":"2024-09-12T12:13:13.359546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:49:18.909282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508574+0000","last_change":"2024-09-12T12:11:12.503391+0000","last_active":"2024-09-12T12:13:13.508574+0000","last_peered":"2024-09-12T12:13:13.508574+0000","last_clean":"2024-09-12T12:13:13.508574+0000","last_became_active":"2024-09-12T12:11:12.503060+0000","last_became_peered":"2024-09-12T12:11:12.503060+0000","last_unstale":"2024-09-12T12:13:13.508574+0000","last_undegraded":"2024-09-12T12:13:13.508574+0000","last_fullsized":"2024-09-12T12:13:13.508574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:17:07.200180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357918+0000","last_change":"2024-09-12T12:11:12.501455+0000","last_active":"2024-09-12T12:13:13.357918+0000","last_peered":"2024-09-12T12:13:13.357918+0000","last_clean":"2024-09-12T12:13:13.357918+0000","last_became_active":"2024-09-12T12:11:12.501162+0000","last_became_peered":"2024-09-12T12:11:12.501162+0000","last_unstale":"2024-09-12T12:13:13.357918+0000","last_undegraded":"2024-09-12T12:13:13.357918+0000","last_fullsized":"2024-09-12T12:13:13.357918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:21:10.690192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360208+0000","last_change":"2024-09-12T12:11:12.500544+0000","last_active":"2024-09-12T12:13:13.360208+0000","last_peered":"2024-09-12T12:13:13.360208+0000","last_clean":"2024-09-12T12:13:13.360208+0000","last_became_active":"2024-09-12T12:11:12.500224+0000","last_became_peered":"2024-09-12T12:11:12.500224+0000","last_unstale":"2024-09-12T12:13:13.360208+0000","last_undegraded":"2024-09-12T12:13:13.360208+0000","last_fullsized":"2024-09-12T12:13:13.360208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:20:06.165422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457029+0000","last_change":"2024-09-12T12:11:12.505628+0000","last_active":"2024-09-12T12:13:13.457029+0000","last_peered":"2024-09-12T12:13:13.457029+0000","last_clean":"2024-09-12T12:13:13.457029+0000","last_became_active":"2024-09-12T12:11:12.505345+0000","last_became_peered":"2024-09-12T12:11:12.505345+0000","last_unstale":"2024-09-12T12:13:13.457029+0000","last_undegraded":"2024-09-12T12:13:13.457029+0000","last_fullsized":"2024-09-12T12:13:13.457029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:07:55.164828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464351+0000","last_change":"2024-09-12T12:11:12.498060+0000","last_active":"2024-09-12T12:13:13.464351+0000","last_peered":"2024-09-12T12:13:13.464351+0000","last_clean":"2024-09-12T12:13:13.464351+0000","last_became_active":"2024-09-12T12:11:12.497713+0000","last_became_peered":"2024-09-12T12:11:12.497713+0000","last_unstale":"2024-09-12T12:13:13.464351+0000","last_undegraded":"2024-09-12T12:13:13.464351+0000","last_fullsized":"2024-09-12T12:13:13.464351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:14:24.426708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458079+0000","last_change":"2024-09-12T12:11:12.502151+0000","last_active":"2024-09-12T12:13:13.458079+0000","last_peered":"2024-09-12T12:13:13.458079+0000","last_clean":"2024-09-12T12:13:13.458079+0000","last_became_active":"2024-09-12T12:11:12.500733+0000","last_became_peered":"2024-09-12T12:11:12.500733+0000","last_unstale":"2024-09-12T12:13:13.458079+0000","last_undegraded":"2024-09-12T12:13:13.458079+0000","last_fullsized":"2024-09-12T12:13:13.458079+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:15:08.091356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359666+0000","last_change":"2024-09-12T12:11:12.499988+0000","last_active":"2024-09-12T12:13:13.359666+0000","last_peered":"2024-09-12T12:13:13.359666+0000","last_clean":"2024-09-12T12:13:13.359666+0000","last_became_active":"2024-09-12T12:11:12.499652+0000","last_became_peered":"2024-09-12T12:11:12.499652+0000","last_unstale":"2024-09-12T12:13:13.359666+0000","last_undegraded":"2024-09-12T12:13:13.359666+0000","last_fullsized":"2024-09-12T12:13:13.359666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:20:39.422120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508765+0000","last_change":"2024-09-12T12:11:12.502567+0000","last_active":"2024-09-12T12:13:13.508765+0000","last_peered":"2024-09-12T12:13:13.508765+0000","last_clean":"2024-09-12T12:13:13.508765+0000","last_became_active":"2024-09-12T12:11:12.498566+0000","last_became_peered":"2024-09-12T12:11:12.498566+0000","last_unstale":"2024-09-12T12:13:13.508765+0000","last_undegraded":"2024-09-12T12:13:13.508765+0000","last_fullsized":"2024-09-12T12:13:13.508765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:07:20.685456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357834+0000","last_change":"2024-09-12T12:12:17.201138+0000","last_active":"2024-09-12T12:13:13.357834+0000","last_peered":"2024-09-12T12:13:13.357834+0000","last_clean":"2024-09-12T12:13:13.357834+0000","last_became_active":"2024-09-12T12:12:17.200768+0000","last_became_peered":"2024-09-12T12:12:17.200768+0000","last_unstale":"2024-09-12T12:13:13.357834+0000","last_undegraded":"2024-09-12T12:13:13.357834+0000","last_fullsized":"2024-09-12T12:13:13.357834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:48.749285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.510159+0000","last_change":"2024-09-12T12:11:12.510163+0000","last_active":"2024-09-12T12:13:13.510159+0000","last_peered":"2024-09-12T12:13:13.510159+0000","last_clean":"2024-09-12T12:13:13.510159+0000","last_became_active":"2024-09-12T12:11:12.509860+0000","last_became_peered":"2024-09-12T12:11:12.509860+0000","last_unstale":"2024-09-12T12:13:13.510159+0000","last_undegraded":"2024-09-12T12:13:13.510159+0000","last_fullsized":"2024-09-12T12:13:13.510159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:40:09.740517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359303+0000","last_change":"2024-09-12T12:11:12.505944+0000","last_active":"2024-09-12T12:13:13.359303+0000","last_peered":"2024-09-12T12:13:13.359303+0000","last_clean":"2024-09-12T12:13:13.359303+0000","last_became_active":"2024-09-12T12:11:12.504244+0000","last_became_peered":"2024-09-12T12:11:12.504244+0000","last_unstale":"2024-09-12T12:13:13.359303+0000","last_undegraded":"2024-09-12T12:13:13.359303+0000","last_fullsized":"2024-09-12T12:13:13.359303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:58:41.268543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359429+0000","last_change":"2024-09-12T12:12:17.030334+0000","last_active":"2024-09-12T12:13:13.359429+0000","last_peered":"2024-09-12T12:13:13.359429+0000","last_clean":"2024-09-12T12:13:13.359429+0000","last_became_active":"2024-09-12T12:12:17.029947+0000","last_became_peered":"2024-09-12T12:12:17.029947+0000","last_unstale":"2024-09-12T12:13:13.359429+0000","last_undegraded":"2024-09-12T12:13:13.359429+0000","last_fullsized":"2024-09-12T12:13:13.359429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:48:10.299557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508480+0000","last_change":"2024-09-12T12:11:12.497983+0000","last_active":"2024-09-12T12:13:13.508480+0000","last_peered":"2024-09-12T12:13:13.508480+0000","last_clean":"2024-09-12T12:13:13.508480+0000","last_became_active":"2024-09-12T12:11:12.496983+0000","last_became_peered":"2024-09-12T12:11:12.496983+0000","last_unstale":"2024-09-12T12:13:13.508480+0000","last_undegraded":"2024-09-12T12:13:13.508480+0000","last_fullsized":"2024-09-12T12:13:13.508480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:27:32.439016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456459+0000","last_change":"2024-09-12T12:11:12.495148+0000","last_active":"2024-09-12T12:13:13.456459+0000","last_peered":"2024-09-12T12:13:13.456459+0000","last_clean":"2024-09-12T12:13:13.456459+0000","last_became_active":"2024-09-12T12:11:12.494823+0000","last_became_peered":"2024-09-12T12:11:12.494823+0000","last_unstale":"2024-09-12T12:13:13.456459+0000","last_undegraded":"2024-09-12T12:13:13.456459+0000","last_fullsized":"2024-09-12T12:13:13.456459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:09:52.510630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":152,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:17.625222+0000","last_change":"2024-09-12T12:11:09.485475+0000","last_active":"2024-09-12T12:13:17.625222+0000","last_peered":"2024-09-12T12:13:17.625222+0000","last_clean":"2024-09-12T12:13:17.625222+0000","last_became_active":"2024-09-12T12:11:09.485143+0000","last_became_peered":"2024-09-12T12:11:09.485143+0000","last_unstale":"2024-09-12T12:13:17.625222+0000","last_undegraded":"2024-09-12T12:13:17.625222+0000","last_fullsized":"2024-09-12T12:13:17.625222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:01:44.583270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464566+0000","last_change":"2024-09-12T12:12:17.037302+0000","last_active":"2024-09-12T12:13:13.464566+0000","last_peered":"2024-09-12T12:13:13.464566+0000","last_clean":"2024-09-12T12:13:13.464566+0000","last_became_active":"2024-09-12T12:12:17.036907+0000","last_became_peered":"2024-09-12T12:12:17.036907+0000","last_unstale":"2024-09-12T12:13:13.464566+0000","last_undegraded":"2024-09-12T12:13:13.464566+0000","last_fullsized":"2024-09-12T12:13:13.464566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:06:37.061690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470256+0000","last_change":"2024-09-12T12:11:09.486909+0000","last_active":"2024-09-12T12:13:13.470256+0000","last_peered":"2024-09-12T12:13:13.470256+0000","last_clean":"2024-09-12T12:13:13.470256+0000","last_became_active":"2024-09-12T12:11:09.486542+0000","last_became_peered":"2024-09-12T12:11:09.486542+0000","last_unstale":"2024-09-12T12:13:13.470256+0000","last_undegraded":"2024-09-12T12:13:13.470256+0000","last_fullsized":"2024-09-12T12:13:13.470256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:07:44.762779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357661+0000","last_change":"2024-09-12T12:11:09.481840+0000","last_active":"2024-09-12T12:13:13.357661+0000","last_peered":"2024-09-12T12:13:13.357661+0000","last_clean":"2024-09-12T12:13:13.357661+0000","last_became_active":"2024-09-12T12:11:09.481504+0000","last_became_peered":"2024-09-12T12:11:09.481504+0000","last_unstale":"2024-09-12T12:13:13.357661+0000","last_undegraded":"2024-09-12T12:13:13.357661+0000","last_fullsized":"2024-09-12T12:13:13.357661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:13:28.337035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457130+0000","last_change":"2024-09-12T12:11:12.506592+0000","last_active":"2024-09-12T12:13:13.457130+0000","last_peered":"2024-09-12T12:13:13.457130+0000","last_clean":"2024-09-12T12:13:13.457130+0000","last_became_active":"2024-09-12T12:11:12.504930+0000","last_became_peered":"2024-09-12T12:11:12.504930+0000","last_unstale":"2024-09-12T12:13:13.457130+0000","last_undegraded":"2024-09-12T12:13:13.457130+0000","last_fullsized":"2024-09-12T12:13:13.457130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:02:58.022609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464646+0000","last_change":"2024-09-12T12:11:09.486567+0000","last_active":"2024-09-12T12:13:13.464646+0000","last_peered":"2024-09-12T12:13:13.464646+0000","last_clean":"2024-09-12T12:13:13.464646+0000","last_became_active":"2024-09-12T12:11:09.486222+0000","last_became_peered":"2024-09-12T12:11:09.486222+0000","last_unstale":"2024-09-12T12:13:13.464646+0000","last_undegraded":"2024-09-12T12:13:13.464646+0000","last_fullsized":"2024-09-12T12:13:13.464646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T00:05:47.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454075+0000","last_change":"2024-09-12T12:11:12.492540+0000","last_active":"2024-09-12T12:13:13.454075+0000","last_peered":"2024-09-12T12:13:13.454075+0000","last_clean":"2024-09-12T12:13:13.454075+0000","last_became_active":"2024-09-12T12:11:12.492229+0000","last_became_peered":"2024-09-12T12:11:12.492229+0000","last_unstale":"2024-09-12T12:13:13.454075+0000","last_undegraded":"2024-09-12T12:13:13.454075+0000","last_fullsized":"2024-09-12T12:13:13.454075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:15.728802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":142,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.618935+0000","last_change":"2024-09-12T12:11:09.485785+0000","last_active":"2024-09-12T12:13:13.618935+0000","last_peered":"2024-09-12T12:13:13.618935+0000","last_clean":"2024-09-12T12:13:13.618935+0000","last_became_active":"2024-09-12T12:11:09.485451+0000","last_became_peered":"2024-09-12T12:11:09.485451+0000","last_unstale":"2024-09-12T12:13:13.618935+0000","last_undegraded":"2024-09-12T12:13:13.618935+0000","last_fullsized":"2024-09-12T12:13:13.618935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:28:59.301611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360344+0000","last_change":"2024-09-12T12:11:12.498227+0000","last_active":"2024-09-12T12:13:13.360344+0000","last_peered":"2024-09-12T12:13:13.360344+0000","last_clean":"2024-09-12T12:13:13.360344+0000","last_became_active":"2024-09-12T12:11:12.497301+0000","last_became_peered":"2024-09-12T12:11:12.497301+0000","last_unstale":"2024-09-12T12:13:13.360344+0000","last_undegraded":"2024-09-12T12:13:13.360344+0000","last_fullsized":"2024-09-12T12:13:13.360344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:24:54.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508916+0000","last_change":"2024-09-12T12:11:09.486179+0000","last_active":"2024-09-12T12:13:13.508916+0000","last_peered":"2024-09-12T12:13:13.508916+0000","last_clean":"2024-09-12T12:13:13.508916+0000","last_became_active":"2024-09-12T12:11:09.484646+0000","last_became_peered":"2024-09-12T12:11:09.484646+0000","last_unstale":"2024-09-12T12:13:13.508916+0000","last_undegraded":"2024-09-12T12:13:13.508916+0000","last_fullsized":"2024-09-12T12:13:13.508916+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:20:36.416776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511649+0000","last_change":"2024-09-12T12:11:12.510331+0000","last_active":"2024-09-12T12:13:13.511649+0000","last_peered":"2024-09-12T12:13:13.511649+0000","last_clean":"2024-09-12T12:13:13.511649+0000","last_became_active":"2024-09-12T12:11:12.509924+0000","last_became_peered":"2024-09-12T12:11:12.509924+0000","last_unstale":"2024-09-12T12:13:13.511649+0000","last_undegraded":"2024-09-12T12:13:13.511649+0000","last_fullsized":"2024-09-12T12:13:13.511649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:04:34.846529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464697+0000","last_change":"2024-09-12T12:11:09.483710+0000","last_active":"2024-09-12T12:13:13.464697+0000","last_peered":"2024-09-12T12:13:13.464697+0000","last_clean":"2024-09-12T12:13:13.464697+0000","last_became_active":"2024-09-12T12:11:09.483173+0000","last_became_peered":"2024-09-12T12:11:09.483173+0000","last_unstale":"2024-09-12T12:13:13.464697+0000","last_undegraded":"2024-09-12T12:13:13.464697+0000","last_fullsized":"2024-09-12T12:13:13.464697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:40:49.098753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360194+0000","last_change":"2024-09-12T12:11:12.509158+0000","last_active":"2024-09-12T12:13:13.360194+0000","last_peered":"2024-09-12T12:13:13.360194+0000","last_clean":"2024-09-12T12:13:13.360194+0000","last_became_active":"2024-09-12T12:11:12.508834+0000","last_became_peered":"2024-09-12T12:11:12.508834+0000","last_unstale":"2024-09-12T12:13:13.360194+0000","last_undegraded":"2024-09-12T12:13:13.360194+0000","last_fullsized":"2024-09-12T12:13:13.360194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:14:55.942096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'34","reported_seq":294,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.662153+0000","last_change":"2024-09-12T12:11:09.489065+0000","last_active":"2024-09-12T12:13:13.662153+0000","last_peered":"2024-09-12T12:13:13.662153+0000","last_clean":"2024-09-12T12:13:13.662153+0000","last_became_active":"2024-09-12T12:11:09.488575+0000","last_became_peered":"2024-09-12T12:11:09.488575+0000","last_unstale":"2024-09-12T12:13:13.662153+0000","last_undegraded":"2024-09-12T12:13:13.662153+0000","last_fullsized":"2024-09-12T12:13:13.662153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:38.933413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":180,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456736+0000","last_change":"2024-09-12T12:11:12.503486+0000","last_active":"2024-09-12T12:13:13.456736+0000","last_peered":"2024-09-12T12:13:13.456736+0000","last_clean":"2024-09-12T12:13:13.456736+0000","last_became_active":"2024-09-12T12:11:12.502738+0000","last_became_peered":"2024-09-12T12:11:12.502738+0000","last_unstale":"2024-09-12T12:13:13.456736+0000","last_undegraded":"2024-09-12T12:13:13.456736+0000","last_fullsized":"2024-09-12T12:13:13.456736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:34:53.705745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359428+0000","last_change":"2024-09-12T12:11:09.479381+0000","last_active":"2024-09-12T12:13:13.359428+0000","last_peered":"2024-09-12T12:13:13.359428+0000","last_clean":"2024-09-12T12:13:13.359428+0000","last_became_active":"2024-09-12T12:11:09.478981+0000","last_became_peered":"2024-09-12T12:11:09.478981+0000","last_unstale":"2024-09-12T12:13:13.359428+0000","last_undegraded":"2024-09-12T12:13:13.359428+0000","last_fullsized":"2024-09-12T12:13:13.359428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:23:08.286808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357722+0000","last_change":"2024-09-12T12:11:12.501416+0000","last_active":"2024-09-12T12:13:13.357722+0000","last_peered":"2024-09-12T12:13:13.357722+0000","last_clean":"2024-09-12T12:13:13.357722+0000","last_became_active":"2024-09-12T12:11:12.501095+0000","last_became_peered":"2024-09-12T12:11:12.501095+0000","last_unstale":"2024-09-12T12:13:13.357722+0000","last_undegraded":"2024-09-12T12:13:13.357722+0000","last_fullsized":"2024-09-12T12:13:13.357722+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:21.066843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464553+0000","last_change":"2024-09-12T12:11:09.486701+0000","last_active":"2024-09-12T12:13:13.464553+0000","last_peered":"2024-09-12T12:13:13.464553+0000","last_clean":"2024-09-12T12:13:13.464553+0000","last_became_active":"2024-09-12T12:11:09.486299+0000","last_became_peered":"2024-09-12T12:11:09.486299+0000","last_unstale":"2024-09-12T12:13:13.464553+0000","last_undegraded":"2024-09-12T12:13:13.464553+0000","last_fullsized":"2024-09-12T12:13:13.464553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:49.821093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508794+0000","last_change":"2024-09-12T12:11:12.503605+0000","last_active":"2024-09-12T12:13:13.508794+0000","last_peered":"2024-09-12T12:13:13.508794+0000","last_clean":"2024-09-12T12:13:13.508794+0000","last_became_active":"2024-09-12T12:11:12.503253+0000","last_became_peered":"2024-09-12T12:11:12.503253+0000","last_unstale":"2024-09-12T12:13:13.508794+0000","last_undegraded":"2024-09-12T12:13:13.508794+0000","last_fullsized":"2024-09-12T12:13:13.508794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:58.147676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'8","reported_seq":157,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:17.625221+0000","last_change":"2024-09-12T12:11:09.485901+0000","last_active":"2024-09-12T12:13:17.625221+0000","last_peered":"2024-09-12T12:13:17.625221+0000","last_clean":"2024-09-12T12:13:17.625221+0000","last_became_active":"2024-09-12T12:11:09.485484+0000","last_became_peered":"2024-09-12T12:11:09.485484+0000","last_unstale":"2024-09-12T12:13:17.625221+0000","last_undegraded":"2024-09-12T12:13:17.625221+0000","last_fullsized":"2024-09-12T12:13:17.625221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:27:27.958043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":7,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508424+0000","last_change":"2024-09-12T12:12:17.031317+0000","last_active":"2024-09-12T12:13:13.508424+0000","last_peered":"2024-09-12T12:13:13.508424+0000","last_clean":"2024-09-12T12:13:13.508424+0000","last_became_active":"2024-09-12T12:12:17.030842+0000","last_became_peered":"2024-09-12T12:12:17.030842+0000","last_unstale":"2024-09-12T12:13:13.508424+0000","last_undegraded":"2024-09-12T12:13:13.508424+0000","last_fullsized":"2024-09-12T12:13:13.508424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:44:11.616682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456456+0000","last_change":"2024-09-12T12:11:09.480378+0000","last_active":"2024-09-12T12:13:13.456456+0000","last_peered":"2024-09-12T12:13:13.456456+0000","last_clean":"2024-09-12T12:13:13.456456+0000","last_became_active":"2024-09-12T12:11:09.479838+0000","last_became_peered":"2024-09-12T12:11:09.479838+0000","last_unstale":"2024-09-12T12:13:13.456456+0000","last_undegraded":"2024-09-12T12:13:13.456456+0000","last_fullsized":"2024-09-12T12:13:13.456456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:00:53.593385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464470+0000","last_change":"2024-09-12T12:11:12.500935+0000","last_active":"2024-09-12T12:13:13.464470+0000","last_peered":"2024-09-12T12:13:13.464470+0000","last_clean":"2024-09-12T12:13:13.464470+0000","last_became_active":"2024-09-12T12:11:12.500534+0000","last_became_peered":"2024-09-12T12:11:12.500534+0000","last_unstale":"2024-09-12T12:13:13.464470+0000","last_undegraded":"2024-09-12T12:13:13.464470+0000","last_fullsized":"2024-09-12T12:13:13.464470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:05:25.613368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456886+0000","last_change":"2024-09-12T12:11:09.481106+0000","last_active":"2024-09-12T12:13:13.456886+0000","last_peered":"2024-09-12T12:13:13.456886+0000","last_clean":"2024-09-12T12:13:13.456886+0000","last_became_active":"2024-09-12T12:11:09.480726+0000","last_became_peered":"2024-09-12T12:11:09.480726+0000","last_unstale":"2024-09-12T12:13:13.456886+0000","last_undegraded":"2024-09-12T12:13:13.456886+0000","last_fullsized":"2024-09-12T12:13:13.456886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:43:32.584985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508769+0000","last_change":"2024-09-12T12:11:12.503436+0000","last_active":"2024-09-12T12:13:13.508769+0000","last_peered":"2024-09-12T12:13:13.508769+0000","last_clean":"2024-09-12T12:13:13.508769+0000","last_became_active":"2024-09-12T12:11:12.503110+0000","last_became_peered":"2024-09-12T12:11:12.503110+0000","last_unstale":"2024-09-12T12:13:13.508769+0000","last_undegraded":"2024-09-12T12:13:13.508769+0000","last_fullsized":"2024-09-12T12:13:13.508769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:34:04.488795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.358004+0000","last_change":"2024-09-12T12:11:09.477349+0000","last_active":"2024-09-12T12:13:13.358004+0000","last_peered":"2024-09-12T12:13:13.358004+0000","last_clean":"2024-09-12T12:13:13.358004+0000","last_became_active":"2024-09-12T12:11:09.476971+0000","last_became_peered":"2024-09-12T12:11:09.476971+0000","last_unstale":"2024-09-12T12:13:13.358004+0000","last_undegraded":"2024-09-12T12:13:13.358004+0000","last_fullsized":"2024-09-12T12:13:13.358004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:59.137541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508965+0000","last_change":"2024-09-12T12:11:12.506512+0000","last_active":"2024-09-12T12:13:13.508965+0000","last_peered":"2024-09-12T12:13:13.508965+0000","last_clean":"2024-09-12T12:13:13.508965+0000","last_became_active":"2024-09-12T12:11:12.506157+0000","last_became_peered":"2024-09-12T12:11:12.506157+0000","last_unstale":"2024-09-12T12:13:13.508965+0000","last_undegraded":"2024-09-12T12:13:13.508965+0000","last_fullsized":"2024-09-12T12:13:13.508965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:16:33.121276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'65","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:15.140569+0000","last_change":"2024-09-12T12:11:09.483498+0000","last_active":"2024-09-12T12:13:15.140569+0000","last_peered":"2024-09-12T12:13:15.140569+0000","last_clean":"2024-09-12T12:13:15.140569+0000","last_became_active":"2024-09-12T12:11:09.483162+0000","last_became_peered":"2024-09-12T12:11:09.483162+0000","last_unstale":"2024-09-12T12:13:15.140569+0000","last_undegraded":"2024-09-12T12:13:15.140569+0000","last_fullsized":"2024-09-12T12:13:15.140569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:57:36.606612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508595+0000","last_change":"2024-09-12T12:11:12.502633+0000","last_active":"2024-09-12T12:13:13.508595+0000","last_peered":"2024-09-12T12:13:13.508595+0000","last_clean":"2024-09-12T12:13:13.508595+0000","last_became_active":"2024-09-12T12:11:12.502280+0000","last_became_peered":"2024-09-12T12:11:12.502280+0000","last_unstale":"2024-09-12T12:13:13.508595+0000","last_undegraded":"2024-09-12T12:13:13.508595+0000","last_fullsized":"2024-09-12T12:13:13.508595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:51:51.064202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.359543+0000","last_change":"2024-09-12T12:11:10.026566+0000","last_active":"2024-09-12T12:13:13.359543+0000","last_peered":"2024-09-12T12:13:13.359543+0000","last_clean":"2024-09-12T12:13:13.359543+0000","last_became_active":"2024-09-12T12:11:10.026001+0000","last_became_peered":"2024-09-12T12:11:10.026001+0000","last_unstale":"2024-09-12T12:13:13.359543+0000","last_undegraded":"2024-09-12T12:13:13.359543+0000","last_fullsized":"2024-09-12T12:13:13.359543+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:24:44.792863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508748+0000","last_change":"2024-09-12T12:11:12.506386+0000","last_active":"2024-09-12T12:13:13.508748+0000","last_peered":"2024-09-12T12:13:13.508748+0000","last_clean":"2024-09-12T12:13:13.508748+0000","last_became_active":"2024-09-12T12:11:12.506066+0000","last_became_peered":"2024-09-12T12:11:12.506066+0000","last_unstale":"2024-09-12T12:13:13.508748+0000","last_undegraded":"2024-09-12T12:13:13.508748+0000","last_fullsized":"2024-09-12T12:13:13.508748+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:36:51.723989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"85'148","reported_seq":394,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:15.472327+0000","last_change":"2024-09-12T12:09:56.345033+0000","last_active":"2024-09-12T12:13:15.472327+0000","last_peered":"2024-09-12T12:13:15.472327+0000","last_clean":"2024-09-12T12:13:15.472327+0000","last_became_active":"2024-09-12T12:09:56.344531+0000","last_became_peered":"2024-09-12T12:09:56.344531+0000","last_unstale":"2024-09-12T12:13:15.472327+0000","last_undegraded":"2024-09-12T12:13:15.472327+0000","last_fullsized":"2024-09-12T12:13:15.472327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:06:29.739353+0000","last_clean_scrub_stamp":"2024-09-12T12:06:29.739353+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:33:17.680663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509280+0000","last_change":"2024-09-12T12:11:09.483389+0000","last_active":"2024-09-12T12:13:13.509280+0000","last_peered":"2024-09-12T12:13:13.509280+0000","last_clean":"2024-09-12T12:13:13.509280+0000","last_became_active":"2024-09-12T12:11:09.483015+0000","last_became_peered":"2024-09-12T12:11:09.483015+0000","last_unstale":"2024-09-12T12:13:13.509280+0000","last_undegraded":"2024-09-12T12:13:13.509280+0000","last_fullsized":"2024-09-12T12:13:13.509280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:26:08.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457494+0000","last_change":"2024-09-12T12:11:12.499342+0000","last_active":"2024-09-12T12:13:13.457494+0000","last_peered":"2024-09-12T12:13:13.457494+0000","last_clean":"2024-09-12T12:13:13.457494+0000","last_became_active":"2024-09-12T12:11:12.499029+0000","last_became_peered":"2024-09-12T12:11:12.499029+0000","last_unstale":"2024-09-12T12:13:13.457494+0000","last_undegraded":"2024-09-12T12:13:13.457494+0000","last_fullsized":"2024-09-12T12:13:13.457494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T21:09:28.174332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508614+0000","last_change":"2024-09-12T12:11:09.482475+0000","last_active":"2024-09-12T12:13:13.508614+0000","last_peered":"2024-09-12T12:13:13.508614+0000","last_clean":"2024-09-12T12:13:13.508614+0000","last_became_active":"2024-09-12T12:11:09.482161+0000","last_became_peered":"2024-09-12T12:11:09.482161+0000","last_unstale":"2024-09-12T12:13:13.508614+0000","last_undegraded":"2024-09-12T12:13:13.508614+0000","last_fullsized":"2024-09-12T12:13:13.508614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:37.178868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357651+0000","last_change":"2024-09-12T12:11:12.503018+0000","last_active":"2024-09-12T12:13:13.357651+0000","last_peered":"2024-09-12T12:13:13.357651+0000","last_clean":"2024-09-12T12:13:13.357651+0000","last_became_active":"2024-09-12T12:11:12.502749+0000","last_became_peered":"2024-09-12T12:11:12.502749+0000","last_unstale":"2024-09-12T12:13:13.357651+0000","last_undegraded":"2024-09-12T12:13:13.357651+0000","last_fullsized":"2024-09-12T12:13:13.357651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:43:06.704541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":150,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:17.624646+0000","last_change":"2024-09-12T12:11:09.477867+0000","last_active":"2024-09-12T12:13:17.624646+0000","last_peered":"2024-09-12T12:13:17.624646+0000","last_clean":"2024-09-12T12:13:17.624646+0000","last_became_active":"2024-09-12T12:11:09.477446+0000","last_became_peered":"2024-09-12T12:11:09.477446+0000","last_unstale":"2024-09-12T12:13:17.624646+0000","last_undegraded":"2024-09-12T12:13:17.624646+0000","last_fullsized":"2024-09-12T12:13:17.624646+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:58.098073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464453+0000","last_change":"2024-09-12T12:11:12.500325+0000","last_active":"2024-09-12T12:13:13.464453+0000","last_peered":"2024-09-12T12:13:13.464453+0000","last_clean":"2024-09-12T12:13:13.464453+0000","last_became_active":"2024-09-12T12:11:12.499910+0000","last_became_peered":"2024-09-12T12:11:12.499910+0000","last_unstale":"2024-09-12T12:13:13.464453+0000","last_undegraded":"2024-09-12T12:13:13.464453+0000","last_fullsized":"2024-09-12T12:13:13.464453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:30:33.772368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508667+0000","last_change":"2024-09-12T12:11:09.486753+0000","last_active":"2024-09-12T12:13:13.508667+0000","last_peered":"2024-09-12T12:13:13.508667+0000","last_clean":"2024-09-12T12:13:13.508667+0000","last_became_active":"2024-09-12T12:11:09.484950+0000","last_became_peered":"2024-09-12T12:11:09.484950+0000","last_unstale":"2024-09-12T12:13:13.508667+0000","last_undegraded":"2024-09-12T12:13:13.508667+0000","last_fullsized":"2024-09-12T12:13:13.508667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:18:55.873980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470385+0000","last_change":"2024-09-12T12:11:12.501054+0000","last_active":"2024-09-12T12:13:13.470385+0000","last_peered":"2024-09-12T12:13:13.470385+0000","last_clean":"2024-09-12T12:13:13.470385+0000","last_became_active":"2024-09-12T12:11:12.500379+0000","last_became_peered":"2024-09-12T12:11:12.500379+0000","last_unstale":"2024-09-12T12:13:13.470385+0000","last_undegraded":"2024-09-12T12:13:13.470385+0000","last_fullsized":"2024-09-12T12:13:13.470385+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:54:14.463543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454372+0000","last_change":"2024-09-12T12:11:09.477858+0000","last_active":"2024-09-12T12:13:13.454372+0000","last_peered":"2024-09-12T12:13:13.454372+0000","last_clean":"2024-09-12T12:13:13.454372+0000","last_became_active":"2024-09-12T12:11:09.477514+0000","last_became_peered":"2024-09-12T12:11:09.477514+0000","last_unstale":"2024-09-12T12:13:13.454372+0000","last_undegraded":"2024-09-12T12:13:13.454372+0000","last_fullsized":"2024-09-12T12:13:13.454372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:27:48.849839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.470318+0000","last_change":"2024-09-12T12:11:12.495694+0000","last_active":"2024-09-12T12:13:13.470318+0000","last_peered":"2024-09-12T12:13:13.470318+0000","last_clean":"2024-09-12T12:13:13.470318+0000","last_became_active":"2024-09-12T12:11:12.495343+0000","last_became_peered":"2024-09-12T12:11:12.495343+0000","last_unstale":"2024-09-12T12:13:13.470318+0000","last_undegraded":"2024-09-12T12:13:13.470318+0000","last_fullsized":"2024-09-12T12:13:13.470318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:24:31.495653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456892+0000","last_change":"2024-09-12T12:11:09.480301+0000","last_active":"2024-09-12T12:13:13.456892+0000","last_peered":"2024-09-12T12:13:13.456892+0000","last_clean":"2024-09-12T12:13:13.456892+0000","last_became_active":"2024-09-12T12:11:09.479831+0000","last_became_peered":"2024-09-12T12:11:09.479831+0000","last_unstale":"2024-09-12T12:13:13.456892+0000","last_undegraded":"2024-09-12T12:13:13.456892+0000","last_fullsized":"2024-09-12T12:13:13.456892+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:06:43.965593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.357958+0000","last_change":"2024-09-12T12:11:12.499322+0000","last_active":"2024-09-12T12:13:13.357958+0000","last_peered":"2024-09-12T12:13:13.357958+0000","last_clean":"2024-09-12T12:13:13.357958+0000","last_became_active":"2024-09-12T12:11:12.498976+0000","last_became_peered":"2024-09-12T12:11:12.498976+0000","last_unstale":"2024-09-12T12:13:13.357958+0000","last_undegraded":"2024-09-12T12:13:13.357958+0000","last_fullsized":"2024-09-12T12:13:13.357958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:24:36.454343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'47","reported_seq":96,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.598676+0000","last_change":"2024-09-12T12:11:09.476650+0000","last_active":"2024-09-12T12:13:13.598676+0000","last_peered":"2024-09-12T12:13:13.598676+0000","last_clean":"2024-09-12T12:13:13.598676+0000","last_became_active":"2024-09-12T12:11:09.476330+0000","last_became_peered":"2024-09-12T12:11:09.476330+0000","last_unstale":"2024-09-12T12:13:13.598676+0000","last_undegraded":"2024-09-12T12:13:13.598676+0000","last_fullsized":"2024-09-12T12:13:13.598676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:11:31.373055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454442+0000","last_change":"2024-09-12T12:12:17.032314+0000","last_active":"2024-09-12T12:13:13.454442+0000","last_peered":"2024-09-12T12:13:13.454442+0000","last_clean":"2024-09-12T12:13:13.454442+0000","last_became_active":"2024-09-12T12:12:17.032022+0000","last_became_peered":"2024-09-12T12:12:17.032022+0000","last_unstale":"2024-09-12T12:13:13.454442+0000","last_undegraded":"2024-09-12T12:13:13.454442+0000","last_fullsized":"2024-09-12T12:13:13.454442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T23:01:55.111688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.509481+0000","last_change":"2024-09-12T12:11:10.025967+0000","last_active":"2024-09-12T12:13:13.509481+0000","last_peered":"2024-09-12T12:13:13.509481+0000","last_clean":"2024-09-12T12:13:13.509481+0000","last_became_active":"2024-09-12T12:11:10.025638+0000","last_became_peered":"2024-09-12T12:11:10.025638+0000","last_unstale":"2024-09-12T12:13:13.509481+0000","last_undegraded":"2024-09-12T12:13:13.509481+0000","last_fullsized":"2024-09-12T12:13:13.509481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:18:09.941692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511235+0000","last_change":"2024-09-12T12:11:12.500202+0000","last_active":"2024-09-12T12:13:13.511235+0000","last_peered":"2024-09-12T12:13:13.511235+0000","last_clean":"2024-09-12T12:13:13.511235+0000","last_became_active":"2024-09-12T12:11:12.499886+0000","last_became_peered":"2024-09-12T12:11:12.499886+0000","last_unstale":"2024-09-12T12:13:13.511235+0000","last_undegraded":"2024-09-12T12:13:13.511235+0000","last_fullsized":"2024-09-12T12:13:13.511235+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:29:26.695365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360305+0000","last_change":"2024-09-12T12:11:09.483170+0000","last_active":"2024-09-12T12:13:13.360305+0000","last_peered":"2024-09-12T12:13:13.360305+0000","last_clean":"2024-09-12T12:13:13.360305+0000","last_became_active":"2024-09-12T12:11:09.480827+0000","last_became_peered":"2024-09-12T12:11:09.480827+0000","last_unstale":"2024-09-12T12:13:13.360305+0000","last_undegraded":"2024-09-12T12:13:13.360305+0000","last_fullsized":"2024-09-12T12:13:13.360305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:56.059557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456450+0000","last_change":"2024-09-12T12:11:12.492558+0000","last_active":"2024-09-12T12:13:13.456450+0000","last_peered":"2024-09-12T12:13:13.456450+0000","last_clean":"2024-09-12T12:13:13.456450+0000","last_became_active":"2024-09-12T12:11:12.492220+0000","last_became_peered":"2024-09-12T12:11:12.492220+0000","last_unstale":"2024-09-12T12:13:13.456450+0000","last_undegraded":"2024-09-12T12:13:13.456450+0000","last_fullsized":"2024-09-12T12:13:13.456450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:10:23.079219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457291+0000","last_change":"2024-09-12T12:11:09.480119+0000","last_active":"2024-09-12T12:13:13.457291+0000","last_peered":"2024-09-12T12:13:13.457291+0000","last_clean":"2024-09-12T12:13:13.457291+0000","last_became_active":"2024-09-12T12:11:09.479556+0000","last_became_peered":"2024-09-12T12:11:09.479556+0000","last_unstale":"2024-09-12T12:13:13.457291+0000","last_undegraded":"2024-09-12T12:13:13.457291+0000","last_fullsized":"2024-09-12T12:13:13.457291+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:33:07.068081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508548+0000","last_change":"2024-09-12T12:11:12.499788+0000","last_active":"2024-09-12T12:13:13.508548+0000","last_peered":"2024-09-12T12:13:13.508548+0000","last_clean":"2024-09-12T12:13:13.508548+0000","last_became_active":"2024-09-12T12:11:12.498181+0000","last_became_peered":"2024-09-12T12:11:12.498181+0000","last_unstale":"2024-09-12T12:13:13.508548+0000","last_undegraded":"2024-09-12T12:13:13.508548+0000","last_fullsized":"2024-09-12T12:13:13.508548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:43:56.123897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508524+0000","last_change":"2024-09-12T12:12:17.038499+0000","last_active":"2024-09-12T12:13:13.508524+0000","last_peered":"2024-09-12T12:13:13.508524+0000","last_clean":"2024-09-12T12:13:13.508524+0000","last_became_active":"2024-09-12T12:12:17.034622+0000","last_became_peered":"2024-09-12T12:12:17.034622+0000","last_unstale":"2024-09-12T12:13:13.508524+0000","last_undegraded":"2024-09-12T12:13:13.508524+0000","last_fullsized":"2024-09-12T12:13:13.508524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T18:06:30.684068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.457597+0000","last_change":"2024-09-12T12:11:12.499334+0000","last_active":"2024-09-12T12:13:13.457597+0000","last_peered":"2024-09-12T12:13:13.457597+0000","last_clean":"2024-09-12T12:13:13.457597+0000","last_became_active":"2024-09-12T12:11:12.498046+0000","last_became_peered":"2024-09-12T12:11:12.498046+0000","last_unstale":"2024-09-12T12:13:13.457597+0000","last_undegraded":"2024-09-12T12:13:13.457597+0000","last_fullsized":"2024-09-12T12:13:13.457597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:51:24.665580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508525+0000","last_change":"2024-09-12T12:12:17.041616+0000","last_active":"2024-09-12T12:13:13.508525+0000","last_peered":"2024-09-12T12:13:13.508525+0000","last_clean":"2024-09-12T12:13:13.508525+0000","last_became_active":"2024-09-12T12:12:17.035895+0000","last_became_peered":"2024-09-12T12:12:17.035895+0000","last_unstale":"2024-09-12T12:13:13.508525+0000","last_undegraded":"2024-09-12T12:13:13.508525+0000","last_fullsized":"2024-09-12T12:13:13.508525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:34:11.722682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.508538+0000","last_change":"2024-09-12T12:11:12.503682+0000","last_active":"2024-09-12T12:13:13.508538+0000","last_peered":"2024-09-12T12:13:13.508538+0000","last_clean":"2024-09-12T12:13:13.508538+0000","last_became_active":"2024-09-12T12:11:12.503326+0000","last_became_peered":"2024-09-12T12:11:12.503326+0000","last_unstale":"2024-09-12T12:13:13.508538+0000","last_undegraded":"2024-09-12T12:13:13.508538+0000","last_fullsized":"2024-09-12T12:13:13.508538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:11:19.586855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.464560+0000","last_change":"2024-09-12T12:11:12.507249+0000","last_active":"2024-09-12T12:13:13.464560+0000","last_peered":"2024-09-12T12:13:13.464560+0000","last_clean":"2024-09-12T12:13:13.464560+0000","last_became_active":"2024-09-12T12:11:12.506907+0000","last_became_peered":"2024-09-12T12:11:12.506907+0000","last_unstale":"2024-09-12T12:13:13.464560+0000","last_undegraded":"2024-09-12T12:13:13.464560+0000","last_fullsized":"2024-09-12T12:13:13.464560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:37:39.610634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456728+0000","last_change":"2024-09-12T12:11:09.477123+0000","last_active":"2024-09-12T12:13:13.456728+0000","last_peered":"2024-09-12T12:13:13.456728+0000","last_clean":"2024-09-12T12:13:13.456728+0000","last_became_active":"2024-09-12T12:11:09.476798+0000","last_became_peered":"2024-09-12T12:11:09.476798+0000","last_unstale":"2024-09-12T12:13:13.456728+0000","last_undegraded":"2024-09-12T12:13:13.456728+0000","last_fullsized":"2024-09-12T12:13:13.456728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T16:08:09.312263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.360132+0000","last_change":"2024-09-12T12:11:12.498772+0000","last_active":"2024-09-12T12:13:13.360132+0000","last_peered":"2024-09-12T12:13:13.360132+0000","last_clean":"2024-09-12T12:13:13.360132+0000","last_became_active":"2024-09-12T12:11:12.498407+0000","last_became_peered":"2024-09-12T12:11:12.498407+0000","last_unstale":"2024-09-12T12:13:13.360132+0000","last_undegraded":"2024-09-12T12:13:13.360132+0000","last_fullsized":"2024-09-12T12:13:13.360132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T19:40:05.072037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.456757+0000","last_change":"2024-09-12T12:11:09.484955+0000","last_active":"2024-09-12T12:13:13.456757+0000","last_peered":"2024-09-12T12:13:13.456757+0000","last_clean":"2024-09-12T12:13:13.456757+0000","last_became_active":"2024-09-12T12:11:09.484607+0000","last_became_peered":"2024-09-12T12:11:09.484607+0000","last_unstale":"2024-09-12T12:13:13.456757+0000","last_undegraded":"2024-09-12T12:13:13.456757+0000","last_fullsized":"2024-09-12T12:13:13.456757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:46:03.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.511549+0000","last_change":"2024-09-12T12:11:12.503424+0000","last_active":"2024-09-12T12:13:13.511549+0000","last_peered":"2024-09-12T12:13:13.511549+0000","last_clean":"2024-09-12T12:13:13.511549+0000","last_became_active":"2024-09-12T12:11:12.501441+0000","last_became_peered":"2024-09-12T12:11:12.501441+0000","last_unstale":"2024-09-12T12:13:13.511549+0000","last_undegraded":"2024-09-12T12:13:13.511549+0000","last_fullsized":"2024-09-12T12:13:13.511549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T22:15:23.688023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458550+0000","last_change":"2024-09-12T12:11:09.478017+0000","last_active":"2024-09-12T12:13:13.458550+0000","last_peered":"2024-09-12T12:13:13.458550+0000","last_clean":"2024-09-12T12:13:13.458550+0000","last_became_active":"2024-09-12T12:11:09.477390+0000","last_became_peered":"2024-09-12T12:11:09.477390+0000","last_unstale":"2024-09-12T12:13:13.458550+0000","last_undegraded":"2024-09-12T12:13:13.458550+0000","last_fullsized":"2024-09-12T12:13:13.458550+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:48:34.748587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.458311+0000","last_change":"2024-09-12T12:11:12.504774+0000","last_active":"2024-09-12T12:13:13.458311+0000","last_peered":"2024-09-12T12:13:13.458311+0000","last_clean":"2024-09-12T12:13:13.458311+0000","last_became_active":"2024-09-12T12:11:12.503787+0000","last_became_peered":"2024-09-12T12:11:12.503787+0000","last_unstale":"2024-09-12T12:13:13.458311+0000","last_undegraded":"2024-09-12T12:13:13.458311+0000","last_fullsized":"2024-09-12T12:13:13.458311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:11.470920+0000","last_clean_scrub_stamp":"2024-09-12T12:11:11.470920+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T17:49:16.311336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-12T12:13:13.454620+0000","last_change":"2024-09-12T12:11:09.480414+0000","last_active":"2024-09-12T12:13:13.454620+0000","last_peered":"2024-09-12T12:13:13.454620+0000","last_clean":"2024-09-12T12:13:13.454620+0000","last_became_active":"2024-09-12T12:11:09.480059+0000","last_became_peered":"2024-09-12T12:11:09.480059+0000","last_unstale":"2024-09-12T12:13:13.454620+0000","last_undegraded":"2024-09-12T12:13:13.454620+0000","last_fullsized":"2024-09-12T12:13:13.454620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T12:11:08.458251+0000","last_clean_scrub_stamp":"2024-09-12T12:11:08.458251+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T20:28:46.683075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":793006,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":833,"num_read_kb":532,"num_write":317,"num_write_kb":1902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":614400,"data_stored":2182386,"data_compressed":94284,"data_compressed_allocated":122880,"data_compressed_original":1916928,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107349,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":453099,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":1243735,"data_compressed":44842,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":962560,"data_stored":846551,"data_compressed":22049,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168631,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":846471,"data_compressed":22192,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397564,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1046815,"data_compressed":38686,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":452724,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":1373103,"data_compressed":53888,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":454789,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":1174483,"data_compressed":37394,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1015808,"data_stored":520187,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":518444,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":453995,"data_compressed":2903,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35783,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330101,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200733,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T12:13:21.003 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:13:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:20 smithi189 ceph-mon[35475]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 93 KiB/s wr, 38 op/s 2024-09-12T12:13:21.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:20 smithi189 ceph-mon[35475]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T12:13:21.250 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:21.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.771+0000 7f497eb14640 1 -- 172.21.15.137:0/4202717583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978101420 msgr2=0x7f497810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:21.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.771+0000 7f497eb14640 1 --2- 172.21.15.137:0/4202717583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978101420 0x7f497810d950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4960009930 tx=0x7f496002a420 comp rx=0 tx=0).stop 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- 172.21.15.137:0/4202717583 shutdown_connections 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 --2- 172.21.15.137:0/4202717583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978101420 0x7f497810d950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 --2- 172.21.15.137:0/4202717583 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4978100a80 0x7f4978100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 --2- 172.21.15.137:0/4202717583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4978106a80 0x7f4978106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- 172.21.15.137:0/4202717583 >> 172.21.15.137:0/4202717583 conn(0x7f49780fc7a0 msgr2=0x7f49780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- 172.21.15.137:0/4202717583 shutdown_connections 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- 172.21.15.137:0/4202717583 wait complete. 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 Processor -- start 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- start start 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4978100a80 0x7f4978199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4978101420 0x7f4978199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:21.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978106a80 0x7f49781a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4978110e80 con 0x7f4978106a80 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4978110d00 con 0x7f4978101420 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.772+0000 7f497eb14640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4978111000 con 0x7f4978100a80 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978106a80 0x7f49781a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978106a80 0x7f49781a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51590/0 (socket says 172.21.15.137:51590) 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 -- 172.21.15.137:0/802623760 learned_addr learned my addr 172.21.15.137:0/802623760 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497c8b0640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4978100a80 0x7f4978199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f4977fff640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4978101420 0x7f4978199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 -- 172.21.15.137:0/802623760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4978100a80 msgr2=0x7f4978199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4978100a80 0x7f4978199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 -- 172.21.15.137:0/802623760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4978101420 msgr2=0x7f4978199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:21.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4978101420 0x7f4978199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 -- 172.21.15.137:0/802623760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49781a4800 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497d0b1640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978106a80 0x7f49781a42c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4960005ab0 tx=0x7f49600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f4975ffb640 1 -- 172.21.15.137:0/802623760 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496003f030 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f4975ffb640 1 -- 172.21.15.137:0/802623760 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4960003ce0 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49781a49d0 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f4975ffb640 1 -- 172.21.15.137:0/802623760 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4960035070 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.773+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49781a4eb0 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.774+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4978100ee0 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.776+0000 7f4975ffb640 1 -- 172.21.15.137:0/802623760 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f4960042030 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.777+0000 7f4975ffb640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49640a2e00 0x7f49640a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.777+0000 7f497c8b0640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49640a2e00 0x7f49640a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.777+0000 7f4975ffb640 1 -- 172.21.15.137:0/802623760 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f49600efea0 con 0x7f4978106a80 2024-09-12T12:13:21.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.777+0000 7f497c8b0640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49640a2e00 0x7f49640a52c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49700058c0 tx=0x7f497000a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:21.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.780+0000 7f4975ffb640 1 -- 172.21.15.137:0/802623760 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49600bc9a0 con 0x7f4978106a80 2024-09-12T12:13:21.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.979+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4978109010 con 0x7f4978106a80 2024-09-12T12:13:21.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.980+0000 7f4975ffb640 1 -- 172.21.15.137:0/802623760 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7f49600bc2a0 con 0x7f4978106a80 2024-09-12T12:13:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:21.981 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"btime":"2024-09-12T12:13:12:456388+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:13:12.406179+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:13:21.981 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-09-12T12:13:21.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49640a2e00 msgr2=0x7f49640a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:21.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49640a2e00 0x7f49640a52c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f49700058c0 tx=0x7f497000a2d0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978106a80 msgr2=0x7f49781a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:21.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978106a80 0x7f49781a42c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4960005ab0 tx=0x7f49600042f0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 shutdown_connections 2024-09-12T12:13:21.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49640a2e00 0x7f49640a52c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4978106a80 0x7f49781a42c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4978101420 0x7f4978199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 --2- 172.21.15.137:0/802623760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4978100a80 0x7f4978199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:21.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.983+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 >> 172.21.15.137:0/802623760 conn(0x7f49780fc7a0 msgr2=0x7f49780fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:21.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.984+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 shutdown_connections 2024-09-12T12:13:21.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:21.984+0000 7f497eb14640 1 -- 172.21.15.137:0/802623760 wait complete. 2024-09-12T12:13:22.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:13:22.919 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:22.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:22 smithi171 ceph-mon[35098]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-09-12T12:13:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:22 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/802623760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:23.040 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:22 smithi137 ceph-mon[30950]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-09-12T12:13:23.040 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:22 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/802623760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:22 smithi189 ceph-mon[35475]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 131 KiB/s wr, 76 op/s 2024-09-12T12:13:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:22 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/802623760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:23.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.489+0000 7efe5922e640 1 -- 172.21.15.137:0/1855211671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe541002f0 msgr2=0x7efe541006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.489+0000 7efe5922e640 1 --2- 172.21.15.137:0/1855211671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe541002f0 0x7efe541006d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efe44009930 tx=0x7efe4402a420 comp rx=0 tx=0).stop 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.489+0000 7efe5922e640 1 -- 172.21.15.137:0/1855211671 shutdown_connections 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.489+0000 7efe5922e640 1 --2- 172.21.15.137:0/1855211671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54106f50 0x7efe5410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.489+0000 7efe5922e640 1 --2- 172.21.15.137:0/1855211671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe54100ca0 0x7efe54106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.489+0000 7efe5922e640 1 --2- 172.21.15.137:0/1855211671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe541002f0 0x7efe541006d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.489+0000 7efe5922e640 1 -- 172.21.15.137:0/1855211671 >> 172.21.15.137:0/1855211671 conn(0x7efe54077210 msgr2=0x7efe540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.490+0000 7efe5922e640 1 -- 172.21.15.137:0/1855211671 shutdown_connections 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.490+0000 7efe5922e640 1 -- 172.21.15.137:0/1855211671 wait complete. 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.490+0000 7efe5922e640 1 Processor -- start 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.490+0000 7efe5922e640 1 -- start start 2024-09-12T12:13:23.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.490+0000 7efe5922e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe541002f0 0x7efe54101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.490+0000 7efe5922e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54100ca0 0x7efe541017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe5922e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54106f50 0x7efe54101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe5922e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe54112000 con 0x7efe54100ca0 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe5922e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111e80 con 0x7efe54106f50 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe5922e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe54112180 con 0x7efe541002f0 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54100ca0 0x7efe541017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54100ca0 0x7efe541017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51608/0 (socket says 172.21.15.137:51608) 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 -- 172.21.15.137:0/1873269422 learned_addr learned my addr 172.21.15.137:0/1873269422 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe52d9d640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe541002f0 0x7efe54101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 -- 172.21.15.137:0/1873269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe541002f0 msgr2=0x7efe54101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe5359e640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54106f50 0x7efe54101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe541002f0 0x7efe54101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 -- 172.21.15.137:0/1873269422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54106f50 msgr2=0x7efe54101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54106f50 0x7efe54101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 -- 172.21.15.137:0/1873269422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541a6a10 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe4bfff640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54100ca0 0x7efe541017b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c970 tx=0x7efe3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe50d99640 1 -- 172.21.15.137:0/1873269422 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c007bb0 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe50d99640 1 -- 172.21.15.137:0/1873269422 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c007d50 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe50d99640 1 -- 172.21.15.137:0/1873269422 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c00f450 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541a6d00 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.491+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a7230 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.493+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe541006d0 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.493+0000 7efe50d99640 1 -- 172.21.15.137:0/1873269422 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7efe3c0046c0 con 0x7efe54100ca0 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.494+0000 7efe50d99640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2e00 0x7efe400a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.494+0000 7efe52d9d640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2e00 0x7efe400a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.495+0000 7efe50d99640 1 -- 172.21.15.137:0/1873269422 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7efe3c0cf570 con 0x7efe54100ca0 2024-09-12T12:13:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.495+0000 7efe52d9d640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2e00 0x7efe400a52c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe44009900 tx=0x7efe440058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.499+0000 7efe50d99640 1 -- 172.21.15.137:0/1873269422 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe3c014030 con 0x7efe54100ca0 2024-09-12T12:13:23.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.697+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe5410a140 con 0x7efe54100ca0 2024-09-12T12:13:23.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.698+0000 7efe50d99640 1 -- 172.21.15.137:0/1873269422 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7efe3c09c070 con 0x7efe54100ca0 2024-09-12T12:13:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:23.699 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"btime":"2024-09-12T12:13:12:456388+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:13:12.406179+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:13:23.699 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2e00 msgr2=0x7efe400a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2e00 0x7efe400a52c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efe44009900 tx=0x7efe440058d0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54100ca0 msgr2=0x7efe541017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54100ca0 0x7efe541017b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c970 tx=0x7efe3c00ce30 comp rx=0 tx=0).stop 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 shutdown_connections 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2e00 0x7efe400a52c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54106f50 0x7efe54101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54100ca0 0x7efe541017b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 --2- 172.21.15.137:0/1873269422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe541002f0 0x7efe54101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:23.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 >> 172.21.15.137:0/1873269422 conn(0x7efe54077210 msgr2=0x7efe541052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:23.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 shutdown_connections 2024-09-12T12:13:23.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:23.701+0000 7efe5922e640 1 -- 172.21.15.137:0/1873269422 wait complete. 2024-09-12T12:13:24.420 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:13:24.669 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:24.944 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:24 smithi137 ceph-mon[30950]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T12:13:24.944 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:24 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1873269422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:24.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:24 smithi171 ceph-mon[35098]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T12:13:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:24 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1873269422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:25.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:24 smithi189 ceph-mon[35475]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 59 op/s 2024-09-12T12:13:25.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:24 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1873269422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 -- 172.21.15.137:0/3337420924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 msgr2=0x7f291c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 --2- 172.21.15.137:0/3337420924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c1029b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2918009900 tx=0x7f291802a480 comp rx=0 tx=0).stop 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 -- 172.21.15.137:0/3337420924 shutdown_connections 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 --2- 172.21.15.137:0/3337420924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c102ef0 0x7f291c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 --2- 172.21.15.137:0/3337420924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c1029b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 --2- 172.21.15.137:0/3337420924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f291c100e20 0x7f291c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 -- 172.21.15.137:0/3337420924 >> 172.21.15.137:0/3337420924 conn(0x7f291c0fc7d0 msgr2=0x7f291c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:25.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 -- 172.21.15.137:0/3337420924 shutdown_connections 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 -- 172.21.15.137:0/3337420924 wait complete. 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.173+0000 7f292435a640 1 Processor -- start 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 -- start start 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c100e20 0x7f291c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f291c102ef0 0x7f291c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f291c110f80 con 0x7f291c102570 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f291c110e00 con 0x7f291c100e20 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f291c111100 con 0x7f291c102ef0 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51636/0 (socket says 172.21.15.137:51636) 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 -- 172.21.15.137:0/3295577636 learned_addr learned my addr 172.21.15.137:0/3295577636 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:25.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29220f6640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c100e20 0x7f291c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29228f7640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f291c102ef0 0x7f291c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 -- 172.21.15.137:0/3295577636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f291c102ef0 msgr2=0x7f291c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f291c102ef0 0x7f291c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 -- 172.21.15.137:0/3295577636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c100e20 msgr2=0x7f291c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c100e20 0x7f291c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 -- 172.21.15.137:0/3295577636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c19d480 con 0x7f291c102570 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29228f7640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f291c102ef0 0x7f291c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f29218f5640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c198a10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2918006010 tx=0x7f2918003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f290f7fe640 1 -- 172.21.15.137:0/3295577636 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291803f070 con 0x7f291c102570 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f290f7fe640 1 -- 172.21.15.137:0/3295577636 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2918031c30 con 0x7f291c102570 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.174+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c19d710 con 0x7f291c102570 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.175+0000 7f290f7fe640 1 -- 172.21.15.137:0/3295577636 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918035030 con 0x7f291c102570 2024-09-12T12:13:25.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.175+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1a5050 con 0x7f291c102570 2024-09-12T12:13:25.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.175+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291c1021b0 con 0x7f291c102570 2024-09-12T12:13:25.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.177+0000 7f290f7fe640 1 -- 172.21.15.137:0/3295577636 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f2918042070 con 0x7f291c102570 2024-09-12T12:13:25.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.177+0000 7f290f7fe640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29080a2d80 0x7f29080a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:25.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.177+0000 7f29220f6640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29080a2d80 0x7f29080a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:25.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.177+0000 7f290f7fe640 1 -- 172.21.15.137:0/3295577636 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f29180f0220 con 0x7f291c102570 2024-09-12T12:13:25.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.178+0000 7f29220f6640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29080a2d80 0x7f29080a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2910005fd0 tx=0x7f291000b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:25.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.180+0000 7f290f7fe640 1 -- 172.21.15.137:0/3295577636 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2918031de0 con 0x7f291c102570 2024-09-12T12:13:25.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.348+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f291c10a0a0 con 0x7f291c102570 2024-09-12T12:13:25.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.350+0000 7f290f7fe640 1 -- 172.21.15.137:0/3295577636 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23117 (secure 0 0 0) 0x7f29180bcd20 con 0x7f291c102570 2024-09-12T12:13:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:25.351 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":85,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:13:13.350351+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/4099842792":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/123979974":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/2550482799":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2039555152":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1789953576":"2024-09-13T12:13:13.350302+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1789953576":"2024-09-13T12:13:13.350302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:13:25.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29080a2d80 msgr2=0x7f29080a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29080a2d80 0x7f29080a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2910005fd0 tx=0x7f291000b390 comp rx=0 tx=0).stop 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 msgr2=0x7f291c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c198a10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2918006010 tx=0x7f2918003c10 comp rx=0 tx=0).stop 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 shutdown_connections 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29080a2d80 0x7f29080a5240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f291c102ef0 0x7f291c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f291c102570 0x7f291c198a10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 --2- 172.21.15.137:0/3295577636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c100e20 0x7f291c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 >> 172.21.15.137:0/3295577636 conn(0x7f291c0fc7d0 msgr2=0x7f291c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 shutdown_connections 2024-09-12T12:13:25.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:25.352+0000 7f292435a640 1 -- 172.21.15.137:0/3295577636 wait complete. 2024-09-12T12:13:25.917 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:25 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3295577636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:13:25.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:25 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3295577636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:13:26.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:25 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3295577636' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:13:26.106 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T12:13:26.354 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:26.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:26 smithi137 ceph-mon[30950]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-12T12:13:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:26 smithi171 ceph-mon[35098]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-12T12:13:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:26 smithi189 ceph-mon[35475]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-09-12T12:13:27.614 INFO:teuthology.orchestra.run.smithi137.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T12:13:27.615 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T12:13:27.615 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T12:13:27.615 DEBUG:tasks.quiescer:Yielding 2024-09-12T12:13:27.615 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T12:13:27.631 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:13:27.632 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T12:13:27.632 INFO:tasks.workunit:timeout=3h 2024-09-12T12:13:27.632 INFO:tasks.workunit:cleanup=True 2024-09-12T12:13:27.632 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:27.633 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:13:27.650 INFO:teuthology.orchestra.run.smithi137.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:27.650 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T12:13:27.650 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T12:13:27.651 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T12:13:27.651 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T12:13:27.651 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-09-12 12:11:31.238607096 +0000 2024-09-12T12:13:27.651 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-09-12 12:11:31.238607096 +0000 2024-09-12T12:13:27.651 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-09-12 12:13:11.727809577 +0000 2024-09-12T12:13:27.651 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-09-12 12:11:31.238607096 +0000 2024-09-12T12:13:27.651 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T12:13:27.651 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T12:13:27.723 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T12:13:27.780 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T12:13:27.889 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:28.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 -- 172.21.15.137:0/706347043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102ef0 msgr2=0x7ff84810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 --2- 172.21.15.137:0/706347043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102ef0 0x7ff84810da40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff840009ca0 tx=0x7ff84002a950 comp rx=0 tx=0).stop 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 -- 172.21.15.137:0/706347043 shutdown_connections 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 --2- 172.21.15.137:0/706347043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102ef0 0x7ff84810da40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 --2- 172.21.15.137:0/706347043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff848102570 0x7ff8481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 --2- 172.21.15.137:0/706347043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff848100e20 0x7ff848101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 -- 172.21.15.137:0/706347043 >> 172.21.15.137:0/706347043 conn(0x7ff8480fc7d0 msgr2=0x7ff8480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 -- 172.21.15.137:0/706347043 shutdown_connections 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.361+0000 7ff84ea56640 1 -- 172.21.15.137:0/706347043 wait complete. 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 Processor -- start 2024-09-12T12:13:28.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 -- start start 2024-09-12T12:13:28.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff848100e20 0x7ff848198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102570 0x7ff848198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff848102ef0 0x7ff84819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff848110f80 con 0x7ff848102570 2024-09-12T12:13:28.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff848110e00 con 0x7ff848102ef0 2024-09-12T12:13:28.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff84ea56640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff848111100 con 0x7ff848100e20 2024-09-12T12:13:28.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff8477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102570 0x7ff848198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.362+0000 7ff8477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102570 0x7ff848198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51654/0 (socket says 172.21.15.137:51654) 2024-09-12T12:13:28.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8477fe640 1 -- 172.21.15.137:0/3689992669 learned_addr learned my addr 172.21.15.137:0/3689992669 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:28.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff847fff640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff848100e20 0x7ff848198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff84cff3640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff848102ef0 0x7ff84819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8477fe640 1 -- 172.21.15.137:0/3689992669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff848100e20 msgr2=0x7ff848198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8477fe640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff848100e20 0x7ff848198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8477fe640 1 -- 172.21.15.137:0/3689992669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff848102ef0 msgr2=0x7ff84819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8477fe640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff848102ef0 0x7ff84819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8477fe640 1 -- 172.21.15.137:0/3689992669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84819d4e0 con 0x7ff848102570 2024-09-12T12:13:28.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8477fe640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102570 0x7ff848198a70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00d770 tx=0x7ff83c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8457fa640 1 -- 172.21.15.137:0/3689992669 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c017070 con 0x7ff848102570 2024-09-12T12:13:28.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8457fa640 1 -- 172.21.15.137:0/3689992669 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c005500 con 0x7ff848102570 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff8457fa640 1 -- 172.21.15.137:0/3689992669 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c004e00 con 0x7ff848102570 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84819d7d0 con 0x7ff848102570 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.363+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481a50b0 con 0x7ff848102570 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.364+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff848101200 con 0x7ff848102570 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.366+0000 7ff8457fa640 1 -- 172.21.15.137:0/3689992669 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7ff83c004fa0 con 0x7ff848102570 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.367+0000 7ff8457fa640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8300a2e00 0x7ff8300a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.367+0000 7ff847fff640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8300a2e00 0x7ff8300a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.367+0000 7ff8457fa640 1 -- 172.21.15.137:0/3689992669 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7ff83c0cee90 con 0x7ff848102570 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.367+0000 7ff847fff640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8300a2e00 0x7ff8300a52c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff83800bbe0 tx=0x7ff83800b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.371+0000 7ff8457fa640 1 -- 172.21.15.137:0/3689992669 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff83c09b340 con 0x7ff848102570 2024-09-12T12:13:28.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.582+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff84810a0d0 con 0x7ff848102570 2024-09-12T12:13:28.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.583+0000 7ff8457fa640 1 -- 172.21.15.137:0/3689992669 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7ff83c09f640 con 0x7ff848102570 2024-09-12T12:13:28.583 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:28.584 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"btime":"2024-09-12T12:13:12:456388+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:13:12.406179+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:13:28.584 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8300a2e00 msgr2=0x7ff8300a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8300a2e00 0x7ff8300a52c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff83800bbe0 tx=0x7ff83800b390 comp rx=0 tx=0).stop 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102570 msgr2=0x7ff848198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102570 0x7ff848198a70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00d770 tx=0x7ff83c00dc30 comp rx=0 tx=0).stop 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 shutdown_connections 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8300a2e00 0x7ff8300a52c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff848102ef0 0x7ff84819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff848102570 0x7ff848198a70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 --2- 172.21.15.137:0/3689992669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff848100e20 0x7ff848198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 >> 172.21.15.137:0/3689992669 conn(0x7ff8480fc7d0 msgr2=0x7ff84806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 shutdown_connections 2024-09-12T12:13:28.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:28.585+0000 7ff84ea56640 1 -- 172.21.15.137:0/3689992669 wait complete. 2024-09-12T12:13:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.739+0000 7faa57d98640 1 -- 172.21.15.137:0/2517060301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa500745c0 msgr2=0x7faa500749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.739+0000 7faa57d98640 1 --2- 172.21.15.137:0/2517060301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa500745c0 0x7faa500749a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009900 tx=0x7faa4c02a480 comp rx=0 tx=0).stop 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.739+0000 7faa57d98640 1 -- 172.21.15.137:0/2517060301 shutdown_connections 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.739+0000 7faa57d98640 1 --2- 172.21.15.137:0/2517060301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa5010e380 0x7faa50110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.739+0000 7faa57d98640 1 --2- 172.21.15.137:0/2517060301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa50074ee0 0x7faa5010de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.739+0000 7faa57d98640 1 --2- 172.21.15.137:0/2517060301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa500745c0 0x7faa500749a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.739+0000 7faa57d98640 1 -- 172.21.15.137:0/2517060301 >> 172.21.15.137:0/2517060301 conn(0x7faa500fd7e0 msgr2=0x7faa500ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 -- 172.21.15.137:0/2517060301 shutdown_connections 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 -- 172.21.15.137:0/2517060301 wait complete. 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 Processor -- start 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 -- start start 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa500745c0 0x7faa5019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa50074ee0 0x7faa5019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa5010e380 0x7faa5019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa50114110 con 0x7faa50074ee0 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faa50113f90 con 0x7faa500745c0 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa57d98640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa50114290 con 0x7faa5010e380 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.740+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa50074ee0 0x7faa5019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa50074ee0 0x7faa5019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51666/0 (socket says 172.21.15.137:51666) 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 -- 172.21.15.137:0/403298251 learned_addr learned my addr 172.21.15.137:0/403298251 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55b34640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa500745c0 0x7faa5019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa56335640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa5010e380 0x7faa5019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 -- 172.21.15.137:0/403298251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa5010e380 msgr2=0x7faa5019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa5010e380 0x7faa5019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 -- 172.21.15.137:0/403298251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa500745c0 msgr2=0x7faa5019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa500745c0 0x7faa5019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 -- 172.21.15.137:0/403298251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5019f550 con 0x7faa50074ee0 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa55333640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa50074ee0 0x7faa5019aaa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa4000c970 tx=0x7faa4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa46ffd640 1 -- 172.21.15.137:0/403298251 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40007bb0 con 0x7faa50074ee0 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa46ffd640 1 -- 172.21.15.137:0/403298251 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa40007d50 con 0x7faa50074ee0 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa46ffd640 1 -- 172.21.15.137:0/403298251 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4000f450 con 0x7faa50074ee0 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5019f840 con 0x7faa50074ee0 2024-09-12T12:13:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.741+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501a70d0 con 0x7faa50074ee0 2024-09-12T12:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.743+0000 7faa46ffd640 1 -- 172.21.15.137:0/403298251 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7faa40005020 con 0x7faa50074ee0 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 Processor -- start 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 -- start start 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406c3af0 0x55ae406c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406cecb0 0x55ae406d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae406282f0 con 0x55ae406cecb0 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ae405d0270 con 0x55ae406c4410 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.744+0000 7faa5ac10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae406242b0 con 0x55ae406c3af0 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406cecb0 0x55ae406d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406cecb0 0x55ae406d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51676/0 (socket says 172.21.15.137:51676) 2024-09-12T12:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 -- 172.21.15.137:0/3013712897 learned_addr learned my addr 172.21.15.137:0/3013712897 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa46ffd640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa3c0a2e40 0x7faa3c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa56335640 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406c3af0 0x55ae406c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55b34640 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa56335640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa3c0a2e40 0x7faa3c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa46ffd640 1 -- 172.21.15.137:0/403298251 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7faa400cf3f0 con 0x7faa50074ee0 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 -- 172.21.15.137:0/3013712897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406c3af0 msgr2=0x55ae406c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406c3af0 0x55ae406c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 -- 172.21.15.137:0/3013712897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 msgr2=0x55ae406ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 -- 172.21.15.137:0/3013712897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae405cf340 con 0x55ae406cecb0 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa55333640 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406cecb0 0x55ae406d10a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faa4009b490 tx=0x7faa40004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31935a90bc2b8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa56335640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa3c0a2e40 0x7faa3c0a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa4800bfd0 tx=0x7faa48004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa3bfff640 1 -- 172.21.15.137:0/3013712897 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4001d050 con 0x55ae406cecb0 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa3bfff640 1 -- 172.21.15.137:0/3013712897 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa40097f80 con 0x55ae406cecb0 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.745+0000 7faa3bfff640 1 -- 172.21.15.137:0/3013712897 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40099a90 con 0x55ae406cecb0 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 -- 172.21.15.137:0/3013712897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406cecb0 msgr2=0x55ae406d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406cecb0 0x55ae406d10a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faa4009b490 tx=0x7faa40004a20 comp rx=0 tx=0).stop 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 -- 172.21.15.137:0/3013712897 shutdown_connections 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406cecb0 0x55ae406d10a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3013712897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406c3af0 0x55ae406c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 -- 172.21.15.137:0/3013712897 >> 172.21.15.137:0/3013712897 conn(0x55ae406c0060 msgr2=0x55ae406c0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 -- 172.21.15.137:0/3013712897 shutdown_connections 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 -- 172.21.15.137:0/3013712897 wait complete. 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.746+0000 7faa5ac10740 1 -- start start 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 1 Processor -- start 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c3af0 0x55ae406cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406cc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae406d1880 con 0x55ae406c3af0 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ae405b1290 con 0x55ae406c4410 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae406d1a00 con 0x55ae406cecb0 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa5ac10740 10 client.0 _ll_register_callbacks cb 0x55ae405e2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c3af0 0x55ae406cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c3af0 0x55ae406cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51686/0 (socket says 172.21.15.137:51686) 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 -- 172.21.15.137:0/969216992 learned_addr learned my addr 172.21.15.137:0/969216992 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa56335640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55333640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 -- 172.21.15.137:0/969216992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 msgr2=0x55ae406cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 -- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 msgr2=0x55ae406cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa55b34640 1 -- 172.21.15.137:0/969216992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4033b210 con 0x55ae406c3af0 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.747+0000 7faa56335640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa55b34640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c3af0 0x55ae406cbb90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faa4c033000 tx=0x7faa4c031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5cef0d213422df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa397fa640 1 -- 172.21.15.137:0/969216992 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c035040 con 0x55ae406c3af0 2024-09-12T12:13:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae40635a20 con 0x55ae406c3af0 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa397fa640 1 -- 172.21.15.137:0/969216992 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7faa4c004400 con 0x55ae406c3af0 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa397fa640 1 -- 172.21.15.137:0/969216992 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa4c005020 con 0x55ae406c3af0 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 10 client.15318 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ae40407620 con 0x55ae406c3af0 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa397fa640 1 -- 172.21.15.137:0/969216992 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ae40635a20 con 0x55ae406c3af0 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa397fa640 1 -- 172.21.15.137:0/969216992 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7faa4c0035f0 con 0x55ae406c3af0 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 10 client.15318 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 10 client.15318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 20 client.15318 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae406deba0 0x55ae406e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 4 client.15318 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.748+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae4062fd70 con 0x55ae406deba0 2024-09-12T12:13:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.749+0000 7faa55333640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae406deba0 0x55ae406e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.749+0000 7faa55333640 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae406deba0 0x55ae406e0f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc607dcfd4820dd server_cookie=cd605db1add04cf6 in_seq=0 out_seq=0 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.749+0000 7faa397fa640 10 client.15318 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa397fa640 1 -- 172.21.15.137:0/969216992 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae4062fd70 con 0x55ae406deba0 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa397fa640 10 client.15318 handle_command_reply: tid=0 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 client.15318 shutdown 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c3af0 msgr2=0x55ae406cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c3af0 0x55ae406cbb90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faa4c033000 tx=0x7faa4c031a60 comp rx=0 tx=0).stop 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 shutdown_connections 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c4410 0x55ae406cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae406deba0 0x55ae406e0f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa397fa640 0 client.15318 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:28.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 --2- 172.21.15.137:0/969216992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c3af0 0x55ae406cbb90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 >> 172.21.15.137:0/969216992 conn(0x55ae406c0060 msgr2=0x55ae406bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 shutdown_connections 2024-09-12T12:13:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 1 -- 172.21.15.137:0/969216992 wait complete. 2024-09-12T12:13:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.750+0000 7faa5ac10740 20 client.15318 trim_cache size 0 max 16384 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 Processor -- start 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 -- start start 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406e6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa501b2170 con 0x55ae406c4410 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faa501afa40 con 0x55ae406c3af0 2024-09-12T12:13:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa5ac10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa501ad890 con 0x55ae406cecb0 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51694/0 (socket says 172.21.15.137:51694) 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.764+0000 7faa55333640 1 -- 172.21.15.137:0/3613078550 learned_addr learned my addr 172.21.15.137:0/3613078550 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55b34640 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55333640 1 -- 172.21.15.137:0/3613078550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 msgr2=0x55ae406e6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55333640 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55333640 1 -- 172.21.15.137:0/3613078550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 msgr2=0x55ae406c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55333640 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55333640 1 -- 172.21.15.137:0/3613078550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae406c4240 con 0x55ae406c4410 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55b34640 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa55333640 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406cc440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa40097bf0 tx=0x7faa40086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296e3938e1256346 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa3b7fe640 1 -- 172.21.15.137:0/3613078550 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4001d070 con 0x55ae406c4410 2024-09-12T12:13:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa3b7fe640 1 -- 172.21.15.137:0/3613078550 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4009fb40 con 0x55ae406c4410 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.765+0000 7faa3b7fe640 1 -- 172.21.15.137:0/3613078550 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa400ccf80 con 0x55ae406c4410 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- 172.21.15.137:0/3613078550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 msgr2=0x55ae406cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406cc440 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7faa40097bf0 tx=0x7faa40086040 comp rx=0 tx=0).stop 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- 172.21.15.137:0/3613078550 shutdown_connections 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406e6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406cc440 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 --2- 172.21.15.137:0/3613078550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- 172.21.15.137:0/3613078550 >> 172.21.15.137:0/3613078550 conn(0x55ae406c0060 msgr2=0x55ae406cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- 172.21.15.137:0/3613078550 shutdown_connections 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- 172.21.15.137:0/3613078550 wait complete. 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- start start 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 Processor -- start 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406d71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa501adf00 con 0x55ae406c4410 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faa501af370 con 0x55ae406c3af0 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa501ae5d0 con 0x55ae406cecb0 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.766+0000 7faa5ac10740 10 client.0 _ll_register_callbacks cb 0x7faa4c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51700/0 (socket says 172.21.15.137:51700) 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 -- 172.21.15.137:0/1764049600 learned_addr learned my addr 172.21.15.137:0/1764049600 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55b34640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406d71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa56335640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 -- 172.21.15.137:0/1764049600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 msgr2=0x55ae406d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 -- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 msgr2=0x55ae406d71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406d71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 -- 172.21.15.137:0/1764049600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae406d3090 con 0x55ae406c4410 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa56335640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa55333640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406d2510 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa400058c0 tx=0x7faa4009bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4462026b6b92c0ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa397fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa397fa640 1 -- 172.21.15.137:0/1764049600 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa400b3020 con 0x55ae406c4410 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa397fa640 1 -- 172.21.15.137:0/1764049600 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7faa400860b0 con 0x55ae406c4410 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa4009fb40 con 0x55ae406c4410 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa397fa640 1 -- 172.21.15.137:0/1764049600 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7faa4001d040 con 0x55ae406c4410 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa5ac10740 10 client.15324 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ae407495e0 con 0x55ae406c4410 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.767+0000 7faa397fa640 1 -- 172.21.15.137:0/1764049600 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4009fb40 con 0x55ae406c4410 2024-09-12T12:13:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa397fa640 1 -- 172.21.15.137:0/1764049600 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7faa4009abe0 con 0x55ae406c4410 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa5ac10740 10 client.15324 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa5ac10740 10 client.15324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa5ac10740 20 client.15324 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa5ac10740 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae407498d0 0x55ae4074bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa5ac10740 4 client.15324 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae405ac900 con 0x55ae407498d0 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa55b34640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae407498d0 0x55ae4074bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa397fa640 10 client.15324 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.768+0000 7faa55b34640 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae407498d0 0x55ae4074bcc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bafb2648913aea9 server_cookie=1e4a891765b30804 in_seq=0 out_seq=0 2024-09-12T12:13:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa397fa640 1 -- 172.21.15.137:0/1764049600 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae405ac900 con 0x55ae407498d0 2024-09-12T12:13:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa397fa640 10 client.15324 handle_command_reply: tid=0 2024-09-12T12:13:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 client.15324 shutdown 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 msgr2=0x55ae406d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406d2510 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa400058c0 tx=0x7faa4009bf80 comp rx=0 tx=0).stop 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 shutdown_connections 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae406cecb0 0x55ae406d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae407498d0 0x55ae4074bcc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae406c4410 0x55ae406d2510 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 --2- 172.21.15.137:0/1764049600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae406c3af0 0x55ae406d71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa397fa640 0 client.15324 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 >> 172.21.15.137:0/1764049600 conn(0x55ae406c0060 msgr2=0x55ae406bf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 shutdown_connections 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 1 -- 172.21.15.137:0/1764049600 wait complete. 2024-09-12T12:13:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa5ac10740 20 client.15324 trim_cache size 0 max 16384 2024-09-12T12:13:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:13:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:13:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:13:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa3c0a2e40 msgr2=0x7faa3c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa3c0a2e40 0x7faa3c0a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faa4800bfd0 tx=0x7faa48004a40 comp rx=0 tx=0).stop 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa50074ee0 msgr2=0x7faa5019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa50074ee0 0x7faa5019aaa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faa4000c970 tx=0x7faa4000ce30 comp rx=0 tx=0).stop 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 shutdown_connections 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa3c0a2e40 0x7faa3c0a5300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa5010e380 0x7faa5019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa50074ee0 0x7faa5019aaa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 --2- 172.21.15.137:0/403298251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa500745c0 0x7faa5019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 >> 172.21.15.137:0/403298251 conn(0x7faa500fd7e0 msgr2=0x7faa500fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 shutdown_connections 2024-09-12T12:13:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:28.781+0000 7faa57d98640 1 -- 172.21.15.137:0/403298251 wait complete. 2024-09-12T12:13:28.939 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:28 smithi137 ceph-mon[30950]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-09-12T12:13:28.940 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:28 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3689992669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:28.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:28 smithi171 ceph-mon[35098]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-09-12T12:13:28.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:28 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3689992669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:29.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:28 smithi189 ceph-mon[35475]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 76 KiB/s wr, 47 op/s 2024-09-12T12:13:29.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:28 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3689992669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:29.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T12:11:13.196665+0000', 'modified': '2024-09-12T12:13:12.406179+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24476, 'mds_1': 24475, 'mds_2': 24454}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6833/4107944214', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 4107944214}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 4107944214}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/1673181827', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 1673181827}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 1673181827}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/2972240277', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 2972240277}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 2972240277}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.189:6837/2107760146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 2107760146}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 2107760146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24475': {'gid': 24475, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.189:6839/4067701297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 4067701297}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 4067701297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24476': {'gid': 24476, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.171:6839/1705495422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 1705495422}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 1705495422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24476, 'qdb_cluster': [24454, 24475, 24476]} 2024-09-12T12:13:29.333 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-12T12:13:29.333 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T12:13:29.625 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:30.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 -- 172.21.15.137:0/795917686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20106a80 msgr2=0x7f1b20106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 --2- 172.21.15.137:0/795917686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20106a80 0x7f1b20106e60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009930 tx=0x7f1b1402a420 comp rx=0 tx=0).stop 2024-09-12T12:13:30.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 -- 172.21.15.137:0/795917686 shutdown_connections 2024-09-12T12:13:30.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 --2- 172.21.15.137:0/795917686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20101420 0x7f1b2010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 --2- 172.21.15.137:0/795917686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b20100a80 0x7f1b20100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 --2- 172.21.15.137:0/795917686 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20106a80 0x7f1b20106e60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 -- 172.21.15.137:0/795917686 >> 172.21.15.137:0/795917686 conn(0x7f1b200fc7a0 msgr2=0x7f1b200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 -- 172.21.15.137:0/795917686 shutdown_connections 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.112+0000 7f1b26b53640 1 -- 172.21.15.137:0/795917686 wait complete. 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 Processor -- start 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 -- start start 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20100a80 0x7f1b20198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20101420 0x7f1b201989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b20106a80 0x7f1b2019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20110e80 con 0x7f1b20100a80 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20110d00 con 0x7f1b20101420 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b26b53640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20111000 con 0x7f1b20106a80 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b248ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20100a80 0x7f1b20198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b248ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20100a80 0x7f1b20198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44758/0 (socket says 172.21.15.137:44758) 2024-09-12T12:13:30.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b248ef640 1 -- 172.21.15.137:0/2109049384 learned_addr learned my addr 172.21.15.137:0/2109049384 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20101420 0x7f1b201989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b250f0640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b20106a80 0x7f1b2019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.113+0000 7f1b248ef640 1 -- 172.21.15.137:0/2109049384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b20106a80 msgr2=0x7f1b2019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b248ef640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b20106a80 0x7f1b2019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b248ef640 1 -- 172.21.15.137:0/2109049384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20101420 msgr2=0x7f1b201989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b248ef640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20101420 0x7f1b201989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b248ef640 1 -- 172.21.15.137:0/2109049384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2019d420 con 0x7f1b20100a80 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20101420 0x7f1b201989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b248ef640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20100a80 0x7f1b20198470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009230 tx=0x7f1b14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b1dffb640 1 -- 172.21.15.137:0/2109049384 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1403f030 con 0x7f1b20100a80 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b1dffb640 1 -- 172.21.15.137:0/2109049384 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b14003ad0 con 0x7f1b20100a80 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2019d6b0 con 0x7f1b20100a80 2024-09-12T12:13:30.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b1dffb640 1 -- 172.21.15.137:0/2109049384 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14035070 con 0x7f1b20100a80 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.114+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b201a4ff0 con 0x7f1b20100a80 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.116+0000 7f1b1dffb640 1 -- 172.21.15.137:0/2109049384 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f1b14042030 con 0x7f1b20100a80 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 Processor -- start 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 -- start start 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b667db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b672670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b674fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56328b5ce090 con 0x56328b6682f0 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56328b575a70 con 0x56328b6679d0 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56328b5ca050 con 0x56328b672bb0 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b1dffb640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f1b100a2d80 0x7f1b100a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b672670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b672670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44774/0 (socket says 172.21.15.137:44774) 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 -- 172.21.15.137:0/367397523 learned_addr learned my addr 172.21.15.137:0/367397523 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b250f0640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f1b100a2d80 0x7f1b100a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b674fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b250f0640 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b667db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b1dffb640 1 -- 172.21.15.137:0/2109049384 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f1b140efb40 con 0x7f1b20100a80 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 -- 172.21.15.137:0/367397523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 msgr2=0x56328b674fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b674fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 -- 172.21.15.137:0/367397523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 msgr2=0x56328b667db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b667db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b248ef640 1 -- 172.21.15.137:0/367397523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328b582130 con 0x56328b6682f0 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b674fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.117+0000 7f1b250f0640 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b667db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.118+0000 7f1b248ef640 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b672670 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1b140c4680 tx=0x7f1b14005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae43fa972907d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:30.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.118+0000 7f1b250f0640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f1b100a2d80 0x7f1b100a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b180137c0 tx=0x7f1b1800b380 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.118+0000 7f1b0affd640 1 -- 172.21.15.137:0/367397523 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b140c4bb0 con 0x56328b6682f0 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.118+0000 7f1b0affd640 1 -- 172.21.15.137:0/367397523 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b140bbf40 con 0x56328b6682f0 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.118+0000 7f1b0affd640 1 -- 172.21.15.137:0/367397523 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b140b85b0 con 0x56328b6682f0 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.118+0000 7f1b28f2c740 1 -- 172.21.15.137:0/367397523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 msgr2=0x56328b672670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.118+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b672670 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f1b140c4680 tx=0x7f1b14005e50 comp rx=0 tx=0).stop 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- 172.21.15.137:0/367397523 shutdown_connections 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b674fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b672670 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/367397523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b667db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- 172.21.15.137:0/367397523 >> 172.21.15.137:0/367397523 conn(0x56328b663f60 msgr2=0x56328b664340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- 172.21.15.137:0/367397523 shutdown_connections 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- 172.21.15.137:0/367397523 wait complete. 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- start start 2024-09-12T12:13:30.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 Processor -- start 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b672380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b66f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56328b675540 con 0x56328b6682f0 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56328b588f20 con 0x56328b6679d0 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56328b6756c0 con 0x56328b672bb0 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b28f2c740 10 client.0 _ll_register_callbacks cb 0x56328b552030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.119+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b66f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b66f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44776/0 (socket says 172.21.15.137:44776) 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 -- 172.21.15.137:0/1016309887 learned_addr learned my addr 172.21.15.137:0/1016309887 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b250f0640 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b248ef640 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b672380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 -- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 msgr2=0x56328b66fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 -- 172.21.15.137:0/1016309887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 msgr2=0x56328b672380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b672380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 -- 172.21.15.137:0/1016309887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328b33f2b0 con 0x56328b6682f0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b66f6d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00bf40 tx=0x7f1b0c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57868b7b2645e494 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1afffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1afffff640 1 -- 172.21.15.137:0/1016309887 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c012710 con 0x56328b6682f0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1afffff640 1 -- 172.21.15.137:0/1016309887 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f1b0c0128b0 con 0x56328b6682f0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b140bbf40 con 0x56328b6682f0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1afffff640 1 -- 172.21.15.137:0/1016309887 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1b0c012ab0 con 0x56328b6682f0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b28f2c740 10 client.15336 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.120+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56328b680d90 con 0x56328b6682f0 2024-09-12T12:13:30.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1afffff640 1 -- 172.21.15.137:0/1016309887 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b140bbf40 con 0x56328b6682f0 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1afffff640 1 -- 172.21.15.137:0/1016309887 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1b0c01a3d0 con 0x56328b6682f0 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1b28f2c740 10 client.15336 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1b28f2c740 10 client.15336 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1b28f2c740 20 client.15336 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b683450 0x56328b6848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1b28f2c740 4 client.15336 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 --> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56328b5db600 con 0x56328b683450 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.121+0000 7f1b248ef640 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b683450 0x56328b6848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1afffff640 10 client.15336 ms_handle_connect on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1b248ef640 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b683450 0x56328b6848c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63dfb345d7681147 server_cookie=119be2d7dc2846f in_seq=0 out_seq=0 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1afffff640 1 -- 172.21.15.137:0/1016309887 <== mds.2 v2:172.21.15.189:6836/2107760146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56328b5db600 con 0x56328b683450 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1afffff640 10 client.15336 handle_command_reply: tid=0 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1b28f2c740 1 client.15336 shutdown 2024-09-12T12:13:30.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 msgr2=0x56328b66f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b66f6d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00bf40 tx=0x7f1b0c011870 comp rx=0 tx=0).stop 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 shutdown_connections 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.122+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b683450 0x56328b6848c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b66f6d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1016309887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b672380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 >> 172.21.15.137:0/1016309887 conn(0x56328b663f60 msgr2=0x56328b669c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1afffff640 0 client.15336 ms_handle_reset on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 shutdown_connections 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1016309887 wait complete. 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.123+0000 7f1b28f2c740 20 client.15336 trim_cache size 0 max 16384 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 Processor -- start 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 -- start start 2024-09-12T12:13:30.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b683450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b67e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56328b682c00 con 0x56328b6682f0 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56328b6832d0 con 0x56328b6679d0 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56328b6839a0 con 0x56328b672bb0 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b67e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b67e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44792/0 (socket says 172.21.15.137:44792) 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 -- 172.21.15.137:0/81944336 learned_addr learned my addr 172.21.15.137:0/81944336 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b250f0640 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 -- 172.21.15.137:0/81944336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 msgr2=0x56328b66a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 -- 172.21.15.137:0/81944336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 msgr2=0x56328b683450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b683450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 -- 172.21.15.137:0/81944336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56328b663d90 con 0x56328b6682f0 2024-09-12T12:13:30.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.141+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b67e590 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0058c0 tx=0x7f1b0c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4600f775bae9f6de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:30.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b0a7fc640 1 -- 172.21.15.137:0/81944336 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c000e60 con 0x56328b6682f0 2024-09-12T12:13:30.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b0a7fc640 1 -- 172.21.15.137:0/81944336 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0c01acb0 con 0x56328b6682f0 2024-09-12T12:13:30.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b0a7fc640 1 -- 172.21.15.137:0/81944336 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c028040 con 0x56328b6682f0 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 -- 172.21.15.137:0/81944336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 msgr2=0x56328b67e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b67e590 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0058c0 tx=0x7f1b0c00bab0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 -- 172.21.15.137:0/81944336 shutdown_connections 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b672bb0 0x56328b66a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6682f0 0x56328b67e590 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/81944336 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b6679d0 0x56328b683450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 -- 172.21.15.137:0/81944336 >> 172.21.15.137:0/81944336 conn(0x56328b663f60 msgr2=0x56328b66dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 -- 172.21.15.137:0/81944336 shutdown_connections 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.142+0000 7f1b28f2c740 1 -- 172.21.15.137:0/81944336 wait complete. 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 -- start start 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 Processor -- start 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6679d0 0x56328b66bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b6682f0 0x56328b66c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b672bb0 0x56328b68bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56328b684740 con 0x56328b6679d0 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56328b684070 con 0x56328b672bb0 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56328b684e10 con 0x56328b6682f0 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b28f2c740 10 client.0 _ll_register_callbacks cb 0x7f1b0c00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b248ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6679d0 0x56328b66bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b248ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6679d0 0x56328b66bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44806/0 (socket says 172.21.15.137:44806) 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b248ef640 1 -- 172.21.15.137:0/1823825312 learned_addr learned my addr 172.21.15.137:0/1823825312 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b6682f0 0x56328b66c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.143+0000 7f1b250f0640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b672bb0 0x56328b68bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b248ef640 1 -- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b6682f0 msgr2=0x56328b66c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b248ef640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b6682f0 0x56328b66c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b248ef640 1 -- 172.21.15.137:0/1823825312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b672bb0 msgr2=0x56328b68bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b248ef640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b672bb0 0x56328b68bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b248ef640 1 -- 172.21.15.137:0/1823825312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56328b68c050 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b250f0640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b672bb0 0x56328b68bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b248ef640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6679d0 0x56328b66bd20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1b140c0230 tx=0x7f1b140092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efda8a7e2a16c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1afffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1afffff640 1 -- 172.21.15.137:0/1823825312 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14049040 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1afffff640 1 -- 172.21.15.137:0/1823825312 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f1b14003c80 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b0c01acb0 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1afffff640 1 -- 172.21.15.137:0/1823825312 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1b140c4bb0 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b28f2c740 10 client.15342 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56328b68d1b0 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1afffff640 1 -- 172.21.15.137:0/1823825312 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b0c01acb0 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1afffff640 1 -- 172.21.15.137:0/1823825312 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f1b140ed4d0 con 0x56328b6679d0 2024-09-12T12:13:30.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.144+0000 7f1b28f2c740 10 client.15342 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1b28f2c740 10 client.15342 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1b28f2c740 20 client.15342 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b6f6040 0x56328b6f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1b28f2c740 4 client.15342 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 --> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] -- command(tid 0: {"prefix": "status"}) -- 0x56328b5d5b10 con 0x56328b6f6040 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b6f6040 0x56328b6f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1afffff640 10 client.15342 ms_handle_connect on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.145+0000 7f1b1ffff640 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b6f6040 0x56328b6f4d50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89909e31c4c36775 server_cookie=15698fd2351ca1f7 in_seq=0 out_seq=0 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1afffff640 1 -- 172.21.15.137:0/1823825312 <== mds.2 v2:172.21.15.189:6836/2107760146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56328b5d5b10 con 0x56328b6f6040 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1afffff640 10 client.15342 handle_command_reply: tid=0 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 client.15342 shutdown 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6679d0 msgr2=0x56328b66bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6679d0 0x56328b66bd20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1b140c0230 tx=0x7f1b140092f0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 shutdown_connections 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:6836/2107760146,v1:172.21.15.189:6837/2107760146] conn(0x56328b6f6040 0x56328b6f4d50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56328b672bb0 0x56328b68bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56328b6682f0 0x56328b66c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1afffff640 0 client.15342 ms_handle_reset on v2:172.21.15.189:6836/2107760146 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 --2- 172.21.15.137:0/1823825312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56328b6679d0 0x56328b66bd20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 >> 172.21.15.137:0/1823825312 conn(0x56328b663f60 msgr2=0x56328b66a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 shutdown_connections 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 1 -- 172.21.15.137:0/1823825312 wait complete. 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.146+0000 7f1b28f2c740 20 client.15342 trim_cache size 0 max 16384 2024-09-12T12:13:30.161 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 2, 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24454, 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 128.59878730700001, 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 24, 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 85, 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 162.287933385 2024-09-12T12:13:30.162 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:13:30.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f1b100a2d80 msgr2=0x7f1b100a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f1b100a2d80 0x7f1b100a5240 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b180137c0 tx=0x7f1b1800b380 comp rx=0 tx=0).stop 2024-09-12T12:13:30.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20100a80 msgr2=0x7f1b20198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:30.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20100a80 0x7f1b20198470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009230 tx=0x7f1b14004270 comp rx=0 tx=0).stop 2024-09-12T12:13:30.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 shutdown_connections 2024-09-12T12:13:30.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f1b100a2d80 0x7f1b100a5240 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1b20106a80 0x7f1b2019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20101420 0x7f1b201989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 --2- 172.21.15.137:0/2109049384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1b20100a80 0x7f1b20198470 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:30.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 >> 172.21.15.137:0/2109049384 conn(0x7f1b200fc7a0 msgr2=0x7f1b2006a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:30.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 shutdown_connections 2024-09-12T12:13:30.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:30.147+0000 7f1b26b53640 1 -- 172.21.15.137:0/2109049384 wait complete. 2024-09-12T12:13:30.938 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:30 smithi137 ceph-mon[30950]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-09-12T12:13:30.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:30 smithi171 ceph-mon[35098]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-09-12T12:13:30.970 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T12:13:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:30 smithi189 ceph-mon[35475]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-09-12T12:13:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 -- 172.21.15.137:0/1399584696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8101420 msgr2=0x7fe6a810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1399584696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8101420 0x7fe6a810d950 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009cd0 tx=0x7fe6a002a8e0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 -- 172.21.15.137:0/1399584696 shutdown_connections 2024-09-12T12:13:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1399584696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8101420 0x7fe6a810d950 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1399584696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a8100a80 0x7fe6a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1399584696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a8106a80 0x7fe6a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 -- 172.21.15.137:0/1399584696 >> 172.21.15.137:0/1399584696 conn(0x7fe6a80fc7a0 msgr2=0x7fe6a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 -- 172.21.15.137:0/1399584696 shutdown_connections 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.671+0000 7fe6b0358640 1 -- 172.21.15.137:0/1399584696 wait complete. 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 Processor -- start 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 -- start start 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8100a80 0x7fe6a8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a8101420 0x7fe6a81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a8106a80 0x7fe6a819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8110ec0 con 0x7fe6a8100a80 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8110d40 con 0x7fe6a8106a80 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6b0358640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a8111040 con 0x7fe6a8101420 2024-09-12T12:13:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6ae0f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8100a80 0x7fe6a8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6ae0f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8100a80 0x7fe6a8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44818/0 (socket says 172.21.15.137:44818) 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1870630871 learned_addr learned my addr 172.21.15.137:0/1870630871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a8101420 0x7fe6a81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.672+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a8106a80 0x7fe6a819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1870630871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a8101420 msgr2=0x7fe6a81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a8101420 0x7fe6a81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1870630871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a8106a80 msgr2=0x7fe6a819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a8106a80 0x7fe6a819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1870630871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a819d410 con 0x7fe6a8100a80 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a8106a80 0x7fe6a819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8100a80 0x7fe6a8198460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400d770 tx=0x7fe6a400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe69f7fe640 1 -- 172.21.15.137:0/1870630871 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4017070 con 0x7fe6a8100a80 2024-09-12T12:13:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe69f7fe640 1 -- 172.21.15.137:0/1870630871 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a4009820 con 0x7fe6a8100a80 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a819d700 con 0x7fe6a8100a80 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe69f7fe640 1 -- 172.21.15.137:0/1870630871 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004cd0 con 0x7fe6a8100a80 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.673+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a81a4fe0 con 0x7fe6a8100a80 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.675+0000 7fe69f7fe640 1 -- 172.21.15.137:0/1870630871 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fe6a40099d0 con 0x7fe6a8100a80 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.675+0000 7fe6b2731740 1 Processor -- start 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6b2731740 1 -- start start 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6b2731740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d36f223090 con 0x55d36f2c0de0 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6b2731740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d36f1caa70 con 0x55d36f2c1700 2024-09-12T12:13:31.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6b2731740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d36f21f050 con 0x55d36f2cbfc0 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe69f7fe640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6940a2d80 0x7fe6940a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2c11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44820/0 (socket says 172.21.15.137:44820) 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 -- 172.21.15.137:0/822319755 learned_addr learned my addr 172.21.15.137:0/822319755 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6940a2d80 0x7fe6940a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe69f7fe640 1 -- 172.21.15.137:0/1870630871 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fe6a40cefe0 con 0x7fe6a8100a80 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 -- 172.21.15.137:0/822319755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 msgr2=0x55d36f2ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 -- 172.21.15.137:0/822319755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 msgr2=0x55d36f2cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.676+0000 7fe6ae8f5640 1 -- 172.21.15.137:0/822319755 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36f1d7130 con 0x55d36f2c0de0 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.677+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2c11c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0031b50 tx=0x7fe6a0031b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff363a8f85a8758c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.677+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6940a2d80 0x7fe6940a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0031bf0 tx=0x7fe6a0049000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.677+0000 7fe69cdea640 1 -- 172.21.15.137:0/822319755 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0004240 con 0x55d36f2c0de0 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.677+0000 7fe69cdea640 1 -- 172.21.15.137:0/822319755 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a0036040 con 0x55d36f2c0de0 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.677+0000 7fe69cdea640 1 -- 172.21.15.137:0/822319755 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0030cb0 con 0x55d36f2c0de0 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.677+0000 7fe6b2731740 1 -- 172.21.15.137:0/822319755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 msgr2=0x55d36f2c11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.677+0000 7fe6b2731740 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2c11c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0031b50 tx=0x7fe6a0031b80 comp rx=0 tx=0).stop 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 -- 172.21.15.137:0/822319755 shutdown_connections 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 --2- 172.21.15.137:0/822319755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2c11c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 -- 172.21.15.137:0/822319755 >> 172.21.15.137:0/822319755 conn(0x55d36f2bd370 msgr2=0x55d36f2bd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 -- 172.21.15.137:0/822319755 shutdown_connections 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 -- 172.21.15.137:0/822319755 wait complete. 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 -- start start 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 Processor -- start 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.678+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6b2731740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2ceb30 con 0x55d36f2c0de0 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6b2731740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d36f1ddf20 con 0x55d36f2c1700 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6b2731740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2cecb0 con 0x55d36f2cbfc0 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6b2731740 10 client.0 _ll_register_callbacks cb 0x55d36f1a7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:31.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44834/0 (socket says 172.21.15.137:44834) 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1387465947 learned_addr learned my addr 172.21.15.137:0/1387465947 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 msgr2=0x55d36f2c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1387465947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 msgr2=0x55d36f2c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1387465947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36ef942b0 con 0x55d36f2c0de0 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2cb850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4097db0 tx=0x7fe6a409bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed12639af04ce11e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:31.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe691ffb640 1 -- 172.21.15.137:0/1387465947 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40b4060 con 0x55d36f2c0de0 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a0036040 con 0x55d36f2c0de0 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe691ffb640 1 -- 172.21.15.137:0/1387465947 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fe6a40b4200 con 0x55d36f2c0de0 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe691ffb640 1 -- 172.21.15.137:0/1387465947 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6a4020040 con 0x55d36f2c0de0 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6b2731740 10 client.15354 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d36f2da4f0 con 0x55d36f2c0de0 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.679+0000 7fe691ffb640 1 -- 172.21.15.137:0/1387465947 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6a0036040 con 0x55d36f2c0de0 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe691ffb640 1 -- 172.21.15.137:0/1387465947 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe6a4066250 con 0x55d36f2c0de0 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6b2731740 10 client.15354 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6b2731740 10 client.15354 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6b2731740 20 client.15354 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f2dcb50 0x55d36f2ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6b2731740 4 client.15354 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 --> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d36f230600 con 0x55d36f2dcb50 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f2dcb50 0x55d36f2ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f2dcb50 0x55d36f2ddfc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a29cf0aed08991ee server_cookie=f9295f39b1b4067b in_seq=0 out_seq=0 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.680+0000 7fe691ffb640 10 client.15354 ms_handle_connect on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe691ffb640 1 -- 172.21.15.137:0/1387465947 <== mds.1 v2:172.21.15.189:6838/4067701297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d36f230600 con 0x55d36f2dcb50 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe691ffb640 10 client.15354 handle_command_reply: tid=0 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 client.15354 shutdown 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 msgr2=0x55d36f2cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2cb850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4097db0 tx=0x7fe6a409bae0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 shutdown_connections 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f2dcb50 0x55d36f2ddfc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2c1700 0x55d36f2c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe691ffb640 0 client.15354 ms_handle_reset on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:31.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1387465947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2cb850 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 >> 172.21.15.137:0/1387465947 conn(0x55d36f2bd370 msgr2=0x55d36f2c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:31.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 shutdown_connections 2024-09-12T12:13:31.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 1 -- 172.21.15.137:0/1387465947 wait complete. 2024-09-12T12:13:31.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.681+0000 7fe6b2731740 20 client.15354 trim_cache size 0 max 16384 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 Processor -- start 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 -- start start 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c0de0 0x55d36f2ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c1700 0x55d36f2c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2dc300 con 0x55d36f2c1700 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2dc9d0 con 0x55d36f2cbfc0 2024-09-12T12:13:31.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6b2731740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2dd0a0 con 0x55d36f2c0de0 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6ad8f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c1700 0x55d36f2c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6ad8f3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c1700 0x55d36f2c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44840/0 (socket says 172.21.15.137:44840) 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6ad8f3640 1 -- 172.21.15.137:0/2923839089 learned_addr learned my addr 172.21.15.137:0/2923839089 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c0de0 0x55d36f2ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.700+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ad8f3640 1 -- 172.21.15.137:0/2923839089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c0de0 msgr2=0x55d36f2ddfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c0de0 0x55d36f2ddfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ad8f3640 1 -- 172.21.15.137:0/2923839089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 msgr2=0x55d36f2c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ad8f3640 1 -- 172.21.15.137:0/2923839089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36f2cbbc0 con 0x55d36f2c1700 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ae8f5640 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c0de0 0x55d36f2ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:31.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c1700 0x55d36f2c8460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe698000c00 tx=0x7fe698008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5e166462a9af64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe693fff640 1 -- 172.21.15.137:0/2923839089 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698020070 con 0x55d36f2c1700 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe693fff640 1 -- 172.21.15.137:0/2923839089 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe698015d10 con 0x55d36f2c1700 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.701+0000 7fe693fff640 1 -- 172.21.15.137:0/2923839089 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698014680 con 0x55d36f2c1700 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 -- 172.21.15.137:0/2923839089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c1700 msgr2=0x55d36f2c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c1700 0x55d36f2c8460 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe698000c00 tx=0x7fe698008040 comp rx=0 tx=0).stop 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 -- 172.21.15.137:0/2923839089 shutdown_connections 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f2c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c1700 0x55d36f2c8460 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 --2- 172.21.15.137:0/2923839089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c0de0 0x55d36f2ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 -- 172.21.15.137:0/2923839089 >> 172.21.15.137:0/2923839089 conn(0x55d36f2bd370 msgr2=0x55d36f2bd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 -- 172.21.15.137:0/2923839089 shutdown_connections 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 -- 172.21.15.137:0/2923839089 wait complete. 2024-09-12T12:13:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 -- start start 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.702+0000 7fe6b2731740 1 Processor -- start 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2e1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c1700 0x55d36f2e1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f345250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2dde40 con 0x55d36f2c0de0 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2dd770 con 0x55d36f2cbfc0 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d36f2de510 con 0x55d36f2c1700 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 10 client.0 _ll_register_callbacks cb 0x7fe6a4095f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2e1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44844/0 (socket says 172.21.15.137:44844) 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1338873302 learned_addr learned my addr 172.21.15.137:0/1338873302 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c1700 0x55d36f2e1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c1700 msgr2=0x55d36f2e1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c1700 0x55d36f2e1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1338873302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 msgr2=0x55d36f345250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f345250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 -- 172.21.15.137:0/1338873302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36f345810 con 0x55d36f2c0de0 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c1700 0x55d36f2e1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6ae0f4640 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2e1800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400dfd0 tx=0x7fe6a40a21b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ff596d129348bb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe691ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe691ffb640 1 -- 172.21.15.137:0/1338873302 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40b8020 con 0x55d36f2c0de0 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe691ffb640 1 -- 172.21.15.137:0/1338873302 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fe6a40cc970 con 0x55d36f2c0de0 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe698015d10 con 0x55d36f2c0de0 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe691ffb640 1 -- 172.21.15.137:0/1338873302 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe6a409af20 con 0x55d36f2c0de0 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 10 client.15360 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.703+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d36f346a80 con 0x55d36f2c0de0 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe691ffb640 1 -- 172.21.15.137:0/1338873302 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe698015d10 con 0x55d36f2c0de0 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe691ffb640 1 -- 172.21.15.137:0/1338873302 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe6a40b4180 con 0x55d36f2c0de0 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe6b2731740 10 client.15360 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe6b2731740 10 client.15360 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe6b2731740 20 client.15360 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f34b5b0 0x55d36f34a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe6b2731740 4 client.15360 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 --> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] -- command(tid 0: {"prefix": "status"}) -- 0x55d36f22ab10 con 0x55d36f34b5b0 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.704+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f34b5b0 0x55d36f34a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6ad8f3640 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f34b5b0 0x55d36f34a2c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e29a7e848f84c5d server_cookie=e2b977d9f1f4ac11 in_seq=0 out_seq=0 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe691ffb640 10 client.15360 ms_handle_connect on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe691ffb640 1 -- 172.21.15.137:0/1338873302 <== mds.1 v2:172.21.15.189:6838/4067701297 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d36f22ab10 con 0x55d36f34b5b0 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe691ffb640 10 client.15360 handle_command_reply: tid=0 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 client.15360 shutdown 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 msgr2=0x55d36f2e1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2e1800 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400dfd0 tx=0x7fe6a40a21b0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 shutdown_connections 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:6838/4067701297,v1:172.21.15.189:6839/4067701297] conn(0x55d36f34b5b0 0x55d36f34a2c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d36f2cbfc0 0x55d36f345250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d36f2c1700 0x55d36f2e1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe691ffb640 0 client.15360 ms_handle_reset on v2:172.21.15.189:6838/4067701297 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 --2- 172.21.15.137:0/1338873302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d36f2c0de0 0x55d36f2e1800 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 >> 172.21.15.137:0/1338873302 conn(0x55d36f2bd370 msgr2=0x55d36f2c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 shutdown_connections 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 1 -- 172.21.15.137:0/1338873302 wait complete. 2024-09-12T12:13:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.705+0000 7fe6b2731740 20 client.15360 trim_cache size 0 max 16384 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 1, 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24475, 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 131.169728883, 2024-09-12T12:13:31.715 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 24, 2024-09-12T12:13:31.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 85, 2024-09-12T12:13:31.716 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T12:13:31.716 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 158.564036273 2024-09-12T12:13:31.716 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:13:31.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6940a2d80 msgr2=0x7fe6940a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6940a2d80 0x7fe6940a5240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0031bf0 tx=0x7fe6a0049000 comp rx=0 tx=0).stop 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8100a80 msgr2=0x7fe6a8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8100a80 0x7fe6a8198460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400d770 tx=0x7fe6a400db70 comp rx=0 tx=0).stop 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 shutdown_connections 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6940a2d80 0x7fe6940a5240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6a8106a80 0x7fe6a819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a8101420 0x7fe6a81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 --2- 172.21.15.137:0/1870630871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6a8100a80 0x7fe6a8198460 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 >> 172.21.15.137:0/1870630871 conn(0x7fe6a80fc7a0 msgr2=0x7fe6a806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 shutdown_connections 2024-09-12T12:13:31.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:31.706+0000 7fe6b0358640 1 -- 172.21.15.137:0/1870630871 wait complete. 2024-09-12T12:13:32.494 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T12:13:32.743 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:32.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:32 smithi137 ceph-mon[30950]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 39 op/s 2024-09-12T12:13:32.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:32 smithi171 ceph-mon[35098]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 39 op/s 2024-09-12T12:13:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:32 smithi189 ceph-mon[35475]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 40 KiB/s wr, 39 op/s 2024-09-12T12:13:33.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 -- 172.21.15.137:0/337925639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 msgr2=0x7f4b08106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 --2- 172.21.15.137:0/337925639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b08106e60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009930 tx=0x7f4afc02a420 comp rx=0 tx=0).stop 2024-09-12T12:13:33.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 -- 172.21.15.137:0/337925639 shutdown_connections 2024-09-12T12:13:33.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 --2- 172.21.15.137:0/337925639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08101420 0x7f4b0810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 --2- 172.21.15.137:0/337925639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b08100a80 0x7f4b08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 --2- 172.21.15.137:0/337925639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b08106e60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 -- 172.21.15.137:0/337925639 >> 172.21.15.137:0/337925639 conn(0x7f4b080fc7a0 msgr2=0x7f4b080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 -- 172.21.15.137:0/337925639 shutdown_connections 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 -- 172.21.15.137:0/337925639 wait complete. 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 Processor -- start 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.264+0000 7f4b1062d640 1 -- start start 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b1062d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b08100a80 0x7f4b081983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b1062d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08101420 0x7f4b08198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b1062d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b0819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b1062d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08110e80 con 0x7f4b08106a80 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b1062d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08110d00 con 0x7f4b08100a80 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b1062d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b08111000 con 0x7f4b08101420 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b0819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b0819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44864/0 (socket says 172.21.15.137:44864) 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 -- 172.21.15.137:0/3915607537 learned_addr learned my addr 172.21.15.137:0/3915607537 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b08100a80 0x7f4b081983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08101420 0x7f4b08198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 -- 172.21.15.137:0/3915607537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08101420 msgr2=0x7f4b08198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08101420 0x7f4b08198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 -- 172.21.15.137:0/3915607537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b08100a80 msgr2=0x7f4b081983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b08100a80 0x7f4b081983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 -- 172.21.15.137:0/3915607537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0819d390 con 0x7f4b08106a80 2024-09-12T12:13:33.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b0ebca640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b0819ccb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000eaa0 tx=0x7f4b0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4afb7fe640 1 -- 172.21.15.137:0/3915607537 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000cdb0 con 0x7f4b08106a80 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.265+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0819d680 con 0x7f4b08106a80 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.266+0000 7f4afb7fe640 1 -- 172.21.15.137:0/3915607537 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b00005020 con 0x7f4b08106a80 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.266+0000 7f4afb7fe640 1 -- 172.21.15.137:0/3915607537 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b000116d0 con 0x7f4b08106a80 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.266+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b081a4f60 con 0x7f4b08106a80 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4afb7fe640 1 -- 172.21.15.137:0/3915607537 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f4b000054d0 con 0x7f4b08106a80 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 Processor -- start 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 -- start start 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f386ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386b1b0 0x55a5f3879a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f3879fc0 0x55a5f387c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a5f37d1090 con 0x55a5f3879fc0 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a5f3778a70 con 0x55a5f386b1b0 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4b12a06740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5f37cd050 con 0x55a5f386a890 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.268+0000 7f4afb7fe640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4af40a2d80 0x7f4af40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f3879fc0 0x55a5f387c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f3879fc0 0x55a5f387c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44870/0 (socket says 172.21.15.137:44870) 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 -- 172.21.15.137:0/3733486260 learned_addr learned my addr 172.21.15.137:0/3733486260 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386b1b0 0x55a5f3879a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0ebca640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4af40a2d80 0x7f4af40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f386ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4afb7fe640 1 -- 172.21.15.137:0/3915607537 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f4b000cf830 con 0x7f4b08106a80 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 -- 172.21.15.137:0/3733486260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 msgr2=0x55a5f386ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f386ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 -- 172.21.15.137:0/3733486260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386b1b0 msgr2=0x55a5f3879a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386b1b0 0x55a5f3879a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 -- 172.21.15.137:0/3733486260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5f3785130 con 0x55a5f3879fc0 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386b1b0 0x55a5f3879a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f3879fc0 0x55a5f387c3b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b04012780 tx=0x7f4b040096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fe5e2ecfbaa66b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4b0ebca640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4af40a2d80 0x7f4af40a5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b000ccf70 tx=0x7f4b00097f50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4af8ff9640 1 -- 172.21.15.137:0/3733486260 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04012d00 con 0x55a5f3879fc0 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4af8ff9640 1 -- 172.21.15.137:0/3733486260 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b040107d0 con 0x55a5f3879fc0 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.269+0000 7f4af8ff9640 1 -- 172.21.15.137:0/3733486260 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04003850 con 0x55a5f3879fc0 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 -- 172.21.15.137:0/3733486260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f3879fc0 msgr2=0x55a5f387c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f3879fc0 0x55a5f387c3b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4b04012780 tx=0x7f4b040096e0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 -- 172.21.15.137:0/3733486260 shutdown_connections 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f3879fc0 0x55a5f387c3b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386b1b0 0x55a5f3879a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3733486260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f386ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 -- 172.21.15.137:0/3733486260 >> 172.21.15.137:0/3733486260 conn(0x55a5f3866f60 msgr2=0x55a5f3867340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 -- 172.21.15.137:0/3733486260 shutdown_connections 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.270+0000 7f4b12a06740 1 -- 172.21.15.137:0/3733486260 wait complete. 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 -- start start 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 Processor -- start 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386a890 0x55a5f38798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3876c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f3879fc0 0x55a5f3877160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a5f387cb90 con 0x55a5f386b1b0 2024-09-12T12:13:33.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a5f378bf20 con 0x55a5f386a890 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5f387cd10 con 0x55a5f3879fc0 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b12a06740 10 client.0 _ll_register_callbacks cb 0x55a5f3755030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3876c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3876c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44878/0 (socket says 172.21.15.137:44878) 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3907464793 learned_addr learned my addr 172.21.15.137:0/3907464793 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386a890 0x55a5f38798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3907464793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f3879fc0 msgr2=0x55a5f3877160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f3879fc0 0x55a5f3877160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386a890 msgr2=0x55a5f38798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386a890 0x55a5f38798d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3907464793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5f35422b0 con 0x55a5f386b1b0 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386a890 0x55a5f38798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:33.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3876c20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009930 tx=0x7f4afc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7be30cad02c9c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.271+0000 7f4af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4af1ffb640 1 -- 172.21.15.137:0/3907464793 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc035040 con 0x55a5f386b1b0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b040107d0 con 0x55a5f386b1b0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4af1ffb640 1 -- 172.21.15.137:0/3907464793 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f4afc004040 con 0x55a5f386b1b0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4af1ffb640 1 -- 172.21.15.137:0/3907464793 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4afc004240 con 0x55a5f386b1b0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 10 client.15372 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a5f3888570 con 0x55a5f386b1b0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4af1ffb640 1 -- 172.21.15.137:0/3907464793 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b040107d0 con 0x55a5f386b1b0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4af1ffb640 1 -- 172.21.15.137:0/3907464793 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4afc004a60 con 0x55a5f386b1b0 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 10 client.15372 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 10 client.15372 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 20 client.15372 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f388aba0 0x55a5f388c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 4 client.15372 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5f37de600 con 0x55a5f388aba0 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.272+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f388aba0 0x55a5f388c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.273+0000 7f4af1ffb640 10 client.15372 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.273+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f388aba0 0x55a5f388c010 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba95f4d2abca01e server_cookie=1855b3a0e3b1aa27 in_seq=0 out_seq=0 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.273+0000 7f4af1ffb640 1 -- 172.21.15.137:0/3907464793 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5f37de600 con 0x55a5f388aba0 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.273+0000 7f4af1ffb640 10 client.15372 handle_command_reply: tid=0 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.273+0000 7f4b12a06740 1 client.15372 shutdown 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 msgr2=0x55a5f3876c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3876c20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009930 tx=0x7f4afc0317e0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 shutdown_connections 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f3879fc0 0x55a5f3877160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f388aba0 0x55a5f388c010 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3876c20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4af1ffb640 0 client.15372 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:33.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3907464793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f386a890 0x55a5f38798d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 >> 172.21.15.137:0/3907464793 conn(0x55a5f3866f60 msgr2=0x55a5f386ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:33.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 shutdown_connections 2024-09-12T12:13:33.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 1 -- 172.21.15.137:0/3907464793 wait complete. 2024-09-12T12:13:33.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.274+0000 7f4b12a06740 20 client.15372 trim_cache size 0 max 16384 2024-09-12T12:13:33.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 Processor -- start 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 -- start start 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f388c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3872130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f3873e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a5f388a350 con 0x55a5f386b1b0 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a5f388aa20 con 0x55a5f3879fc0 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b12a06740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5f388b0f0 con 0x55a5f386a890 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3872130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3872130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44884/0 (socket says 172.21.15.137:44884) 2024-09-12T12:13:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3417240392 learned_addr learned my addr 172.21.15.137:0/3417240392 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f388c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f3873e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3417240392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 msgr2=0x55a5f388c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f388c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3417240392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 msgr2=0x55a5f3873e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f3873e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/3417240392 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5f3866a00 con 0x55a5f386b1b0 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f388c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.292+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3872130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4afc000c00 tx=0x7f4afc003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adea624f3611976b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4af3fff640 1 -- 172.21.15.137:0/3417240392 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc035070 con 0x55a5f386b1b0 2024-09-12T12:13:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4af3fff640 1 -- 172.21.15.137:0/3417240392 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4afc004ba0 con 0x55a5f386b1b0 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4af3fff640 1 -- 172.21.15.137:0/3417240392 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc004040 con 0x55a5f386b1b0 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 -- 172.21.15.137:0/3417240392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 msgr2=0x55a5f3872130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3872130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4afc000c00 tx=0x7f4afc003900 comp rx=0 tx=0).stop 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 -- 172.21.15.137:0/3417240392 shutdown_connections 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f3873e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f3872130 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 --2- 172.21.15.137:0/3417240392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f388c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 -- 172.21.15.137:0/3417240392 >> 172.21.15.137:0/3417240392 conn(0x55a5f3866f60 msgr2=0x55a5f3871a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.293+0000 7f4b12a06740 1 -- 172.21.15.137:0/3417240392 shutdown_connections 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 -- 172.21.15.137:0/3417240392 wait complete. 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 -- start start 2024-09-12T12:13:33.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 Processor -- start 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f3894850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f388db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f388e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a5f388be90 con 0x55a5f386b1b0 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a5f388b7c0 con 0x55a5f3879fc0 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a5f388c560 con 0x55a5f386a890 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b12a06740 10 client.0 _ll_register_callbacks cb 0x7f4afc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b0e3c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f388db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b0e3c9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f388db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44890/0 (socket says 172.21.15.137:44890) 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/1316721603 learned_addr learned my addr 172.21.15.137:0/1316721603 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f388e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.294+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f3894850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/1316721603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 msgr2=0x55a5f3894850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f3894850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 msgr2=0x55a5f388e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f388e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b0e3c9640 1 -- 172.21.15.137:0/1316721603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5f388e760 con 0x55a5f386b1b0 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f388e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b0e3c9640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f388db80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0059d0 tx=0x7f4afc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ede149650bcc162 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4af1ffb640 1 -- 172.21.15.137:0/1316721603 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc035040 con 0x55a5f386b1b0 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4af1ffb640 1 -- 172.21.15.137:0/1316721603 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f4afc003a70 con 0x55a5f386b1b0 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4afc004ba0 con 0x55a5f386b1b0 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4af1ffb640 1 -- 172.21.15.137:0/1316721603 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4afc069050 con 0x55a5f386b1b0 2024-09-12T12:13:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b12a06740 10 client.15378 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a5f38f52b0 con 0x55a5f386b1b0 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4af1ffb640 1 -- 172.21.15.137:0/1316721603 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4afc004ba0 con 0x55a5f386b1b0 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.295+0000 7f4af1ffb640 1 -- 172.21.15.137:0/1316721603 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4afc04f480 con 0x55a5f386b1b0 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b12a06740 10 client.15378 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b12a06740 10 client.15378 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b12a06740 20 client.15378 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b12a06740 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f38f9bd0 0x55a5f38f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b12a06740 4 client.15378 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "status"}) -- 0x55a5f37d8b10 con 0x55a5f38f9bd0 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f38f9bd0 0x55a5f38f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b0dbc8640 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f38f9bd0 0x55a5f38f88e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd1dc30d65a91b5 server_cookie=402b77ffcaf9b6c6 in_seq=0 out_seq=0 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4af1ffb640 10 client.15378 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4af1ffb640 1 -- 172.21.15.137:0/1316721603 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a5f37d8b10 con 0x55a5f38f9bd0 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4af1ffb640 10 client.15378 handle_command_reply: tid=0 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.296+0000 7f4b12a06740 1 client.15378 shutdown 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 msgr2=0x55a5f388db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f388db80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0059d0 tx=0x7f4afc005d50 comp rx=0 tx=0).stop 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 shutdown_connections 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a5f3879fc0 0x55a5f388e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a5f38f9bd0 0x55a5f38f88e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a5f386b1b0 0x55a5f388db80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 --2- 172.21.15.137:0/1316721603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a5f386a890 0x55a5f3894850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 >> 172.21.15.137:0/1316721603 conn(0x55a5f3866f60 msgr2=0x55a5f3872240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4af1ffb640 0 client.15378 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:33.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 shutdown_connections 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 1 -- 172.21.15.137:0/1316721603 wait complete. 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b12a06740 20 client.15378 trim_cache size 0 max 16384 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "dd66268a-70fe-11ef-bcea-c7b262605968", 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 0, 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 24476, 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-09-12T12:13:33.309 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-09-12T12:13:33.310 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 139.80739307499999, 2024-09-12T12:13:33.310 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 24, 2024-09-12T12:13:33.310 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 85, 2024-09-12T12:13:33.310 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 84, 2024-09-12T12:13:33.310 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 151.341320083 2024-09-12T12:13:33.310 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4af40a2d80 msgr2=0x7f4af40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4af40a2d80 0x7f4af40a5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4b000ccf70 tx=0x7f4b00097f50 comp rx=0 tx=0).stop 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 msgr2=0x7f4b0819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b0819ccb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000eaa0 tx=0x7f4b0000ef60 comp rx=0 tx=0).stop 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 shutdown_connections 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4af40a2d80 0x7f4af40a5240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4b08106a80 0x7f4b0819ccb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b08101420 0x7f4b08198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 --2- 172.21.15.137:0/3915607537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b08100a80 0x7f4b081983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 >> 172.21.15.137:0/3915607537 conn(0x7f4b080fc7a0 msgr2=0x7f4b0806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:33.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.297+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 shutdown_connections 2024-09-12T12:13:33.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:33.298+0000 7f4b1062d640 1 -- 172.21.15.137:0/3915607537 wait complete. 2024-09-12T12:13:34.068 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T12:13:34.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:34 smithi171 ceph-mon[35098]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T12:13:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:34 smithi137 ceph-mon[30950]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T12:13:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:34 smithi189 ceph-mon[35475]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T12:13:36.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:36 smithi137 ceph-mon[30950]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T12:13:36.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:36 smithi171 ceph-mon[35098]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T12:13:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:36 smithi189 ceph-mon[35475]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T12:13:38.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:38 smithi171 ceph-mon[35098]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T12:13:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:38 smithi137 ceph-mon[30950]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T12:13:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:38 smithi189 ceph-mon[35475]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-12T12:13:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:40 smithi137 ceph-mon[30950]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T12:13:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:40 smithi189 ceph-mon[35475]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T12:13:41.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:40 smithi171 ceph-mon[35098]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T12:13:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 -- 172.21.15.137:0/4072296850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e4101370 msgr2=0x7f94e410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 --2- 172.21.15.137:0/4072296850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e4101370 0x7f94e410d8a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009930 tx=0x7f94dc02a420 comp rx=0 tx=0).stop 2024-09-12T12:13:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 -- 172.21.15.137:0/4072296850 shutdown_connections 2024-09-12T12:13:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 --2- 172.21.15.137:0/4072296850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e4101370 0x7f94e410d8a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 --2- 172.21.15.137:0/4072296850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94e41009d0 0x7f94e4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 --2- 172.21.15.137:0/4072296850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94e41069e0 0x7f94e4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 -- 172.21.15.137:0/4072296850 >> 172.21.15.137:0/4072296850 conn(0x7f94e40fc6f0 msgr2=0x7f94e40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 -- 172.21.15.137:0/4072296850 shutdown_connections 2024-09-12T12:13:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.878+0000 7f94e88f1640 1 -- 172.21.15.137:0/4072296850 wait complete. 2024-09-12T12:13:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 Processor -- start 2024-09-12T12:13:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 -- start start 2024-09-12T12:13:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94e41009d0 0x7f94e4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94e4101370 0x7f94e41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e41069e0 0x7f94e419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4110e20 con 0x7f94e41069e0 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4110ca0 con 0x7f94e4101370 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e88f1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4110fa0 con 0x7f94e41009d0 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e41069e0 0x7f94e419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e41069e0 0x7f94e419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55186/0 (socket says 172.21.15.137:55186) 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e2d9d640 1 -- 172.21.15.137:0/2948981074 learned_addr learned my addr 172.21.15.137:0/2948981074 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94e4101370 0x7f94e41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.879+0000 7f94e259c640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94e41009d0 0x7f94e4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e2d9d640 1 -- 172.21.15.137:0/2948981074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94e41009d0 msgr2=0x7f94e4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e2d9d640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94e41009d0 0x7f94e4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e2d9d640 1 -- 172.21.15.137:0/2948981074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94e4101370 msgr2=0x7f94e41977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e2d9d640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94e4101370 0x7f94e41977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e2d9d640 1 -- 172.21.15.137:0/2948981074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e419c280 con 0x7f94e41069e0 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e2d9d640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e41069e0 0x7f94e419bb50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005ab0 tx=0x7f94dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94d37fe640 1 -- 172.21.15.137:0/2948981074 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc03f030 con 0x7f94e41069e0 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94d37fe640 1 -- 172.21.15.137:0/2948981074 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc003ce0 con 0x7f94e41069e0 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e419c510 con 0x7f94e41069e0 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94d37fe640 1 -- 172.21.15.137:0/2948981074 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc035070 con 0x7f94e41069e0 2024-09-12T12:13:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.880+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e41a3e00 con 0x7f94e41069e0 2024-09-12T12:13:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.882+0000 7f94d37fe640 1 -- 172.21.15.137:0/2948981074 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f94dc042030 con 0x7f94e41069e0 2024-09-12T12:13:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 Processor -- start 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 -- start start 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997518d00 0x55f9975190e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997519620 0x55f997523980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997523ec0 0x55f9975262b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94d37fe640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94cc0a2ec0 0x7f94cc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f99747b2f0 con 0x55f997523ec0 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f997423270 con 0x55f997519620 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.883+0000 7f94eb692740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f9974772b0 con 0x55f997518d00 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e2d9d640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94cc0a2ec0 0x7f94cc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997518d00 0x55f9975190e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997519620 0x55f997523980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997518d00 0x55f9975190e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:52478/0 (socket says 172.21.15.137:52478) 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 -- 172.21.15.137:0/454264921 learned_addr learned my addr 172.21.15.137:0/454264921 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997519620 0x55f997523980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:57920/0 (socket says 172.21.15.137:57920) 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997523ec0 0x55f9975262b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94d37fe640 1 -- 172.21.15.137:0/2948981074 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f94dc0ef910 con 0x7f94e41069e0 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 -- 172.21.15.137:0/454264921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997519620 msgr2=0x55f997523980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997519620 0x55f997523980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 -- 172.21.15.137:0/454264921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997523ec0 msgr2=0x55f9975262b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997523ec0 0x55f9975262b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 -- 172.21.15.137:0/454264921 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f997422340 con 0x55f997518d00 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e2d9d640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94cc0a2ec0 0x7f94cc0a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0ed050 tx=0x7f94dc08dd80 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997518d00 0x55f9975190e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94d80127c0 tx=0x7f94d800b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98a52fee27494e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94d0ff9640 1 -- 172.21.15.137:0/454264921 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8012e00 con 0x55f997518d00 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94d0ff9640 1 -- 172.21.15.137:0/454264921 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d8010c50 con 0x55f997518d00 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.884+0000 7f94d0ff9640 1 -- 172.21.15.137:0/454264921 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d801ebc0 con 0x55f997518d00 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 -- 172.21.15.137:0/454264921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997518d00 msgr2=0x55f9975190e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997518d00 0x55f9975190e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94d80127c0 tx=0x7f94d800b7c0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 -- 172.21.15.137:0/454264921 shutdown_connections 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997523ec0 0x55f9975262b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997519620 0x55f997523980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 --2- 172.21.15.137:0/454264921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997518d00 0x55f9975190e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 -- 172.21.15.137:0/454264921 >> 172.21.15.137:0/454264921 conn(0x55f997515270 msgr2=0x55f997515650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 -- 172.21.15.137:0/454264921 shutdown_connections 2024-09-12T12:13:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.885+0000 7f94eb692740 1 -- 172.21.15.137:0/454264921 wait complete. 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 -- start start 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 Processor -- start 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997520dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f997521300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f997521840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f997526a90 con 0x55f997518d00 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f997404290 con 0x55f997523ec0 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f997526c10 con 0x55f997519620 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94eb692740 10 client.0 _ll_register_callbacks cb 0x55f997435140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997520dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997520dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55196/0 (socket says 172.21.15.137:55196) 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 -- 172.21.15.137:0/189951826 learned_addr learned my addr 172.21.15.137:0/189951826 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e259c640 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f997521300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 -- 172.21.15.137:0/189951826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 msgr2=0x55f997521300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f997521300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 -- 172.21.15.137:0/189951826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 msgr2=0x55f997521840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f997521840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.886+0000 7f94e1d9b640 1 -- 172.21.15.137:0/189951826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f99718e210 con 0x55f997518d00 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997520dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f94d800b940 tx=0x7f94d800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f9d82e09dabdb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94c9ffb640 1 -- 172.21.15.137:0/189951826 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8010770 con 0x55f997518d00 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94c9ffb640 1 -- 172.21.15.137:0/189951826 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f94d8010910 con 0x55f997518d00 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d8010c50 con 0x55f997518d00 2024-09-12T12:13:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94c9ffb640 1 -- 172.21.15.137:0/189951826 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f94d8004040 con 0x55f997518d00 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 10 client.15390 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f99725a620 con 0x55f997518d00 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94c9ffb640 1 -- 172.21.15.137:0/189951826 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d8010c50 con 0x55f997518d00 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94c9ffb640 1 -- 172.21.15.137:0/189951826 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f94d8003680 con 0x55f997518d00 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 10 client.15390 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 20 client.15390 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f997533db0 0x55f9975361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 4 client.15390 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.887+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f997488a20 con 0x55f997533db0 2024-09-12T12:13:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.888+0000 7f94e259c640 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f997533db0 0x55f9975361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.888+0000 7f94c9ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.888+0000 7f94e259c640 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f997533db0 0x55f9975361a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79cd03611dffa3de server_cookie=a95587a23723d5b6 in_seq=0 out_seq=0 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94c9ffb640 1 -- 172.21.15.137:0/189951826 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f997488a20 con 0x55f997533db0 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94c9ffb640 10 client.15390 handle_command_reply: tid=0 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 client.15390 shutdown 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 msgr2=0x55f997520dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997520dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f94d800b940 tx=0x7f94d800be00 comp rx=0 tx=0).stop 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 shutdown_connections 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f997521840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f997521300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f997533db0 0x55f9975361a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 --2- 172.21.15.137:0/189951826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997520dc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94c9ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 >> 172.21.15.137:0/189951826 conn(0x55f997515270 msgr2=0x55f99751af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 shutdown_connections 2024-09-12T12:13:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 1 -- 172.21.15.137:0/189951826 wait complete. 2024-09-12T12:13:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.889+0000 7f94eb692740 20 client.15390 trim_cache size 0 max 16384 2024-09-12T12:13:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.903+0000 7f94eb692740 1 Processor -- start 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.903+0000 7f94eb692740 1 -- start start 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94eb692740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f99752c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94eb692740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f99752c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94eb692740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f99753b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94eb692740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41adcc0 con 0x55f997518d00 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94eb692740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41ab590 con 0x55f997523ec0 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94eb692740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41a9400 con 0x55f997519620 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f99752c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f99752c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55208/0 (socket says 172.21.15.137:55208) 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 -- 172.21.15.137:0/2969569671 learned_addr learned my addr 172.21.15.137:0/2969569671 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 -- 172.21.15.137:0/2969569671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 msgr2=0x55f99752c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/2969569671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f99752c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 -- 172.21.15.137:0/2969569671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 msgr2=0x55f99753b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/2969569671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f99753b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 -- 172.21.15.137:0/2969569671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f99751bb50 con 0x55f997518d00 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/2969569671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f99752c2b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f94d8005bc0 tx=0x7f94d8012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9296f80820922c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94cbfff640 1 -- 172.21.15.137:0/2969569671 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d800ba90 con 0x55f997518d00 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94cbfff640 1 -- 172.21.15.137:0/2969569671 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d8016070 con 0x55f997518d00 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.904+0000 7f94cbfff640 1 -- 172.21.15.137:0/2969569671 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d80052a0 con 0x55f997518d00 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 -- 172.21.15.137:0/2969569671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 msgr2=0x55f99752c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 --2- 172.21.15.137:0/2969569671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f99752c2b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f94d8005bc0 tx=0x7f94d8012760 comp rx=0 tx=0).stop 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 -- 172.21.15.137:0/2969569671 shutdown_connections 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 --2- 172.21.15.137:0/2969569671 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f99753b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 --2- 172.21.15.137:0/2969569671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f99752c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 --2- 172.21.15.137:0/2969569671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f99752c2b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 -- 172.21.15.137:0/2969569671 >> 172.21.15.137:0/2969569671 conn(0x55f997515270 msgr2=0x55f997522e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 -- 172.21.15.137:0/2969569671 shutdown_connections 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 -- 172.21.15.137:0/2969569671 wait complete. 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 -- start start 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 Processor -- start 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.905+0000 7f94eb692740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997512420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f997512960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f9975998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41a9a50 con 0x55f997518d00 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41aaec0 con 0x55f997523ec0 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41aa120 con 0x55f997519620 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 10 client.0 _ll_register_callbacks cb 0x7f94d800be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997512420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997512420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55222/0 (socket says 172.21.15.137:55222) 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 -- 172.21.15.137:0/1556991843 learned_addr learned my addr 172.21.15.137:0/1556991843 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 -- 172.21.15.137:0/1556991843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 msgr2=0x55f997512960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f997512960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 -- 172.21.15.137:0/1556991843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 msgr2=0x55f9975998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f9975998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 -- 172.21.15.137:0/1556991843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f997599f10 con 0x55f997518d00 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94e1d9b640 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997512420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f94d8003b70 tx=0x7f94d800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78973da0407682e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94c9ffb640 1 -- 172.21.15.137:0/1556991843 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d803e060 con 0x55f997518d00 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94c9ffb640 1 -- 172.21.15.137:0/1556991843 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f94d8010740 con 0x55f997518d00 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d8016070 con 0x55f997518d00 2024-09-12T12:13:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94c9ffb640 1 -- 172.21.15.137:0/1556991843 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f94d8010940 con 0x55f997518d00 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 10 client.15396 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.906+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f9975a12b0 con 0x55f997518d00 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94c9ffb640 1 -- 172.21.15.137:0/1556991843 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d8016070 con 0x55f997518d00 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94c9ffb640 1 -- 172.21.15.137:0/1556991843 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f94d801e8b0 con 0x55f997518d00 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94eb692740 10 client.15396 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94eb692740 10 client.15396 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94eb692740 20 client.15396 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94eb692740 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f9975a1910 0x55f9975a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94eb692740 4 client.15396 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f997482d70 con 0x55f9975a1910 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94e259c640 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f9975a1910 0x55f9975a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94c9ffb640 10 client.15396 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.907+0000 7f94e259c640 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f9975a1910 0x55f9975a3d00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59399e4246a6f47 server_cookie=b840a65fd7d601f0 in_seq=0 out_seq=0 2024-09-12T12:13:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94c9ffb640 1 -- 172.21.15.137:0/1556991843 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f997482d70 con 0x55f9975a1910 2024-09-12T12:13:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94c9ffb640 10 client.15396 handle_command_reply: tid=0 2024-09-12T12:13:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 client.15396 shutdown 2024-09-12T12:13:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 msgr2=0x55f997512420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997512420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f94d8003b70 tx=0x7f94d800ba90 comp rx=0 tx=0).stop 2024-09-12T12:13:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 shutdown_connections 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f997523ec0 0x55f9975998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f997519620 0x55f997512960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f9975a1910 0x55f9975a3d00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 --2- 172.21.15.137:0/1556991843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f997518d00 0x55f997512420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94c9ffb640 0 client.15396 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 >> 172.21.15.137:0/1556991843 conn(0x55f997515270 msgr2=0x55f99751a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 shutdown_connections 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.913+0000 7f94eb692740 1 -- 172.21.15.137:0/1556991843 wait complete. 2024-09-12T12:13:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94eb692740 20 client.15396 trim_cache size 0 max 16384 2024-09-12T12:13:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:13:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:13:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:13:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94cc0a2ec0 msgr2=0x7f94cc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94cc0a2ec0 0x7f94cc0a5380 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0ed050 tx=0x7f94dc08dd80 comp rx=0 tx=0).stop 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e41069e0 msgr2=0x7f94e419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e41069e0 0x7f94e419bb50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005ab0 tx=0x7f94dc0042f0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 shutdown_connections 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94cc0a2ec0 0x7f94cc0a5380 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94e41069e0 0x7f94e419bb50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94e4101370 0x7f94e41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 --2- 172.21.15.137:0/2948981074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94e41009d0 0x7f94e4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 >> 172.21.15.137:0/2948981074 conn(0x7f94e40fc6f0 msgr2=0x7f94e40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 shutdown_connections 2024-09-12T12:13:41.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:41.914+0000 7f94e88f1640 1 -- 172.21.15.137:0/2948981074 wait complete. 2024-09-12T12:13:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:42 smithi137 ceph-mon[30950]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T12:13:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:42 smithi189 ceph-mon[35475]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T12:13:43.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:42 smithi171 ceph-mon[35098]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-12T12:13:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:44 smithi137 ceph-mon[30950]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T12:13:45.480 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:44 smithi171 ceph-mon[35098]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T12:13:45.481 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:44 smithi189 ceph-mon[35475]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-12T12:13:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:46 smithi137 ceph-mon[30950]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:13:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:46 smithi189 ceph-mon[35475]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:13:47.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:46 smithi171 ceph-mon[35098]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:13:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:48 smithi137 ceph-mon[30950]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T12:13:49.069 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T12:13:49.070 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:13:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:48 smithi189 ceph-mon[35475]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T12:13:49.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:48 smithi171 ceph-mon[35098]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-12T12:13:49.351 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 -- 172.21.15.137:0/679853728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 msgr2=0x7fe4481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/679853728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe4481029b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe438009900 tx=0x7fe43802a480 comp rx=0 tx=0).stop 2024-09-12T12:13:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 -- 172.21.15.137:0/679853728 shutdown_connections 2024-09-12T12:13:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/679853728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe448102ef0 0x7fe44810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/679853728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe4481029b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/679853728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe448100e20 0x7fe448101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 -- 172.21.15.137:0/679853728 >> 172.21.15.137:0/679853728 conn(0x7fe4480fc7d0 msgr2=0x7fe4480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 -- 172.21.15.137:0/679853728 shutdown_connections 2024-09-12T12:13:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.861+0000 7fe44f2e1640 1 -- 172.21.15.137:0/679853728 wait complete. 2024-09-12T12:13:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 Processor -- start 2024-09-12T12:13:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 -- start start 2024-09-12T12:13:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe448100e20 0x7fe4481985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe448198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe448102ef0 0x7fe44819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe448110fa0 con 0x7fe448102570 2024-09-12T12:13:49.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe448110e20 con 0x7fe448100e20 2024-09-12T12:13:49.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44f2e1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe448111120 con 0x7fe448102ef0 2024-09-12T12:13:49.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44c87c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe448198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:49.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44c87c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe448198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36934/0 (socket says 172.21.15.137:36934) 2024-09-12T12:13:49.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44c87c640 1 -- 172.21.15.137:0/1411546487 learned_addr learned my addr 172.21.15.137:0/1411546487 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:49.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44d87e640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe448102ef0 0x7fe44819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:49.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.862+0000 7fe44d07d640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe448100e20 0x7fe4481985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:49.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44c87c640 1 -- 172.21.15.137:0/1411546487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe448102ef0 msgr2=0x7fe44819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:49.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44c87c640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe448102ef0 0x7fe44819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:49.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44c87c640 1 -- 172.21.15.137:0/1411546487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe448100e20 msgr2=0x7fe4481985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:49.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44c87c640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe448100e20 0x7fe4481985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44c87c640 1 -- 172.21.15.137:0/1411546487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44819d570 con 0x7fe448102570 2024-09-12T12:13:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44c87c640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe448198b00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe438003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe43e7fc640 1 -- 172.21.15.137:0/1411546487 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43803f070 con 0x7fe448102570 2024-09-12T12:13:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe43e7fc640 1 -- 172.21.15.137:0/1411546487 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe438031e30 con 0x7fe448102570 2024-09-12T12:13:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe43e7fc640 1 -- 172.21.15.137:0/1411546487 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438035030 con 0x7fe448102570 2024-09-12T12:13:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44819d800 con 0x7fe448102570 2024-09-12T12:13:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.863+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4481a5140 con 0x7fe448102570 2024-09-12T12:13:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.864+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe448101200 con 0x7fe448102570 2024-09-12T12:13:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.865+0000 7fe43e7fc640 1 -- 172.21.15.137:0/1411546487 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fe438042070 con 0x7fe448102570 2024-09-12T12:13:49.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.866+0000 7fe43e7fc640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4340a2e00 0x7fe4340a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:49.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.866+0000 7fe44d07d640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4340a2e00 0x7fe4340a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:49.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.866+0000 7fe43e7fc640 1 -- 172.21.15.137:0/1411546487 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fe4380efe70 con 0x7fe448102570 2024-09-12T12:13:49.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.867+0000 7fe44d07d640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4340a2e00 0x7fe4340a52c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe444009890 tx=0x7fe444008040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:49.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:49.870+0000 7fe43e7fc640 1 -- 172.21.15.137:0/1411546487 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4380bc970 con 0x7fe448102570 2024-09-12T12:13:50.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.106+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe44810a0a0 con 0x7fe448102570 2024-09-12T12:13:50.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.107+0000 7fe43e7fc640 1 -- 172.21.15.137:0/1411546487 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7fe4380bc270 con 0x7fe448102570 2024-09-12T12:13:50.108 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:13:50.108 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"btime":"2024-09-12T12:13:12:456388+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:13:12.406179+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:13:50.108 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-09-12T12:13:50.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4340a2e00 msgr2=0x7fe4340a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4340a2e00 0x7fe4340a52c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe444009890 tx=0x7fe444008040 comp rx=0 tx=0).stop 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 msgr2=0x7fe448198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe448198b00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe438006010 tx=0x7fe438003c80 comp rx=0 tx=0).stop 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 shutdown_connections 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4340a2e00 0x7fe4340a52c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe448102ef0 0x7fe44819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe448102570 0x7fe448198b00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 --2- 172.21.15.137:0/1411546487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe448100e20 0x7fe4481985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.109+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 >> 172.21.15.137:0/1411546487 conn(0x7fe4480fc7d0 msgr2=0x7fe44806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:50.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.110+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 shutdown_connections 2024-09-12T12:13:50.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:50.110+0000 7fe44f2e1640 1 -- 172.21.15.137:0/1411546487 wait complete. 2024-09-12T12:13:50.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T12:13:50.832 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T12:13:51.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:50 smithi137 ceph-mon[30950]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:13:51.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:50 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1411546487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:50 smithi189 ceph-mon[35475]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:13:51.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:50 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1411546487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:51.091 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:13:51.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:50 smithi171 ceph-mon[35098]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:13:51.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:50 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1411546487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 -- 172.21.15.137:0/4181723936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914102790 msgr2=0x7f49141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 --2- 172.21.15.137:0/4181723936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914102790 0x7f49141067e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009900 tx=0x7f48fc02a410 comp rx=0 tx=0).stop 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 -- 172.21.15.137:0/4181723936 shutdown_connections 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 --2- 172.21.15.137:0/4181723936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4914106f10 0x7f491410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 --2- 172.21.15.137:0/4181723936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914102790 0x7f49141067e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 --2- 172.21.15.137:0/4181723936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914101de0 0x7f49141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 -- 172.21.15.137:0/4181723936 >> 172.21.15.137:0/4181723936 conn(0x7f49140fd8c0 msgr2=0x7f49140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 -- 172.21.15.137:0/4181723936 shutdown_connections 2024-09-12T12:13:51.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.625+0000 7f4918b72640 1 -- 172.21.15.137:0/4181723936 wait complete. 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 Processor -- start 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 -- start start 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914101de0 0x7f4914198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4914102790 0x7f4914198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914106f10 0x7f491419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4914111f30 con 0x7f4914101de0 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4914111db0 con 0x7f4914102790 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.626+0000 7f4918b72640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49141120b0 con 0x7f4914106f10 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914101de0 0x7f4914198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914101de0 0x7f4914198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36954/0 (socket says 172.21.15.137:36954) 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 -- 172.21.15.137:0/3738607615 learned_addr learned my addr 172.21.15.137:0/3738607615 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f4912d9d640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914106f10 0x7f491419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f490bfff640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4914102790 0x7f4914198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 -- 172.21.15.137:0/3738607615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914106f10 msgr2=0x7f491419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914106f10 0x7f491419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 -- 172.21.15.137:0/3738607615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4914102790 msgr2=0x7f4914198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4914102790 0x7f4914198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 -- 172.21.15.137:0/3738607615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491419d3e0 con 0x7f4914101de0 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f490bfff640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4914102790 0x7f4914198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f4912d9d640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914106f10 0x7f491419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f491259c640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914101de0 0x7f4914198430 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f490400d950 tx=0x7f490400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f490b7fe640 1 -- 172.21.15.137:0/3738607615 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904017030 con 0x7f4914101de0 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f490b7fe640 1 -- 172.21.15.137:0/3738607615 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4904004020 con 0x7f4914101de0 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f490b7fe640 1 -- 172.21.15.137:0/3738607615 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49040047a0 con 0x7f4914101de0 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491419d6d0 con 0x7f4914101de0 2024-09-12T12:13:51.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.627+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49141a4fb0 con 0x7f4914101de0 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.630+0000 7f490b7fe640 1 -- 172.21.15.137:0/3738607615 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f49040041d0 con 0x7f4914101de0 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.630+0000 7f491af4b740 1 Processor -- start 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.630+0000 7f491af4b740 1 -- start start 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491af4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491af4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491af4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491af4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d0025542a0 con 0x55d0025ef080 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491af4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d0024fbb10 con 0x55d0025f9940 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491af4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d002550260 con 0x55d0025ee760 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36958/0 (socket says 172.21.15.137:36958) 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491259c640 1 -- 172.21.15.137:0/4272041782 learned_addr learned my addr 172.21.15.137:0/4272041782 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490bfff640 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f4912d9d640 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490b7fe640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49000a2d80 0x7f49000a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490bfff640 1 -- 172.21.15.137:0/4272041782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 msgr2=0x55d0025eeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490bfff640 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025eeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490bfff640 1 -- 172.21.15.137:0/4272041782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 msgr2=0x55d0025f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490bfff640 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490bfff640 1 -- 172.21.15.137:0/4272041782 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0022c5b90 con 0x55d0025f9940 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f4912d9d640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49000a2d80 0x7f49000a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f491259c640 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.631+0000 7f490b7fe640 1 -- 172.21.15.137:0/3738607615 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f49040cf040 con 0x7f4914101de0 2024-09-12T12:13:51.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.632+0000 7f490bfff640 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025fbd30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009230 tx=0x7f48fc032cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18bc28b76f6b432 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:51.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.632+0000 7f4908ff9640 1 -- 172.21.15.137:0/4272041782 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc03f070 con 0x55d0025f9940 2024-09-12T12:13:51.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.632+0000 7f4908ff9640 1 -- 172.21.15.137:0/4272041782 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc0317b0 con 0x55d0025f9940 2024-09-12T12:13:51.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.632+0000 7f4908ff9640 1 -- 172.21.15.137:0/4272041782 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc035070 con 0x55d0025f9940 2024-09-12T12:13:51.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 -- 172.21.15.137:0/4272041782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 msgr2=0x55d0025fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f4912d9d640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49000a2d80 0x7f49000a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f490c005fd0 tx=0x7f490c004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:51.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025fbd30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009230 tx=0x7f48fc032cb0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 -- 172.21.15.137:0/4272041782 shutdown_connections 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025fbd30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 --2- 172.21.15.137:0/4272041782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025eeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 -- 172.21.15.137:0/4272041782 >> 172.21.15.137:0/4272041782 conn(0x55d0025eacf0 msgr2=0x55d0025eb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 -- 172.21.15.137:0/4272041782 shutdown_connections 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 -- 172.21.15.137:0/4272041782 wait complete. 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 -- start start 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.633+0000 7f491af4b740 1 Processor -- start 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025f7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d002600920 con 0x55d0025ef080 2024-09-12T12:13:51.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d0024dc6e0 con 0x55d0025f9940 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d002600aa0 con 0x55d0025ee760 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 10 client.0 _ll_register_callbacks cb 0x55d002500a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36966/0 (socket says 172.21.15.137:36966) 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 -- 172.21.15.137:0/2306207005 learned_addr learned my addr 172.21.15.137:0/2306207005 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491259c640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f4912d9d640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 -- 172.21.15.137:0/2306207005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 msgr2=0x55d0025f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 -- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 msgr2=0x55d0025f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 -- 172.21.15.137:0/2306207005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d002609e10 con 0x55d0025ef080 2024-09-12T12:13:51.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f4912d9d640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f490bfff640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f6d50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02a930 tx=0x7f48fc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729f9069c907ba76 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f48f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f48f9ffb640 1 -- 172.21.15.137:0/2306207005 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc03f070 con 0x55d0025ef080 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f48f9ffb640 1 -- 172.21.15.137:0/2306207005 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f48fc004440 con 0x55d0025ef080 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.634+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48fc0317b0 con 0x55d0025ef080 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f48f9ffb640 1 -- 172.21.15.137:0/2306207005 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f48fc054050 con 0x55d0025ef080 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 10 client.15414 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d00260c780 con 0x55d0025ef080 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f48f9ffb640 1 -- 172.21.15.137:0/2306207005 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc0317b0 con 0x55d0025ef080 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f48f9ffb640 1 -- 172.21.15.137:0/2306207005 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f48fc004990 con 0x55d0025ef080 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 10 client.15414 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 10 client.15414 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:13:51.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 20 client.15414 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00260fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 4 client.15414 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0025619d0 con 0x55d00260dad0 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.635+0000 7f491259c640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00260fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.636+0000 7f48f9ffb640 10 client.15414 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.636+0000 7f491259c640 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00260fec0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4a8b9270fa1aa9 server_cookie=1b4d1ccc5a075645 in_seq=0 out_seq=0 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f48f9ffb640 1 -- 172.21.15.137:0/2306207005 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0025619d0 con 0x55d00260dad0 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f48f9ffb640 10 client.15414 handle_command_reply: tid=0 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 client.15414 shutdown 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 msgr2=0x55d0025f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f6d50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02a930 tx=0x7f48fc009520 comp rx=0 tx=0).stop 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 shutdown_connections 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d0025f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00260fec0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ef080 0x55d0025f6d50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f48f9ffb640 0 client.15414 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:51.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 --2- 172.21.15.137:0/2306207005 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ee760 0x55d0025f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 >> 172.21.15.137:0/2306207005 conn(0x55d0025eacf0 msgr2=0x55d0025f0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:51.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 shutdown_connections 2024-09-12T12:13:51.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 1 -- 172.21.15.137:0/2306207005 wait complete. 2024-09-12T12:13:51.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.637+0000 7f491af4b740 20 client.15414 trim_cache size 0 max 16384 2024-09-12T12:13:51.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.654+0000 7f491af4b740 1 Processor -- start 2024-09-12T12:13:51.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.654+0000 7f491af4b740 1 -- start start 2024-09-12T12:13:51.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491af4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d0025f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491af4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ef080 0x55d002612ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491af4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d002613420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491af4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49141ade50 con 0x55d0025ee760 2024-09-12T12:13:51.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491af4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f49141ab720 con 0x55d0025f9940 2024-09-12T12:13:51.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491af4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49141a9590 con 0x55d0025ef080 2024-09-12T12:13:51.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d0025f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d0025f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36968/0 (socket says 172.21.15.137:36968) 2024-09-12T12:13:51.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 -- 172.21.15.137:0/1706501400 learned_addr learned my addr 172.21.15.137:0/1706501400 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:51.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f4912d9d640 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d002613420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f490bfff640 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ef080 0x55d002612ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 -- 172.21.15.137:0/1706501400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ef080 msgr2=0x55d002612ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ef080 0x55d002612ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 -- 172.21.15.137:0/1706501400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 msgr2=0x55d002613420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d002613420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 -- 172.21.15.137:0/1706501400 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0025eeeb0 con 0x55d0025ee760 2024-09-12T12:13:51.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f4912d9d640 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d002613420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:13:51.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f491259c640 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d0025f86e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4904005250 tx=0x7f4904097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9908b95b3361cf78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:51.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.655+0000 7f48fbfff640 1 -- 172.21.15.137:0/1706501400 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904083070 con 0x55d0025ee760 2024-09-12T12:13:51.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f48fbfff640 1 -- 172.21.15.137:0/1706501400 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490409bb40 con 0x55d0025ee760 2024-09-12T12:13:51.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f48fbfff640 1 -- 172.21.15.137:0/1706501400 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490409f7f0 con 0x55d0025ee760 2024-09-12T12:13:51.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 -- 172.21.15.137:0/1706501400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 msgr2=0x55d0025f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d0025f86e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4904005250 tx=0x7f4904097040 comp rx=0 tx=0).stop 2024-09-12T12:13:51.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 -- 172.21.15.137:0/1706501400 shutdown_connections 2024-09-12T12:13:51.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025f9940 0x55d002613420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025ef080 0x55d002612ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 --2- 172.21.15.137:0/1706501400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d0025f86e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 -- 172.21.15.137:0/1706501400 >> 172.21.15.137:0/1706501400 conn(0x55d0025eacf0 msgr2=0x55d0025f09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:51.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.656+0000 7f491af4b740 1 -- 172.21.15.137:0/1706501400 shutdown_connections 2024-09-12T12:13:51.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 -- 172.21.15.137:0/1706501400 wait complete. 2024-09-12T12:13:51.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 -- start start 2024-09-12T12:13:51.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 Processor -- start 2024-09-12T12:13:51.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d002611410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025ef080 0x55d002611950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025f9940 0x55d002677410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49141a9be0 con 0x55d0025ee760 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f49141ab050 con 0x55d0025ef080 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f49141aa2b0 con 0x55d0025f9940 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491af4b740 10 client.0 _ll_register_callbacks cb 0x7f48fc005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d002611410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d002611410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36984/0 (socket says 172.21.15.137:36984) 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f491259c640 1 -- 172.21.15.137:0/3569256618 learned_addr learned my addr 172.21.15.137:0/3569256618 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:51.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f490bfff640 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025ef080 0x55d002611950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.657+0000 7f4912d9d640 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025f9940 0x55d002677410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491259c640 1 -- 172.21.15.137:0/3569256618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025f9940 msgr2=0x55d002677410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491259c640 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025f9940 0x55d002677410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491259c640 1 -- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025ef080 msgr2=0x55d002611950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:51.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491259c640 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025ef080 0x55d002611950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491259c640 1 -- 172.21.15.137:0/3569256618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d002677a00 con 0x55d0025ee760 2024-09-12T12:13:51.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491259c640 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d002611410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4904009fd0 tx=0x7f490409ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490af55a38b94b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:51.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f48f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:51.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3569256618 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49040b4060 con 0x55d0025ee760 2024-09-12T12:13:51.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3569256618 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f4904097ab0 con 0x55d0025ee760 2024-09-12T12:13:51.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f490409bb40 con 0x55d0025ee760 2024-09-12T12:13:51.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3569256618 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4904083070 con 0x55d0025ee760 2024-09-12T12:13:51.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491af4b740 10 client.15420 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:51.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55d002678ca0 con 0x55d0025ee760 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.658+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3569256618 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49040a40b0 con 0x55d0025ee760 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3569256618 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f49040d2bd0 con 0x55d0025ee760 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f491af4b740 10 client.15420 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f491af4b740 10 client.15420 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f491af4b740 20 client.15420 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f491af4b740 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00267b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f491af4b740 4 client.15420 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T12:13:51.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x7f490409bb40 con 0x55d00260dad0 2024-09-12T12:13:51.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f490bfff640 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00267b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:51.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f48f9ffb640 10 client.15420 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:51.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.659+0000 7f490bfff640 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00267b410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8843eb972532b5f7 server_cookie=7ee328786b6126bb in_seq=0 out_seq=0 2024-09-12T12:13:51.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3569256618 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f490409bb40 con 0x55d00260dad0 2024-09-12T12:13:51.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f48f9ffb640 10 client.15420 handle_command_reply: tid=0 2024-09-12T12:13:51.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 client.15420 shutdown 2024-09-12T12:13:51.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 msgr2=0x55d002611410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d002611410 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4904009fd0 tx=0x7f490409ada0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 shutdown_connections 2024-09-12T12:13:51.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d0025f9940 0x55d002677410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d0025ef080 0x55d002611950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d00260dad0 0x55d00267b410 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f48f9ffb640 0 client.15420 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:51.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 --2- 172.21.15.137:0/3569256618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d0025ee760 0x55d002611410 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 >> 172.21.15.137:0/3569256618 conn(0x55d0025eacf0 msgr2=0x55d0025e9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:51.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 shutdown_connections 2024-09-12T12:13:51.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 1 -- 172.21.15.137:0/3569256618 wait complete. 2024-09-12T12:13:51.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.670+0000 7f491af4b740 20 client.15420 trim_cache size 0 max 16384 2024-09-12T12:13:51.675 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:13:51.675 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 24, 2024-09-12T12:13:51.676 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:13:51.676 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 2, 2024-09-12T12:13:51.676 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:13:51.676 INFO:teuthology.orchestra.run.smithi137.stdout: "3c9ff456": { 2024-09-12T12:13:51.676 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 2, 2024-09-12T12:13:51.676 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:13:51.676 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.5, 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:13:51.677 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:13:51.678 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:13:51.679 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:13:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49000a2d80 msgr2=0x7f49000a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49000a2d80 0x7f49000a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f490c005fd0 tx=0x7f490c004a40 comp rx=0 tx=0).stop 2024-09-12T12:13:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914101de0 msgr2=0x7f4914198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914101de0 0x7f4914198430 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f490400d950 tx=0x7f490400de10 comp rx=0 tx=0).stop 2024-09-12T12:13:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 shutdown_connections 2024-09-12T12:13:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49000a2d80 0x7f49000a5240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914106f10 0x7f491419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4914102790 0x7f4914198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 --2- 172.21.15.137:0/3738607615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4914101de0 0x7f4914198430 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:51.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 >> 172.21.15.137:0/3738607615 conn(0x7f49140fd8c0 msgr2=0x7f49140fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:51.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 shutdown_connections 2024-09-12T12:13:51.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:13:51.671+0000 7f4918b72640 1 -- 172.21.15.137:0/3738607615 wait complete. 2024-09-12T12:13:52.367 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3c9ff456', quiesce took 3.3 seconds. Will release after: 21.2 2024-09-12T12:13:53.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:52 smithi137 ceph-mon[30950]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T12:13:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:52 smithi189 ceph-mon[35475]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T12:13:53.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:52 smithi171 ceph-mon[35098]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-12T12:13:55.002 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:54 smithi171 ceph-mon[35098]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T12:13:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 -- 172.21.15.137:0/1646597735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f8101350 msgr2=0x7ff1f810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1646597735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f8101350 0x7ff1f810d880 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009cd0 tx=0x7ff1f002a8e0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 -- 172.21.15.137:0/1646597735 shutdown_connections 2024-09-12T12:13:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1646597735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f8101350 0x7ff1f810d880 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1646597735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f81009b0 0x7ff1f8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1646597735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f81069c0 0x7ff1f8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 -- 172.21.15.137:0/1646597735 >> 172.21.15.137:0/1646597735 conn(0x7ff1f8077990 msgr2=0x7ff1f80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 -- 172.21.15.137:0/1646597735 shutdown_connections 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.007+0000 7ff1fef86640 1 -- 172.21.15.137:0/1646597735 wait complete. 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 Processor -- start 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 -- start start 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f81009b0 0x7ff1f81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f8101350 0x7ff1f81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f81069c0 0x7ff1f819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8110e00 con 0x7ff1f81009b0 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8110c80 con 0x7ff1f8101350 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fef86640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8110f80 con 0x7ff1f81069c0 2024-09-12T12:13:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fcd22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f81009b0 0x7ff1f81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fcd22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f81009b0 0x7ff1f81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37002/0 (socket says 172.21.15.137:37002) 2024-09-12T12:13:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fcd22640 1 -- 172.21.15.137:0/1125783347 learned_addr learned my addr 172.21.15.137:0/1125783347 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f8101350 0x7ff1f81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.008+0000 7ff1fd523640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f81069c0 0x7ff1f819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fcd22640 1 -- 172.21.15.137:0/1125783347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f81069c0 msgr2=0x7ff1f819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fcd22640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f81069c0 0x7ff1f819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fcd22640 1 -- 172.21.15.137:0/1125783347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f8101350 msgr2=0x7ff1f81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fcd22640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f8101350 0x7ff1f81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fcd22640 1 -- 172.21.15.137:0/1125783347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f819d290 con 0x7ff1f81009b0 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fcd22640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f81009b0 0x7ff1f81982a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00c920 tx=0x7ff1ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1f5ffb640 1 -- 172.21.15.137:0/1125783347 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec007c70 con 0x7ff1f81009b0 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1f5ffb640 1 -- 172.21.15.137:0/1125783347 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1ec007e10 con 0x7ff1f81009b0 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f819d580 con 0x7ff1f81009b0 2024-09-12T12:13:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1f5ffb640 1 -- 172.21.15.137:0/1125783347 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec018890 con 0x7ff1f81009b0 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.009+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a4e10 con 0x7ff1f81009b0 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.011+0000 7ff201dfd740 1 Processor -- start 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff201dfd740 1 -- start start 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fc0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fc34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff201dfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624d5f162f0 con 0x5624d5fb22f0 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff201dfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624d5ebe270 con 0x5624d5fb19d0 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff201dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5624d5f122b0 con 0x5624d5fc10c0 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fb1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:34028/0 (socket says 172.21.15.137:34028) 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 -- 172.21.15.137:0/3336571095 learned_addr learned my addr 172.21.15.137:0/3336571095 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fc34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 -- 172.21.15.137:0/3336571095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 msgr2=0x5624d5fc34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fc34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 -- 172.21.15.137:0/3336571095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 msgr2=0x5624d5fc0b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T12:13:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fc0b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1fd523640 1 -- 172.21.15.137:0/3336571095 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d5ebd340 con 0x5624d5fb19d0 2024-09-12T12:13:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.012+0000 7ff1f5ffb640 1 -- 172.21.15.137:0/1125783347 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7ff1ec018a70 con 0x7ff1f81009b0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.013+0000 7ff1fd523640 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fb1db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0035000 tx=0x7ff1f0009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6214921760b5ac7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.013+0000 7ff1e2ffd640 1 -- 172.21.15.137:0/3336571095 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0031ac0 con 0x5624d5fb19d0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.013+0000 7ff1e2ffd640 1 -- 172.21.15.137:0/3336571095 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0036040 con 0x5624d5fb19d0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.013+0000 7ff1e2ffd640 1 -- 172.21.15.137:0/3336571095 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0004760 con 0x5624d5fb19d0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.013+0000 7ff1f5ffb640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff1e80a2ec0 0x7ff1e80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.013+0000 7ff1fcd22640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff1e80a2ec0 0x7ff1e80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.013+0000 7ff1f5ffb640 1 -- 172.21.15.137:0/1125783347 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7ff1ec0cf5d0 con 0x7ff1f81009b0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 -- 172.21.15.137:0/3336571095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 msgr2=0x5624d5fb1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fb1db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0035000 tx=0x7ff1f0009a20 comp rx=0 tx=0).stop 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 -- 172.21.15.137:0/3336571095 shutdown_connections 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fc34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff1fcd22640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff1e80a2ec0 0x7ff1e80a5380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0ccd10 tx=0x7ff1ec086040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fc0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3336571095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fb1db0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 -- 172.21.15.137:0/3336571095 >> 172.21.15.137:0/3336571095 conn(0x5624d5fae060 msgr2=0x5624d5fae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 -- 172.21.15.137:0/3336571095 shutdown_connections 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 -- 172.21.15.137:0/3336571095 wait complete. 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 -- start start 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.014+0000 7ff201dfd740 1 Processor -- start 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fbe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fbe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fbeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff201dfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5624d5fc3c90 con 0x5624d5fb22f0 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff201dfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5624d5e9f290 con 0x5624d5fb19d0 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff201dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5624d5fc3e10 con 0x5624d5fc10c0 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff201dfd740 10 client.0 _ll_register_callbacks cb 0x5624d5ed0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fbe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fbe610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37014/0 (socket says 172.21.15.137:37014) 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 -- 172.21.15.137:0/4144707847 learned_addr learned my addr 172.21.15.137:0/4144707847 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1fd523640 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fbeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1fd523640 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fbe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 -- 172.21.15.137:0/4144707847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 msgr2=0x5624d5fbeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fbeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 -- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 msgr2=0x5624d5fbe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fbe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 -- 172.21.15.137:0/4144707847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d5c29210 con 0x5624d5fb22f0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fbe610 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400b9f0 tx=0x7ff1e400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31c1fcd6a6ac2f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.015+0000 7ff1d7fff640 1 -- 172.21.15.137:0/4144707847 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e401b440 con 0x5624d5fb22f0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0036040 con 0x5624d5fb22f0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff1d7fff640 1 -- 172.21.15.137:0/4144707847 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7ff1e401b5e0 con 0x5624d5fb22f0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff1d7fff640 1 -- 172.21.15.137:0/4144707847 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff1e401b7e0 con 0x5624d5fb22f0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 10 client.15429 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5624d5cf5620 con 0x5624d5fb22f0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff1d7fff640 1 -- 172.21.15.137:0/4144707847 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0036040 con 0x5624d5fb22f0 2024-09-12T12:13:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff1d7fff640 1 -- 172.21.15.137:0/4144707847 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff1e4019020 con 0x5624d5fb22f0 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 10 client.15429 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 20 client.15429 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d5fd1180 0x5624d5fd3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 4 client.15429 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.016+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624d5f23a20 con 0x5624d5fd1180 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.017+0000 7ff1fd523640 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d5fd1180 0x5624d5fd3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.017+0000 7ff1fd523640 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d5fd1180 0x5624d5fd3570 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88441383b9ac45d4 server_cookie=1d8d66bb667204aa in_seq=0 out_seq=0 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.017+0000 7ff1d7fff640 10 client.15429 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff1d7fff640 1 -- 172.21.15.137:0/4144707847 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624d5f23a20 con 0x5624d5fd1180 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff1d7fff640 10 client.15429 handle_command_reply: tid=0 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 client.15429 shutdown 2024-09-12T12:13:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 msgr2=0x5624d5fbe610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fbe610 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400b9f0 tx=0x7ff1e400beb0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 shutdown_connections 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fbeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d5fd1180 0x5624d5fd3570 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb22f0 0x5624d5fbe610 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 --2- 172.21.15.137:0/4144707847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb19d0 0x5624d5fbe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff1d7fff640 0 client.15429 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 >> 172.21.15.137:0/4144707847 conn(0x5624d5fae060 msgr2=0x5624d5fb3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 shutdown_connections 2024-09-12T12:13:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 1 -- 172.21.15.137:0/4144707847 wait complete. 2024-09-12T12:13:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.018+0000 7ff201dfd740 20 client.15429 trim_cache size 0 max 16384 2024-09-12T12:13:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 Processor -- start 2024-09-12T12:13:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 -- start start 2024-09-12T12:13:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fb22f0 0x5624d5fd8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fc10c0 0x5624d5fd9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81add20 con 0x5624d5fb19d0 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81ab5f0 con 0x5624d5fc10c0 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.032+0000 7ff201dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81a9440 con 0x5624d5fb22f0 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37024/0 (socket says 172.21.15.137:37024) 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 -- 172.21.15.137:0/3871134878 learned_addr learned my addr 172.21.15.137:0/3871134878 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fb22f0 0x5624d5fd8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 -- 172.21.15.137:0/3871134878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fb22f0 msgr2=0x5624d5fd8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fb22f0 0x5624d5fd8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 -- 172.21.15.137:0/3871134878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fc10c0 msgr2=0x5624d5fd9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fc10c0 0x5624d5fd9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 -- 172.21.15.137:0/3871134878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d5fb8e90 con 0x5624d5fb19d0 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1fd523640 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fad0b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005cc0 tx=0x7ff1f0004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467fb07d383a4a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1e27fc640 1 -- 172.21.15.137:0/3871134878 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0004720 con 0x5624d5fb19d0 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1e27fc640 1 -- 172.21.15.137:0/3871134878 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0036040 con 0x5624d5fb19d0 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.033+0000 7ff1e27fc640 1 -- 172.21.15.137:0/3871134878 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0003cf0 con 0x5624d5fb19d0 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 -- 172.21.15.137:0/3871134878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 msgr2=0x5624d5fad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fad0b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005cc0 tx=0x7ff1f0004260 comp rx=0 tx=0).stop 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 -- 172.21.15.137:0/3871134878 shutdown_connections 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fc10c0 0x5624d5fd9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fb22f0 0x5624d5fd8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 --2- 172.21.15.137:0/3871134878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fad0b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 -- 172.21.15.137:0/3871134878 >> 172.21.15.137:0/3871134878 conn(0x5624d5fae060 msgr2=0x5624d5fac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 -- 172.21.15.137:0/3871134878 shutdown_connections 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 -- 172.21.15.137:0/3871134878 wait complete. 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 -- start start 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.034+0000 7ff201dfd740 1 Processor -- start 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fc51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb22f0 0x5624d5fc56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff201dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fc5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff201dfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81a9ab0 con 0x5624d5fb19d0 2024-09-12T12:13:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff201dfd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81aaf20 con 0x5624d5fb22f0 2024-09-12T12:13:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff201dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f81aa180 con 0x5624d5fc10c0 2024-09-12T12:13:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff201dfd740 10 client.0 _ll_register_callbacks cb 0x7ff1e4004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:13:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fc51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fc51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37028/0 (socket says 172.21.15.137:37028) 2024-09-12T12:13:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 -- 172.21.15.137:0/992960773 learned_addr learned my addr 172.21.15.137:0/992960773 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 -- 172.21.15.137:0/992960773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 msgr2=0x5624d5fc5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fc5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb22f0 0x5624d5fc56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 -- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb22f0 msgr2=0x5624d5fc56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb22f0 0x5624d5fc56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 -- 172.21.15.137:0/992960773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d603b0e0 con 0x5624d5fb19d0 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1fd523640 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fc51b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0002230 tx=0x7ff1f0004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b1e41457294e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.035+0000 7ff1d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff1d7fff640 1 -- 172.21.15.137:0/992960773 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0004d40 con 0x5624d5fb19d0 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff1d7fff640 1 -- 172.21.15.137:0/992960773 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7ff1f00318c0 con 0x5624d5fb19d0 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f0036040 con 0x5624d5fb19d0 2024-09-12T12:13:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff1d7fff640 1 -- 172.21.15.137:0/992960773 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff1f0064050 con 0x5624d5fb19d0 2024-09-12T12:13:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 10 client.15435 fetch_fsmap learned FSMap version 24 2024-09-12T12:13:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5624d603c3f0 con 0x5624d5fb19d0 2024-09-12T12:13:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff1d7fff640 1 -- 172.21.15.137:0/992960773 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1f0036040 con 0x5624d5fb19d0 2024-09-12T12:13:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff1d7fff640 1 -- 172.21.15.137:0/992960773 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7ff1f005c020 con 0x5624d5fb19d0 2024-09-12T12:13:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:13:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:13:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 20 client.15435 populate_metadata read hostname 'smithi137' 2024-09-12T12:13:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d603c610 0x5624d603ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:13:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 4 client.15435 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:13:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.036+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624d5f1dd70 con 0x5624d603c610 2024-09-12T12:13:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.037+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d603c610 0x5624d603ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:13:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.037+0000 7ff1d7fff640 10 client.15435 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.037+0000 7ff1f7fff640 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d603c610 0x5624d603ea00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb12ba2548da3b server_cookie=149ada79328faa69 in_seq=0 out_seq=0 2024-09-12T12:13:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff1d7fff640 1 -- 172.21.15.137:0/992960773 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624d5f1dd70 con 0x5624d603c610 2024-09-12T12:13:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff1d7fff640 10 client.15435 handle_command_reply: tid=0 2024-09-12T12:13:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 client.15435 shutdown 2024-09-12T12:13:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 msgr2=0x5624d5fc51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fc51b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0002230 tx=0x7ff1f0004460 comp rx=0 tx=0).stop 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 shutdown_connections 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5624d5fc10c0 0x5624d5fc5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5624d5fb22f0 0x5624d5fc56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5624d603c610 0x5624d603ea00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 --2- 172.21.15.137:0/992960773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5624d5fb19d0 0x5624d5fc51b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff1d7fff640 0 client.15435 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 >> 172.21.15.137:0/992960773 conn(0x5624d5fae060 msgr2=0x5624d5fad1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 shutdown_connections 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 1 -- 172.21.15.137:0/992960773 wait complete. 2024-09-12T12:13:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.044+0000 7ff201dfd740 20 client.15435 trim_cache size 0 max 16384 2024-09-12T12:13:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:13:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:13:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:13:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff1e80a2ec0 msgr2=0x7ff1e80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff1e80a2ec0 0x7ff1e80a5380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0ccd10 tx=0x7ff1ec086040 comp rx=0 tx=0).stop 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f81009b0 msgr2=0x7ff1f81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f81009b0 0x7ff1f81982a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00c920 tx=0x7ff1ec00cde0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 shutdown_connections 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff1e80a2ec0 0x7ff1e80a5380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff1f81069c0 0x7ff1f819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1f8101350 0x7ff1f81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 --2- 172.21.15.137:0/1125783347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1f81009b0 0x7ff1f81982a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:13:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 >> 172.21.15.137:0/1125783347 conn(0x7ff1f8077990 msgr2=0x7ff1f80ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:13:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 shutdown_connections 2024-09-12T12:13:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:13:55.045+0000 7ff1fef86640 1 -- 172.21.15.137:0/1125783347 wait complete. 2024-09-12T12:13:55.052 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:54 smithi137 ceph-mon[30950]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T12:13:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:54 smithi189 ceph-mon[35475]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-12T12:13:57.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:56 smithi137 ceph-mon[30950]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T12:13:57.092 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:56 smithi189 ceph-mon[35475]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T12:13:57.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:56 smithi171 ceph-mon[35098]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-12T12:13:59.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:13:58 smithi137 ceph-mon[30950]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T12:13:59.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:13:58 smithi189 ceph-mon[35475]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T12:13:59.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:13:58 smithi171 ceph-mon[35098]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-12T12:14:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:00 smithi137 ceph-mon[30950]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T12:14:01.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:14:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:14:00.828+0000 7f4f145ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:14:01.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:14:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:14:00.828+0000 7f4f145ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:14:01.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:14:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:14:00.828+0000 7f4f145ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:14:01.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:14:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:14:00.828+0000 7f4f145ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:14:01.049 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:14:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:14:00.828+0000 7f4f145ba640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T12:14:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:00 smithi189 ceph-mon[35475]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T12:14:01.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:00 smithi171 ceph-mon[35098]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-12T12:14:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:02 smithi137 ceph-mon[30950]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T12:14:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:02 smithi137 ceph-mon[30950]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-09-12T12:14:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:02 smithi189 ceph-mon[35475]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T12:14:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:02 smithi189 ceph-mon[35475]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-09-12T12:14:03.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:02 smithi171 ceph-mon[35098]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T12:14:03.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:02 smithi171 ceph-mon[35098]: mgrmap e24: x(active, since 48s), standbys: y, z 2024-09-12T12:14:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:04 smithi137 ceph-mon[30950]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T12:14:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:04 smithi189 ceph-mon[35475]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T12:14:05.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:04 smithi171 ceph-mon[35098]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-12T12:14:07.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:06 smithi137 ceph-mon[30950]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-12T12:14:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:06 smithi189 ceph-mon[35475]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-12T12:14:07.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:06 smithi171 ceph-mon[35098]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-12T12:14:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.175+0000 7f9dda2ba640 1 -- 172.21.15.137:0/623657216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd4102d20 msgr2=0x7f9dd4103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.175+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/623657216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd4102d20 0x7f9dd4103100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009900 tx=0x7f9dc402a480 comp rx=0 tx=0).stop 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.176+0000 7f9dda2ba640 1 -- 172.21.15.137:0/623657216 shutdown_connections 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.176+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/623657216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd410c0f0 0x7f9dd410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.176+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/623657216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd4103640 0x7f9dd410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.176+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/623657216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd4102d20 0x7f9dd4103100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.176+0000 7f9dda2ba640 1 -- 172.21.15.137:0/623657216 >> 172.21.15.137:0/623657216 conn(0x7f9dd4077990 msgr2=0x7f9dd40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.176+0000 7f9dda2ba640 1 -- 172.21.15.137:0/623657216 shutdown_connections 2024-09-12T12:14:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.176+0000 7f9dda2ba640 1 -- 172.21.15.137:0/623657216 wait complete. 2024-09-12T12:14:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.177+0000 7f9dda2ba640 1 Processor -- start 2024-09-12T12:14:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.177+0000 7f9dda2ba640 1 -- start start 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.177+0000 7f9dda2ba640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd4102d20 0x7f9dd4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.177+0000 7f9dda2ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd4103640 0x7f9dd4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.177+0000 7f9dda2ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd410c0f0 0x7f9dd419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dda2ba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4111ec0 con 0x7f9dd410c0f0 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dda2ba640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4111d40 con 0x7f9dd4102d20 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dda2ba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd4112040 con 0x7f9dd4103640 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd410c0f0 0x7f9dd419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd410c0f0 0x7f9dd419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53490/0 (socket says 172.21.15.137:53490) 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd3fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd4102d20 0x7f9dd4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 -- 172.21.15.137:0/265701254 learned_addr learned my addr 172.21.15.137:0/265701254 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd4103640 0x7f9dd4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 -- 172.21.15.137:0/265701254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd4103640 msgr2=0x7f9dd4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd4103640 0x7f9dd4198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 -- 172.21.15.137:0/265701254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd4102d20 msgr2=0x7f9dd4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd4102d20 0x7f9dd4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 -- 172.21.15.137:0/265701254 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd419d300 con 0x7f9dd410c0f0 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.178+0000 7f9dd8857640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd410c0f0 0x7f9dd419cbd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00eb10 tx=0x7f9dcc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.179+0000 7f9dd17fa640 1 -- 172.21.15.137:0/265701254 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc004b20 con 0x7f9dd410c0f0 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.179+0000 7f9dd17fa640 1 -- 172.21.15.137:0/265701254 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dcc004cc0 con 0x7f9dd410c0f0 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.179+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd419d5f0 con 0x7f9dd410c0f0 2024-09-12T12:14:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.179+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd41a4e80 con 0x7f9dd410c0f0 2024-09-12T12:14:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.179+0000 7f9dd17fa640 1 -- 172.21.15.137:0/265701254 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc010760 con 0x7f9dd410c0f0 2024-09-12T12:14:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.182+0000 7f9dd17fa640 1 -- 172.21.15.137:0/265701254 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7f9dcc010900 con 0x7f9dd410c0f0 2024-09-12T12:14:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 Processor -- start 2024-09-12T12:14:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 -- start start 2024-09-12T12:14:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 0x5605ba1690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba1709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5605ba0bf2f0 con 0x5605ba165000 2024-09-12T12:14:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5605ba067270 con 0x5605ba168cc0 2024-09-12T12:14:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9ddd059740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605ba0bb2b0 con 0x5605ba164660 2024-09-12T12:14:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9dd37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba1709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9dd37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba1709c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53502/0 (socket says 172.21.15.137:53502) 2024-09-12T12:14:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9dd37fe640 1 -- 172.21.15.137:0/423852473 learned_addr learned my addr 172.21.15.137:0/423852473 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.183+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd3fff640 1 -- 172.21.15.137:0/423852473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 msgr2=0x5605ba1690a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 0x5605ba1690a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd3fff640 1 -- 172.21.15.137:0/423852473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 msgr2=0x5605ba1709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba1709c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd3fff640 1 -- 172.21.15.137:0/423852473 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605ba066340 con 0x5605ba164660 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba1709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba164ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4005940 tx=0x7f9dc4005e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2824a4e4618bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd17fa640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9dbc0a2f40 0x7f9dbc0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dba7fc640 1 -- 172.21.15.137:0/423852473 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4040070 con 0x5605ba164660 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9dbc0a2f40 0x7f9dbc0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.184+0000 7f9dba7fc640 1 -- 172.21.15.137:0/423852473 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc4004030 con 0x5605ba164660 2024-09-12T12:14:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.185+0000 7f9dba7fc640 1 -- 172.21.15.137:0/423852473 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4035040 con 0x5605ba164660 2024-09-12T12:14:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.185+0000 7f9dd17fa640 1 -- 172.21.15.137:0/265701254 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f9dcc0cfa30 con 0x7f9dd410c0f0 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.185+0000 7f9ddd059740 1 -- 172.21.15.137:0/423852473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 msgr2=0x5605ba164ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.185+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9dbc0a2f40 0x7f9dbc0a5400 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8004040 tx=0x7f9dc800b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.185+0000 7f9ddd059740 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba164ac0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4005940 tx=0x7f9dc4005e00 comp rx=0 tx=0).stop 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 -- 172.21.15.137:0/423852473 shutdown_connections 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba1709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba164ac0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 --2- 172.21.15.137:0/423852473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 0x5605ba1690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 -- 172.21.15.137:0/423852473 >> 172.21.15.137:0/423852473 conn(0x5605ba156b10 msgr2=0x5605ba156ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 -- 172.21.15.137:0/423852473 shutdown_connections 2024-09-12T12:14:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 -- 172.21.15.137:0/423852473 wait complete. 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 -- start start 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.186+0000 7f9ddd059740 1 Processor -- start 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba16ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba165000 0x5605ba16e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba168cc0 0x5605ba16e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9ddd059740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5605ba171050 con 0x5605ba168cc0 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9ddd059740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5605ba048290 con 0x5605ba165000 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9ddd059740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605ba1711d0 con 0x5605ba164660 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9dd8857640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba168cc0 0x5605ba16e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9dd3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba16ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9dd8857640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba168cc0 0x5605ba16e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53510/0 (socket says 172.21.15.137:53510) 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9dd8857640 1 -- 172.21.15.137:0/1681165457 learned_addr learned my addr 172.21.15.137:0/1681165457 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9ddd059740 10 client.0 _ll_register_callbacks cb 0x5605ba079140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.187+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba165000 0x5605ba16e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9dd8857640 1 -- 172.21.15.137:0/1681165457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 msgr2=0x5605ba16ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9dd8857640 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba16ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9dd8857640 1 -- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba165000 msgr2=0x5605ba16e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9dd8857640 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba165000 0x5605ba16e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9dd8857640 1 -- 172.21.15.137:0/1681165457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b9dd2210 con 0x5605ba168cc0 2024-09-12T12:14:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba165000 0x5605ba16e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:14:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9dd8857640 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba168cc0 0x5605ba16e850 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc09cbc0 tx=0x7f9dcc09c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c44a60b4d1a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9db37fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:14:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9db37fe640 1 -- 172.21.15.137:0/1681165457 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0b9070 con 0x5605ba168cc0 2024-09-12T12:14:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc4004030 con 0x5605ba168cc0 2024-09-12T12:14:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9db37fe640 1 -- 172.21.15.137:0/1681165457 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f9dcc0a56a0 con 0x5605ba168cc0 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9db37fe640 1 -- 172.21.15.137:0/1681165457 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dcc082050 con 0x5605ba168cc0 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9ddd059740 10 client.15453 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.188+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5605b9e9e620 con 0x5605ba168cc0 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9db37fe640 1 -- 172.21.15.137:0/1681165457 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc4004030 con 0x5605ba168cc0 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9db37fe640 1 -- 172.21.15.137:0/1681165457 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dcc07bce0 con 0x5605ba168cc0 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9ddd059740 10 client.15453 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9ddd059740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9ddd059740 20 client.15453 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9ddd059740 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba180900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9ddd059740 4 client.15453 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605ba0cca20 con 0x5605ba17e4d0 2024-09-12T12:14:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.189+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba180900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.190+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba180900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6155b5deff03ab9 server_cookie=451bf538c8921951 in_seq=0 out_seq=0 2024-09-12T12:14:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.190+0000 7f9db37fe640 10 client.15453 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.190+0000 7f9db37fe640 1 -- 172.21.15.137:0/1681165457 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605ba0cca20 con 0x5605ba17e4d0 2024-09-12T12:14:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.190+0000 7f9db37fe640 10 client.15453 handle_command_reply: tid=0 2024-09-12T12:14:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 client.15453 shutdown 2024-09-12T12:14:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba168cc0 msgr2=0x5605ba16e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba168cc0 0x5605ba16e850 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc09cbc0 tx=0x7f9dcc09c8e0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 shutdown_connections 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba180900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9db37fe640 0 client.15453 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba168cc0 0x5605ba16e850 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba165000 0x5605ba16e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 --2- 172.21.15.137:0/1681165457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba16ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 >> 172.21.15.137:0/1681165457 conn(0x5605ba156b10 msgr2=0x5605ba163e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 shutdown_connections 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 1 -- 172.21.15.137:0/1681165457 wait complete. 2024-09-12T12:14:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.191+0000 7f9ddd059740 20 client.15453 trim_cache size 0 max 16384 2024-09-12T12:14:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 Processor -- start 2024-09-12T12:14:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 -- start start 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba164660 0x5605ba186210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba186750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba168cc0 0x5605ba16f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0010f40 con 0x5605ba165000 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000e810 con 0x5605ba164660 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.219+0000 7f9ddd059740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000c660 con 0x5605ba168cc0 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba186750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba186750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53520/0 (socket says 172.21.15.137:53520) 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 -- 172.21.15.137:0/619408662 learned_addr learned my addr 172.21.15.137:0/619408662 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd8857640 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba168cc0 0x5605ba16f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba164660 0x5605ba186210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 -- 172.21.15.137:0/619408662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba168cc0 msgr2=0x5605ba16f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba168cc0 0x5605ba16f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 -- 172.21.15.137:0/619408662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba164660 msgr2=0x5605ba186210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba164660 0x5605ba186210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 -- 172.21.15.137:0/619408662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605ba17a350 con 0x5605ba165000 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd8857640 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba168cc0 0x5605ba16f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba164660 0x5605ba186210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba186750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8003e90 tx=0x7f9dc8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bb13f2234b5124 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9db9ffb640 1 -- 172.21.15.137:0/619408662 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc802c070 con 0x5605ba165000 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9db9ffb640 1 -- 172.21.15.137:0/619408662 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc801edb0 con 0x5605ba165000 2024-09-12T12:14:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.220+0000 7f9db9ffb640 1 -- 172.21.15.137:0/619408662 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8027d20 con 0x5605ba165000 2024-09-12T12:14:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.221+0000 7f9ddd059740 1 -- 172.21.15.137:0/619408662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 msgr2=0x5605ba186750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.221+0000 7f9ddd059740 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba186750 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8003e90 tx=0x7f9dc8016000 comp rx=0 tx=0).stop 2024-09-12T12:14:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.221+0000 7f9ddd059740 1 -- 172.21.15.137:0/619408662 shutdown_connections 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.221+0000 7f9ddd059740 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba168cc0 0x5605ba16f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.221+0000 7f9ddd059740 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba186750 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.221+0000 7f9ddd059740 1 --2- 172.21.15.137:0/619408662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba164660 0x5605ba186210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.222+0000 7f9ddd059740 1 -- 172.21.15.137:0/619408662 >> 172.21.15.137:0/619408662 conn(0x5605ba156b10 msgr2=0x5605ba168250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.222+0000 7f9ddd059740 1 -- 172.21.15.137:0/619408662 shutdown_connections 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.222+0000 7f9ddd059740 1 -- 172.21.15.137:0/619408662 wait complete. 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.222+0000 7f9ddd059740 1 -- start start 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.222+0000 7f9ddd059740 1 Processor -- start 2024-09-12T12:14:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba1764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba176a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9ddd059740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 0x5605ba171480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9ddd059740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000ccd0 con 0x5605ba165000 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9ddd059740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000e140 con 0x5605ba168cc0 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9ddd059740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc000d3a0 con 0x5605ba164660 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9ddd059740 10 client.0 _ll_register_callbacks cb 0x7f9dcc09cd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba176a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba176a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53532/0 (socket says 172.21.15.137:53532) 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 -- 172.21.15.137:0/2383049277 learned_addr learned my addr 172.21.15.137:0/2383049277 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba1764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd8857640 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 0x5605ba171480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 -- 172.21.15.137:0/2383049277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 msgr2=0x5605ba1764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba1764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 -- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 msgr2=0x5605ba171480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 0x5605ba171480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 -- 172.21.15.137:0/2383049277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605ba1719c0 con 0x5605ba165000 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9dd37fe640 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba176a10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8018040 tx=0x7f9dc801e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd34e091e037032f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.223+0000 7f9db37fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9db37fe640 1 -- 172.21.15.137:0/2383049277 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc802c070 con 0x5605ba165000 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9db37fe640 1 -- 172.21.15.137:0/2383049277 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f9dc8027d70 con 0x5605ba165000 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dc801edb0 con 0x5605ba165000 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9db37fe640 1 -- 172.21.15.137:0/2383049277 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9dc8003b10 con 0x5605ba165000 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 10 client.15459 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5605ba1e99d0 con 0x5605ba165000 2024-09-12T12:14:08.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9db37fe640 1 -- 172.21.15.137:0/2383049277 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc801edb0 con 0x5605ba165000 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9db37fe640 1 -- 172.21.15.137:0/2383049277 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9dc803dc30 con 0x5605ba165000 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 10 client.15459 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 20 client.15459 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba1ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 4 client.15459 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.224+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605ba0c6d70 con 0x5605ba17e4d0 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.225+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba1ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.225+0000 7f9db37fe640 10 client.15459 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:08.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.225+0000 7f9dd3fff640 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba1ec070 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d0a5cc6929009f server_cookie=4e7fcfcb98488a71 in_seq=0 out_seq=0 2024-09-12T12:14:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9db37fe640 1 -- 172.21.15.137:0/2383049277 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605ba0c6d70 con 0x5605ba17e4d0 2024-09-12T12:14:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9db37fe640 10 client.15459 handle_command_reply: tid=0 2024-09-12T12:14:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 client.15459 shutdown 2024-09-12T12:14:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 msgr2=0x5605ba176a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba176a10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8018040 tx=0x7f9dc801e300 comp rx=0 tx=0).stop 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 shutdown_connections 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605ba168cc0 0x5605ba171480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605ba17e4d0 0x5605ba1ec070 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605ba165000 0x5605ba176a10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9db37fe640 0 client.15459 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 --2- 172.21.15.137:0/2383049277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605ba164660 0x5605ba1764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.231+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 >> 172.21.15.137:0/2383049277 conn(0x5605ba156b10 msgr2=0x5605ba163e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.232+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 shutdown_connections 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.232+0000 7f9ddd059740 1 -- 172.21.15.137:0/2383049277 wait complete. 2024-09-12T12:14:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.232+0000 7f9ddd059740 20 client.15459 trim_cache size 0 max 16384 2024-09-12T12:14:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:14:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:14:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:14:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.232+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9dbc0a2f40 msgr2=0x7f9dbc0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.232+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9dbc0a2f40 0x7f9dbc0a5400 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8004040 tx=0x7f9dc800b390 comp rx=0 tx=0).stop 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.232+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd410c0f0 msgr2=0x7f9dd419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.232+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd410c0f0 0x7f9dd419cbd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00eb10 tx=0x7f9dcc00c2d0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 shutdown_connections 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9dbc0a2f40 0x7f9dbc0a5400 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dd410c0f0 0x7f9dd419cbd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9dd4103640 0x7f9dd4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 --2- 172.21.15.137:0/265701254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9dd4102d20 0x7f9dd4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 >> 172.21.15.137:0/265701254 conn(0x7f9dd4077990 msgr2=0x7f9dd40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 shutdown_connections 2024-09-12T12:14:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:08.233+0000 7f9dda2ba640 1 -- 172.21.15.137:0/265701254 wait complete. 2024-09-12T12:14:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:08 smithi137 ceph-mon[30950]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T12:14:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:08 smithi189 ceph-mon[35475]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T12:14:09.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:08 smithi171 ceph-mon[35098]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-09-12T12:14:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:10 smithi137 ceph-mon[30950]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-12T12:14:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:10 smithi189 ceph-mon[35475]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-12T12:14:11.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:10 smithi171 ceph-mon[35098]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-12T12:14:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:12 smithi137 ceph-mon[30950]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T12:14:13.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:12 smithi189 ceph-mon[35475]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T12:14:13.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:12 smithi171 ceph-mon[35098]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T12:14:13.568 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:14:13.847 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.405+0000 7fb5724a6640 1 -- 172.21.15.137:0/4047916027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 msgr2=0x7fb56c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.405+0000 7fb5724a6640 1 --2- 172.21.15.137:0/4047916027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 0x7fb56c107b90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb560009930 tx=0x7fb56002a420 comp rx=0 tx=0).stop 2024-09-12T12:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.406+0000 7fb5724a6640 1 -- 172.21.15.137:0/4047916027 shutdown_connections 2024-09-12T12:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.406+0000 7fb5724a6640 1 --2- 172.21.15.137:0/4047916027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb56c1082a0 0x7fb56c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.406+0000 7fb5724a6640 1 --2- 172.21.15.137:0/4047916027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 0x7fb56c107b90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.406+0000 7fb5724a6640 1 --2- 172.21.15.137:0/4047916027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 0x7fb56c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.406+0000 7fb5724a6640 1 -- 172.21.15.137:0/4047916027 >> 172.21.15.137:0/4047916027 conn(0x7fb56c0fc7d0 msgr2=0x7fb56c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.406+0000 7fb5724a6640 1 -- 172.21.15.137:0/4047916027 shutdown_connections 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.406+0000 7fb5724a6640 1 -- 172.21.15.137:0/4047916027 wait complete. 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.407+0000 7fb5724a6640 1 Processor -- start 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.407+0000 7fb5724a6640 1 -- start start 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.407+0000 7fb5724a6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 0x7fb56c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.407+0000 7fb5724a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 0x7fb56c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.407+0000 7fb5724a6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb56c1082a0 0x7fb56c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.407+0000 7fb5724a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1120a0 con 0x7fb56c1036e0 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb5724a6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c111f20 con 0x7fb56c102d30 2024-09-12T12:14:14.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb56b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 0x7fb56c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb5724a6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c112220 con 0x7fb56c1082a0 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb56bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 0x7fb56c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb56bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 0x7fb56c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39266/0 (socket says 172.21.15.137:39266) 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb56bfff640 1 -- 172.21.15.137:0/1621202512 learned_addr learned my addr 172.21.15.137:0/1621202512 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb570a43640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb56c1082a0 0x7fb56c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb570a43640 1 -- 172.21.15.137:0/1621202512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 msgr2=0x7fb56c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb570a43640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 0x7fb56c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb570a43640 1 -- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 msgr2=0x7fb56c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb570a43640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 0x7fb56c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb570a43640 1 -- 172.21.15.137:0/1621202512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c19d5f0 con 0x7fb56c1082a0 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb56bfff640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 0x7fb56c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.408+0000 7fb570a43640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb56c1082a0 0x7fb56c19cf10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fd0 tx=0x7fb56400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.409+0000 7fb5697fa640 1 -- 172.21.15.137:0/1621202512 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56400ce20 con 0x7fb56c1082a0 2024-09-12T12:14:14.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.409+0000 7fb5697fa640 1 -- 172.21.15.137:0/1621202512 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5640137d0 con 0x7fb56c1082a0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.409+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56c19d8e0 con 0x7fb56c1082a0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.409+0000 7fb5697fa640 1 -- 172.21.15.137:0/1621202512 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5640055e0 con 0x7fb56c1082a0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.409+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb56c1008c0 con 0x7fb56c1082a0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.410+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56c103110 con 0x7fb56c1082a0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.411+0000 7fb5697fa640 1 -- 172.21.15.137:0/1621202512 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7fb564013980 con 0x7fb56c1082a0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.412+0000 7fb5697fa640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5540a2f10 0x7fb5540a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.412+0000 7fb56bfff640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5540a2f10 0x7fb5540a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.412+0000 7fb5697fa640 1 -- 172.21.15.137:0/1621202512 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fb5640d3c20 con 0x7fb56c1082a0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.413+0000 7fb56bfff640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5540a2f10 0x7fb5540a53d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb55c011550 tx=0x7fb55c005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:14.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.417+0000 7fb5697fa640 1 -- 172.21.15.137:0/1621202512 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5640a06f0 con 0x7fb56c1082a0 2024-09-12T12:14:14.753 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:14 smithi137 ceph-mon[30950]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T12:14:14.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.795+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb56c10a2b0 con 0x7fb56c1082a0 2024-09-12T12:14:14.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.796+0000 7fb5697fa640 1 -- 172.21.15.137:0/1621202512 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11936 (secure 0 0 0) 0x7fb56409fff0 con 0x7fb56c1082a0 2024-09-12T12:14:14.797 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:14:14.797 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"btime":"2024-09-12T12:13:12:456388+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":24,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:13:12.406179+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2972240277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2972240277},{"type":"v1","addr":"172.21.15.189:6835","nonce":2972240277}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:14:14.797 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-09-12T12:14:14.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.798+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5540a2f10 msgr2=0x7fb5540a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:14.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.798+0000 7fb5724a6640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5540a2f10 0x7fb5540a53d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb55c011550 tx=0x7fb55c005f90 comp rx=0 tx=0).stop 2024-09-12T12:14:14.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.798+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb56c1082a0 msgr2=0x7fb56c19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:14.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.798+0000 7fb5724a6640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb56c1082a0 0x7fb56c19cf10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb564009fd0 tx=0x7fb56400ef10 comp rx=0 tx=0).stop 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 shutdown_connections 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5540a2f10 0x7fb5540a53d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb56c1082a0 0x7fb56c19cf10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb56c1036e0 0x7fb56c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 --2- 172.21.15.137:0/1621202512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb56c102d30 0x7fb56c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 >> 172.21.15.137:0/1621202512 conn(0x7fb56c0fc7d0 msgr2=0x7fb56c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 shutdown_connections 2024-09-12T12:14:14.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:14.799+0000 7fb5724a6640 1 -- 172.21.15.137:0/1621202512 wait complete. 2024-09-12T12:14:15.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:14 smithi189 ceph-mon[35475]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T12:14:15.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:14 smithi171 ceph-mon[35098]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-09-12T12:14:15.698 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 3c9ff456 --release --await' 2024-09-12T12:14:15.699 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 3c9ff456 --release --await 2024-09-12T12:14:15.972 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:14:16.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:15 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1621202512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:14:16.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:15 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1621202512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:14:16.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:15 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1621202512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:14:16.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.513+0000 7fa5debcb640 1 -- 172.21.15.137:0/4199230109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8101470 msgr2=0x7fa5d8101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.513+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4199230109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8101470 0x7fa5d8101850 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00d910 tx=0x7fa5cc02c380 comp rx=0 tx=0).stop 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.513+0000 7fa5debcb640 1 -- 172.21.15.137:0/4199230109 shutdown_connections 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.513+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4199230109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5d8105de0 0x7fa5d810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.513+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4199230109 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5d8101e20 0x7fa5d81056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.513+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4199230109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8101470 0x7fa5d8101850 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.513+0000 7fa5debcb640 1 -- 172.21.15.137:0/4199230109 >> 172.21.15.137:0/4199230109 conn(0x7fa5d8077230 msgr2=0x7fa5d80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.514+0000 7fa5debcb640 1 -- 172.21.15.137:0/4199230109 shutdown_connections 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.514+0000 7fa5debcb640 1 -- 172.21.15.137:0/4199230109 wait complete. 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.514+0000 7fa5debcb640 1 Processor -- start 2024-09-12T12:14:16.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.514+0000 7fa5debcb640 1 -- start start 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.514+0000 7fa5debcb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5d8101470 0x7fa5d8196240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.514+0000 7fa5debcb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5d8101e20 0x7fa5d8196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5debcb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8105de0 0x7fa5d819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5debcb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810fd80 con 0x7fa5d8105de0 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5debcb640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810fc00 con 0x7fa5d8101e20 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5debcb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810ff00 con 0x7fa5d8101470 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8105de0 0x7fa5d819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8105de0 0x7fa5d819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41490/0 (socket says 172.21.15.137:41490) 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 -- 172.21.15.137:0/4087853222 learned_addr learned my addr 172.21.15.137:0/4087853222 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dc967640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5d8101470 0x7fa5d8196240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 -- 172.21.15.137:0/4087853222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5d8101470 msgr2=0x7fa5d8196240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5d8101470 0x7fa5d8196240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 -- 172.21.15.137:0/4087853222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5d8101e20 msgr2=0x7fa5d8196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5d8101e20 0x7fa5d8196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 -- 172.21.15.137:0/4087853222 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d819b1f0 con 0x7fa5d8105de0 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5dd168640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8105de0 0x7fa5d819ab10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009fd0 tx=0x7fa5d000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5d5ffb640 1 -- 172.21.15.137:0/4087853222 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0017070 con 0x7fa5d8105de0 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.515+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d819b4e0 con 0x7fa5d8105de0 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.516+0000 7fa5d5ffb640 1 -- 172.21.15.137:0/4087853222 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d0003e50 con 0x7fa5d8105de0 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.516+0000 7fa5d5ffb640 1 -- 172.21.15.137:0/4087853222 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0020060 con 0x7fa5d8105de0 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.516+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d81a2dc0 con 0x7fa5d8105de0 2024-09-12T12:14:16.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.519+0000 7fa5d5ffb640 1 -- 172.21.15.137:0/4087853222 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7fa5d00040b0 con 0x7fa5d8105de0 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.519+0000 7fa5e0fa4740 1 Processor -- start 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5e0fa4740 1 -- start start 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5d5ffb640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa5c80a2f10 0x7fa5c80a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573575e0 0x561a57355f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a57356880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561a572bb620 con 0x561a57356440 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a57262e50 con 0x561a573698e0 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561a572b75e0 con 0x561a573575e0 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a57356880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a57356880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41494/0 (socket says 172.21.15.137:41494) 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5d7fff640 1 -- 172.21.15.137:0/2498397071 learned_addr learned my addr 172.21.15.137:0/2498397071 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5d5ffb640 1 -- 172.21.15.137:0/4087853222 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fa5d00cf180 con 0x7fa5d8105de0 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.520+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa5c80a2f10 0x7fa5c80a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dc967640 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573575e0 0x561a57355f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dd168640 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dd168640 1 -- 172.21.15.137:0/2498397071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573575e0 msgr2=0x561a57355f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dd168640 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573575e0 0x561a57355f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dd168640 1 -- 172.21.15.137:0/2498397071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 msgr2=0x561a57356880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dd168640 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a57356880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dd168640 1 -- 172.21.15.137:0/2498397071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a57253d40 con 0x561a573698e0 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5dd168640 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736bcd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00cc710 tx=0x7fa5d009ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894fbb3b7dddea17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:16.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa5c80a2f10 0x7fa5c80a53d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4013590 tx=0x7fa5c4009760 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5c2ffd640 1 -- 172.21.15.137:0/2498397071 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d009b440 con 0x561a573698e0 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5c2ffd640 1 -- 172.21.15.137:0/2498397071 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d009f7f0 con 0x561a573698e0 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.521+0000 7fa5c2ffd640 1 -- 172.21.15.137:0/2498397071 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d00ccc80 con 0x561a573698e0 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.522+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2498397071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 msgr2=0x561a5736bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.522+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736bcd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00cc710 tx=0x7fa5d009ada0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.522+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2498397071 shutdown_connections 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.522+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736bcd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.522+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a57356880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.522+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2498397071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573575e0 0x561a57355f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.522+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2498397071 >> 172.21.15.137:0/2498397071 conn(0x561a573525f0 msgr2=0x561a573529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.523+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2498397071 shutdown_connections 2024-09-12T12:14:16.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.523+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2498397071 wait complete. 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.523+0000 7fa5e0fa4740 1 -- start start 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.523+0000 7fa5e0fa4740 1 Processor -- start 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.523+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57359c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.523+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5735a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5735a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561a5736c550 con 0x561a573575e0 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5735a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561a57243f10 con 0x561a573698e0 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561a5736c6d0 con 0x561a57356440 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5735a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41496/0 (socket says 172.21.15.137:41496) 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5e0fa4740 10 client.0 _ll_register_callbacks cb 0x561a57267db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:16.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 -- 172.21.15.137:0/3253104979 learned_addr learned my addr 172.21.15.137:0/3253104979 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:16.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5dc967640 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57359c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 -- 172.21.15.137:0/3253104979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 msgr2=0x561a57359c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57359c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 -- 172.21.15.137:0/3253104979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 msgr2=0x561a5735a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5735a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 -- 172.21.15.137:0/3253104979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a56fca580 con 0x561a573575e0 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5735a180 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40045a0 tx=0x7fa5c4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b67dad5d1018418 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.524+0000 7fa5b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5b7fff640 1 -- 172.21.15.137:0/3253104979 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4003dc0 con 0x561a573575e0 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5b7fff640 1 -- 172.21.15.137:0/3253104979 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fa5c40047a0 con 0x561a573575e0 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5d009f7f0 con 0x561a573575e0 2024-09-12T12:14:16.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5b7fff640 1 -- 172.21.15.137:0/3253104979 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa5c40049a0 con 0x561a573575e0 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 10 client.15477 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561a57094960 con 0x561a573575e0 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5b7fff640 1 -- 172.21.15.137:0/3253104979 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d009f7f0 con 0x561a573575e0 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5b7fff640 1 -- 172.21.15.137:0/3253104979 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa5c40105d0 con 0x561a573575e0 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 10 client.15477 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 10 client.15477 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 20 client.15477 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573795f0 0x561a5737ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 4 client.15477 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.525+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a572c8b90 con 0x561a573795f0 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.526+0000 7fa5dc967640 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573795f0 0x561a5737ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.526+0000 7fa5b7fff640 10 client.15477 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:16.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.526+0000 7fa5dc967640 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573795f0 0x561a5737ba30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50b4f539edf6f88 server_cookie=44d038b7713798ba in_seq=0 out_seq=0 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5b7fff640 1 -- 172.21.15.137:0/3253104979 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a572c8b90 con 0x561a573795f0 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5b7fff640 10 client.15477 handle_command_reply: tid=0 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 client.15477 shutdown 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 msgr2=0x561a5735a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5735a180 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40045a0 tx=0x7fa5c4009650 comp rx=0 tx=0).stop 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 shutdown_connections 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5735a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573795f0 0x561a5737ba30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5735a180 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/3253104979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57359c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 >> 172.21.15.137:0/3253104979 conn(0x561a573525f0 msgr2=0x561a57351750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.527+0000 7fa5b7fff640 0 client.15477 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:16.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.528+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 shutdown_connections 2024-09-12T12:14:16.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.528+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/3253104979 wait complete. 2024-09-12T12:14:16.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.528+0000 7fa5e0fa4740 20 client.15477 trim_cache size 0 max 16384 2024-09-12T12:14:16.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 Processor -- start 2024-09-12T12:14:16.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 -- start start 2024-09-12T12:14:16.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a5735b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573575e0 0x561a57374ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573698e0 0x561a57375430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81add30 con 0x561a57356440 2024-09-12T12:14:16.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81ab600 con 0x561a573575e0 2024-09-12T12:14:16.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81a9470 con 0x561a573698e0 2024-09-12T12:14:16.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5dc967640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a5735b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5dc967640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a5735b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41512/0 (socket says 172.21.15.137:41512) 2024-09-12T12:14:16.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.545+0000 7fa5dc967640 1 -- 172.21.15.137:0/1024348288 learned_addr learned my addr 172.21.15.137:0/1024348288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5dd168640 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573698e0 0x561a57375430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5dc967640 1 -- 172.21.15.137:0/1024348288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573698e0 msgr2=0x561a57375430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5dc967640 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573698e0 0x561a57375430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5dc967640 1 -- 172.21.15.137:0/1024348288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573575e0 msgr2=0x561a57374ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5dc967640 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573575e0 0x561a57374ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5dc967640 1 -- 172.21.15.137:0/1024348288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a57351610 con 0x561a57356440 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5dc967640 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a5735b4a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc005ab0 tx=0x7fa5cc004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6817aaae33301350 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5c27fc640 1 -- 172.21.15.137:0/1024348288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc04a020 con 0x561a57356440 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5c27fc640 1 -- 172.21.15.137:0/1024348288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5cc030040 con 0x561a57356440 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.546+0000 7fa5c27fc640 1 -- 172.21.15.137:0/1024348288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc03fcb0 con 0x561a57356440 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/1024348288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 msgr2=0x561a5735b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a5735b4a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc005ab0 tx=0x7fa5cc004700 comp rx=0 tx=0).stop 2024-09-12T12:14:16.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/1024348288 shutdown_connections 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a573698e0 0x561a57375430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573575e0 0x561a57374ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/1024348288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a57356440 0x561a5735b4a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/1024348288 >> 172.21.15.137:0/1024348288 conn(0x561a573525f0 msgr2=0x561a573747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/1024348288 shutdown_connections 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.547+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/1024348288 wait complete. 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.548+0000 7fa5e0fa4740 1 -- start start 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.548+0000 7fa5e0fa4740 1 Processor -- start 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.548+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57371ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5736e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5e0fa4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81a9ac0 con 0x561a573575e0 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81aaf30 con 0x561a573698e0 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5e0fa4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d81aa190 con 0x561a57356440 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5e0fa4740 10 client.0 _ll_register_callbacks cb 0x561a57267db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5736e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5736e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41522/0 (socket says 172.21.15.137:41522) 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5d7fff640 1 -- 172.21.15.137:0/2215948201 learned_addr learned my addr 172.21.15.137:0/2215948201 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5dc967640 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57371ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5dc967640 1 -- 172.21.15.137:0/2215948201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 msgr2=0x561a5736e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5dc967640 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5dc967640 1 -- 172.21.15.137:0/2215948201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 msgr2=0x561a5736e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5dc967640 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5736e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5dc967640 1 -- 172.21.15.137:0/2215948201 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5736ed90 con 0x561a57356440 2024-09-12T12:14:16.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5736e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5dc967640 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57371ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00d860 tx=0x7fa5cc03fd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec56740e8e67d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5b7fff640 1 -- 172.21.15.137:0/2215948201 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc03f630 con 0x561a57356440 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.549+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5cc030040 con 0x561a57356440 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.550+0000 7fa5b7fff640 1 -- 172.21.15.137:0/2215948201 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fa5cc004020 con 0x561a57356440 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.550+0000 7fa5b7fff640 1 -- 172.21.15.137:0/2215948201 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa5cc03f7d0 con 0x561a57356440 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.550+0000 7fa5e0fa4740 10 client.25051 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.550+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561a573e5130 con 0x561a57356440 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.550+0000 7fa5b7fff640 1 -- 172.21.15.137:0/2215948201 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5cc030040 con 0x561a57356440 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5b7fff640 1 -- 172.21.15.137:0/2215948201 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fa5cc058500 con 0x561a57356440 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5e0fa4740 10 client.25051 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:16.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5e0fa4740 10 client.25051 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5e0fa4740 20 client.25051 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573e57b0 0x561a573e7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5e0fa4740 4 client.25051 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3c9ff456", "release": true, "await": true}] 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3c9ff456", "release": true, "await": true}) -- 0x561a572c30a0 con 0x561a573e57b0 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573e57b0 0x561a573e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5d7fff640 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573e57b0 0x561a573e7ba0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89f43e6ddce30e server_cookie=b362065f039dca3 in_seq=0 out_seq=0 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.551+0000 7fa5b7fff640 10 client.25051 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5b7fff640 1 -- 172.21.15.137:0/2215948201 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561a572c30a0 con 0x561a573e57b0 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5b7fff640 10 client.25051 handle_command_reply: tid=0 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 client.25051 shutdown 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 msgr2=0x561a57371ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57371ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00d860 tx=0x7fa5cc03fd60 comp rx=0 tx=0).stop 2024-09-12T12:14:16.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 shutdown_connections 2024-09-12T12:14:16.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561a573698e0 0x561a5736e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561a573e57b0 0x561a573e7ba0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.560 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:14:16.560 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 24, 2024-09-12T12:14:16.560 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 4, 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "3c9ff456": { 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 4, 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:14:16.561 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.5, 2024-09-12T12:14:16.562 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:14:16.562 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:14:16.562 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:14:16.562 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:14:16.562 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:14:16.562 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:14:16.562 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5b7fff640 0 client.25051 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561a573575e0 0x561a5736e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 --2- 172.21.15.137:0/2215948201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a57356440 0x561a57371ed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 >> 172.21.15.137:0/2215948201 conn(0x561a573525f0 msgr2=0x561a57375000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 shutdown_connections 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 1 -- 172.21.15.137:0/2215948201 wait complete. 2024-09-12T12:14:16.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.553+0000 7fa5e0fa4740 20 client.25051 trim_cache size 0 max 16384 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa5c80a2f10 msgr2=0x7fa5c80a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa5c80a2f10 0x7fa5c80a53d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4013590 tx=0x7fa5c4009760 comp rx=0 tx=0).stop 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8105de0 msgr2=0x7fa5d819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8105de0 0x7fa5d819ab10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009fd0 tx=0x7fa5d000b260 comp rx=0 tx=0).stop 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 shutdown_connections 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa5c80a2f10 0x7fa5c80a53d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d8105de0 0x7fa5d819ab10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5d8101e20 0x7fa5d8196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 --2- 172.21.15.137:0/4087853222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5d8101470 0x7fa5d8196240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 >> 172.21.15.137:0/4087853222 conn(0x7fa5d8077230 msgr2=0x7fa5d80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 shutdown_connections 2024-09-12T12:14:16.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:14:16.554+0000 7fa5debcb640 1 -- 172.21.15.137:0/4087853222 wait complete. 2024-09-12T12:14:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:16 smithi137 ceph-mon[30950]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T12:14:17.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:16 smithi189 ceph-mon[35475]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T12:14:17.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:16 smithi171 ceph-mon[35098]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-12T12:14:17.412 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3c9ff456', total seconds elapsed: 28.3 2024-09-12T12:14:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:18 smithi137 ceph-mon[30950]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T12:14:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:14:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:14:19.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:14:19.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:18 smithi189 ceph-mon[35475]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T12:14:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:14:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:14:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:14:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:18 smithi171 ceph-mon[35098]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-12T12:14:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:14:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:14:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:14:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:14:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:20 smithi137 ceph-mon[30950]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:14:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:20 smithi189 ceph-mon[35475]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:14:21.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:20 smithi171 ceph-mon[35098]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-12T12:14:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 -- 172.21.15.137:0/1040273492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf20102c80 msgr2=0x7fbf20103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 --2- 172.21.15.137:0/1040273492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf20102c80 0x7fbf20103060 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009900 tx=0x7fbf1c02a480 comp rx=0 tx=0).stop 2024-09-12T12:14:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 -- 172.21.15.137:0/1040273492 shutdown_connections 2024-09-12T12:14:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 --2- 172.21.15.137:0/1040273492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf2010c0d0 0x7fbf2010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 --2- 172.21.15.137:0/1040273492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf201035a0 0x7fbf2010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 --2- 172.21.15.137:0/1040273492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf20102c80 0x7fbf20103060 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 -- 172.21.15.137:0/1040273492 >> 172.21.15.137:0/1040273492 conn(0x7fbf20077990 msgr2=0x7fbf200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.325+0000 7fbf2795b640 1 -- 172.21.15.137:0/1040273492 shutdown_connections 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 -- 172.21.15.137:0/1040273492 wait complete. 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 Processor -- start 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 -- start start 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf201035a0 0x7fbf20198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf2010c0d0 0x7fbf20198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf2019cde0 0x7fbf2019d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf20111eb0 con 0x7fbf201035a0 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf20111d30 con 0x7fbf2010c0d0 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.326+0000 7fbf2795b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf20112030 con 0x7fbf2019cde0 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf201035a0 0x7fbf20198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf201035a0 0x7fbf20198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37192/0 (socket says 172.21.15.137:37192) 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 -- 172.21.15.137:0/339899307 learned_addr learned my addr 172.21.15.137:0/339899307 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf2010c0d0 0x7fbf20198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 -- 172.21.15.137:0/339899307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf2019cde0 msgr2=0x7fbf2019d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf2019cde0 0x7fbf2019d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 -- 172.21.15.137:0/339899307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf2010c0d0 msgr2=0x7fbf20198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf2010c0d0 0x7fbf20198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 -- 172.21.15.137:0/339899307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2019d8d0 con 0x7fbf201035a0 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf256f7640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf201035a0 0x7fbf20198520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c005940 tx=0x7fbf1c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf167fc640 1 -- 172.21.15.137:0/339899307 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c03f030 con 0x7fbf201035a0 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2019db60 con 0x7fbf201035a0 2024-09-12T12:14:21.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf167fc640 1 -- 172.21.15.137:0/339899307 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1c004440 con 0x7fbf201035a0 2024-09-12T12:14:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.327+0000 7fbf167fc640 1 -- 172.21.15.137:0/339899307 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c035070 con 0x7fbf201035a0 2024-09-12T12:14:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.328+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf201a52d0 con 0x7fbf201035a0 2024-09-12T12:14:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.331+0000 7fbf167fc640 1 -- 172.21.15.137:0/339899307 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7fbf1c042030 con 0x7fbf201035a0 2024-09-12T12:14:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 Processor -- start 2024-09-12T12:14:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 -- start start 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc49d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc58b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc5b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561e6dbae2f0 con 0x561e6dc590c0 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561e6db56270 con 0x561e6dc4a2d0 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e6dbaa2b0 con 0x561e6dc499b0 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf167fc640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbf0c0a2f40 0x7fbf0c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf167fc640 1 -- 172.21.15.137:0/339899307 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fbf1c0ef900 con 0x7fbf201035a0 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf25ef8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc49d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf25ef8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc49d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:56222/0 (socket says 172.21.15.137:56222) 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf25ef8640 1 -- 172.21.15.137:0/1991763251 learned_addr learned my addr 172.21.15.137:0/1991763251 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.332+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbf0c0a2f40 0x7fbf0c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc5b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf256f7640 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc58b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf24ef6640 1 -- 172.21.15.137:0/1991763251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 msgr2=0x561e6dc49d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc49d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf24ef6640 1 -- 172.21.15.137:0/1991763251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 msgr2=0x561e6dc58b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc58b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf24ef6640 1 -- 172.21.15.137:0/1991763251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6db55340 con 0x561e6dc590c0 2024-09-12T12:14:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbf0c0a2f40 0x7fbf0c0a5400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf18005c40 tx=0x7fbf18009b50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc5b4b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009be0 tx=0x7fbf10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e23ea7ad79827f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.333+0000 7fbf0b7fe640 1 -- 172.21.15.137:0/1991763251 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1001e070 con 0x561e6dc590c0 2024-09-12T12:14:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf0b7fe640 1 -- 172.21.15.137:0/1991763251 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf10009d90 con 0x561e6dc590c0 2024-09-12T12:14:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf0b7fe640 1 -- 172.21.15.137:0/1991763251 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10019670 con 0x561e6dc590c0 2024-09-12T12:14:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1991763251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 msgr2=0x561e6dc5b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc5b4b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009be0 tx=0x7fbf10011810 comp rx=0 tx=0).stop 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1991763251 shutdown_connections 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc5b4b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc58b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1991763251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc49d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1991763251 >> 172.21.15.137:0/1991763251 conn(0x561e6dc46060 msgr2=0x561e6dc46440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1991763251 shutdown_connections 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.334+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1991763251 wait complete. 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf2a7ce740 1 -- start start 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf2a7ce740 1 Processor -- start 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc499b0 0x561e6dc56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc590c0 0x561e6dc56b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf24ef6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf24ef6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37220/0 (socket says 172.21.15.137:37220) 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561e6dc5bc90 con 0x561e6dc4a2d0 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.335+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561e6db37290 con 0x561e6dc499b0 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561e6dc5be10 con 0x561e6dc590c0 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf24ef6640 1 -- 172.21.15.137:0/3070058366 learned_addr learned my addr 172.21.15.137:0/3070058366 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf256f7640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc499b0 0x561e6dc56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf2a7ce740 10 client.0 _ll_register_callbacks cb 0x561e6db68140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf256f7640 1 -- 172.21.15.137:0/3070058366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc590c0 msgr2=0x561e6dc56b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc590c0 0x561e6dc56b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf256f7640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc590c0 0x561e6dc56b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf256f7640 1 -- 172.21.15.137:0/3070058366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 msgr2=0x561e6dc565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf256f7640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf256f7640 1 -- 172.21.15.137:0/3070058366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6d8c1210 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf256f7640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc499b0 0x561e6dc56080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0ecec0 tx=0x7fbf1c0bb6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=585096bbc46f2788 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf08ff9640 1 -- 172.21.15.137:0/3070058366 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0c0040 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf10009d90 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf08ff9640 1 -- 172.21.15.137:0/3070058366 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fbf1c003c40 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf08ff9640 1 -- 172.21.15.137:0/3070058366 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbf1c0d8020 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf2a7ce740 10 client.24992 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.336+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e6d98d620 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf08ff9640 1 -- 172.21.15.137:0/3070058366 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf10009d90 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf08ff9640 1 -- 172.21.15.137:0/3070058366 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbf1c031b50 con 0x561e6dc499b0 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf2a7ce740 10 client.24992 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:21.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf2a7ce740 10 client.24992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf2a7ce740 20 client.24992 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc690f0 0x561e6dc6b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf2a7ce740 4 client.24992 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e6dbbba20 con 0x561e6dc690f0 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.337+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc690f0 0x561e6dc6b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf08ff9640 10 client.24992 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc690f0 0x561e6dc6b560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25735676d8964b77 server_cookie=406483b30c73f364 in_seq=0 out_seq=0 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf08ff9640 1 -- 172.21.15.137:0/3070058366 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e6dbbba20 con 0x561e6dc690f0 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf08ff9640 10 client.24992 handle_command_reply: tid=0 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 client.24992 shutdown 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc499b0 msgr2=0x561e6dc56080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc499b0 0x561e6dc56080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0ecec0 tx=0x7fbf1c0bb6b0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 shutdown_connections 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc590c0 0x561e6dc56b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc690f0 0x561e6dc6b560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf08ff9640 0 client.24992 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/3070058366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc499b0 0x561e6dc56080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.338+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 >> 172.21.15.137:0/3070058366 conn(0x561e6dc46060 msgr2=0x561e6dc4bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.339+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 shutdown_connections 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.339+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/3070058366 wait complete. 2024-09-12T12:14:21.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.339+0000 7fbf2a7ce740 20 client.24992 trim_cache size 0 max 16384 2024-09-12T12:14:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 Processor -- start 2024-09-12T12:14:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 -- start start 2024-09-12T12:14:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201ae310 con 0x561e6dc590c0 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201abbe0 con 0x561e6dc4a2d0 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201a9a50 con 0x561e6dc499b0 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf25ef8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf256f7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc70e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf25ef8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37234/0 (socket says 172.21.15.137:37234) 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf25ef8640 1 -- 172.21.15.137:0/1571363999 learned_addr learned my addr 172.21.15.137:0/1571363999 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.353+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf25ef8640 1 -- 172.21.15.137:0/1571363999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 msgr2=0x561e6dc70e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc70e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf25ef8640 1 -- 172.21.15.137:0/1571363999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 msgr2=0x561e6dc713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf25ef8640 1 -- 172.21.15.137:0/1571363999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6dc46cf0 con 0x561e6dc590c0 2024-09-12T12:14:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf256f7640 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc70e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc532c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009990 tx=0x7fbf1801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7d9482c94f6a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf0affd640 1 -- 172.21.15.137:0/1571363999 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf18023070 con 0x561e6dc590c0 2024-09-12T12:14:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf0affd640 1 -- 172.21.15.137:0/1571363999 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf18028940 con 0x561e6dc590c0 2024-09-12T12:14:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.354+0000 7fbf0affd640 1 -- 172.21.15.137:0/1571363999 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf180040c0 con 0x561e6dc590c0 2024-09-12T12:14:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1571363999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 msgr2=0x561e6dc532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc532c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009990 tx=0x7fbf1801b000 comp rx=0 tx=0).stop 2024-09-12T12:14:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1571363999 shutdown_connections 2024-09-12T12:14:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc590c0 0x561e6dc532c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc4a2d0 0x561e6dc713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/1571363999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc70e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1571363999 >> 172.21.15.137:0/1571363999 conn(0x561e6dc46060 msgr2=0x561e6dc56860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1571363999 shutdown_connections 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.355+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/1571363999 wait complete. 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 -- start start 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 Processor -- start 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc61960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc5cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc590c0 0x561e6dc5d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201aa0a0 con 0x561e6dc4a2d0 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201ab510 con 0x561e6dc590c0 2024-09-12T12:14:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf201aa770 con 0x561e6dc499b0 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf2a7ce740 10 client.0 _ll_register_callbacks cb 0x7fbf1c0b7850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.356+0000 7fbf24ef6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc5cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf256f7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc61960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc5cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37246/0 (socket says 172.21.15.137:37246) 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 -- 172.21.15.137:0/552820948 learned_addr learned my addr 172.21.15.137:0/552820948 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf256f7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc61960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:56246/0 (socket says 172.21.15.137:56246) 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc590c0 0x561e6dc5d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 -- 172.21.15.137:0/552820948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 msgr2=0x561e6dc61960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc61960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 -- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc590c0 msgr2=0x561e6dc5d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc590c0 0x561e6dc5d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 -- 172.21.15.137:0/552820948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e6dc5dac0 con 0x561e6dc4a2d0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf25ef8640 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc590c0 0x561e6dc5d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf24ef6640 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc5cc80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf10014040 tx=0x7fbf10009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beeb6a69a79541d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf08ff9640 1 -- 172.21.15.137:0/552820948 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1001e040 con 0x561e6dc4a2d0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf08ff9640 1 -- 172.21.15.137:0/552820948 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7fbf10009d90 con 0x561e6dc4a2d0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf18028940 con 0x561e6dc4a2d0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf08ff9640 1 -- 172.21.15.137:0/552820948 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbf10005350 con 0x561e6dc4a2d0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.357+0000 7fbf2a7ce740 10 client.15501 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x561e6dc4c880 con 0x561e6dc4a2d0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf08ff9640 1 -- 172.21.15.137:0/552820948 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf100054f0 con 0x561e6dc4a2d0 2024-09-12T12:14:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf08ff9640 1 -- 172.21.15.137:0/552820948 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbf1002d020 con 0x561e6dc4a2d0 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf2a7ce740 10 client.15501 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf2a7ce740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf2a7ce740 20 client.15501 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc4ce70 0x561e6dc4f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf2a7ce740 4 client.15501 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fbf18028940 con 0x561e6dc4ce70 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.358+0000 7fbf256f7640 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc4ce70 0x561e6dc4f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.359+0000 7fbf08ff9640 10 client.15501 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.359+0000 7fbf256f7640 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc4ce70 0x561e6dc4f290 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8cc1a5d1c8ae6b server_cookie=54bc09dab76747d2 in_seq=0 out_seq=0 2024-09-12T12:14:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.362+0000 7fbf08ff9640 1 -- 172.21.15.137:0/552820948 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fbf18028940 con 0x561e6dc4ce70 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf08ff9640 10 client.15501 handle_command_reply: tid=0 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 client.15501 shutdown 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 msgr2=0x561e6dc5cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc5cc80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf10014040 tx=0x7fbf10009820 comp rx=0 tx=0).stop 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 shutdown_connections 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561e6dc590c0 0x561e6dc5d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561e6dc4ce70 0x561e6dc4f290 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e6dc4a2d0 0x561e6dc5cc80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 --2- 172.21.15.137:0/552820948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561e6dc499b0 0x561e6dc61960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf08ff9640 0 client.15501 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 >> 172.21.15.137:0/552820948 conn(0x561e6dc46060 msgr2=0x561e6dc449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 shutdown_connections 2024-09-12T12:14:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 1 -- 172.21.15.137:0/552820948 wait complete. 2024-09-12T12:14:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.363+0000 7fbf2a7ce740 20 client.15501 trim_cache size 0 max 16384 2024-09-12T12:14:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:14:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:14:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:14:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbf0c0a2f40 msgr2=0x7fbf0c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbf0c0a2f40 0x7fbf0c0a5400 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf18005c40 tx=0x7fbf18009b50 comp rx=0 tx=0).stop 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf201035a0 msgr2=0x7fbf20198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf201035a0 0x7fbf20198520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c005940 tx=0x7fbf1c004040 comp rx=0 tx=0).stop 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 shutdown_connections 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbf0c0a2f40 0x7fbf0c0a5400 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf2019cde0 0x7fbf2019d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf2010c0d0 0x7fbf20198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 --2- 172.21.15.137:0/339899307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf201035a0 0x7fbf20198520 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 >> 172.21.15.137:0/339899307 conn(0x7fbf20077990 msgr2=0x7fbf200fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 shutdown_connections 2024-09-12T12:14:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:21.364+0000 7fbf2795b640 1 -- 172.21.15.137:0/339899307 wait complete. 2024-09-12T12:14:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:22 smithi137 ceph-mon[30950]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T12:14:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:22 smithi189 ceph-mon[35475]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T12:14:23.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:22 smithi171 ceph-mon[35098]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T12:14:24.539 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-09-12T12:14:24.540 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-09-12T12:14:24.541 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-09-12T12:14:24.541 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-09-12T12:14:24.541 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-09-12T12:14:24.541 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-09-12T12:14:24.541 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T12:14:24.541 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-09-12T12:14:24.541 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T12:14:24.545 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T12:14:24.607 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T12:14:24.609 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T12:14:24.609 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T12:14:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:24 smithi137 ceph-mon[30950]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T12:14:25.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:24 smithi189 ceph-mon[35475]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T12:14:25.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:24 smithi171 ceph-mon[35098]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-12T12:14:25.309 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T12:14:25.393 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T12:14:25.434 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T12:14:25.436 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T12:14:25.436 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T12:14:25.475 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T12:14:25.479 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:14:25.479 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T12:14:25.536 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-12T12:14:25.538 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-12T12:14:25.538 DEBUG:teuthology.orchestra.run.smithi137:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-12T12:14:25.599 INFO:tasks.workunit.client.0.smithi137.stderr:+ dbench 1 2024-09-12T12:14:25.612 INFO:tasks.workunit.client.0.smithi137.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-12T12:14:25.613 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-09-12T12:14:25.613 INFO:tasks.workunit.client.0.smithi137.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-12T12:14:25.615 INFO:tasks.workunit.client.0.smithi137.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-12T12:14:25.715 INFO:tasks.workunit.client.0.smithi137.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-12T12:14:25.715 INFO:tasks.workunit.client.0.smithi137.stdout:releasing clients 2024-09-12T12:14:26.715 INFO:tasks.workunit.client.0.smithi137.stdout: 1 3633 38.63 MB/sec warmup 1 sec latency 7.098 ms 2024-09-12T12:14:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:26 smithi137 ceph-mon[30950]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T12:14:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:26 smithi189 ceph-mon[35475]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T12:14:27.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:26 smithi171 ceph-mon[35098]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-12T12:14:27.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 7066 28.74 MB/sec warmup 2 sec latency 15.722 ms 2024-09-12T12:14:28.715 INFO:tasks.workunit.client.0.smithi137.stdout: 1 10308 24.17 MB/sec warmup 3 sec latency 11.064 ms 2024-09-12T12:14:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:28 smithi137 ceph-mon[30950]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T12:14:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:28 smithi189 ceph-mon[35475]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T12:14:29.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:28 smithi171 ceph-mon[35098]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-12T12:14:29.715 INFO:tasks.workunit.client.0.smithi137.stdout: 1 13430 22.33 MB/sec warmup 4 sec latency 14.257 ms 2024-09-12T12:14:30.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 16797 21.10 MB/sec warmup 5 sec latency 10.182 ms 2024-09-12T12:14:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:30 smithi137 ceph-mon[30950]: pgmap v41: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 613 KiB/s wr, 76 op/s 2024-09-12T12:14:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:30 smithi189 ceph-mon[35475]: pgmap v41: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 613 KiB/s wr, 76 op/s 2024-09-12T12:14:31.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:30 smithi171 ceph-mon[35098]: pgmap v41: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 613 KiB/s wr, 76 op/s 2024-09-12T12:14:31.715 INFO:tasks.workunit.client.0.smithi137.stdout: 1 20281 20.73 MB/sec warmup 6 sec latency 17.651 ms 2024-09-12T12:14:32.715 INFO:tasks.workunit.client.0.smithi137.stdout: 1 23525 20.22 MB/sec warmup 7 sec latency 14.733 ms 2024-09-12T12:14:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:32 smithi137 ceph-mon[30950]: pgmap v42: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 613 KiB/s wr, 73 op/s 2024-09-12T12:14:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:32 smithi189 ceph-mon[35475]: pgmap v42: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 613 KiB/s wr, 73 op/s 2024-09-12T12:14:33.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:32 smithi171 ceph-mon[35098]: pgmap v42: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 613 KiB/s wr, 73 op/s 2024-09-12T12:14:33.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 26484 20.01 MB/sec warmup 8 sec latency 10.036 ms 2024-09-12T12:14:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 -- 172.21.15.137:0/462255489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 msgr2=0x7f5c9010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 --2- 172.21.15.137:0/462255489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c9010d8a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009cd0 tx=0x7f5c8802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 -- 172.21.15.137:0/462255489 shutdown_connections 2024-09-12T12:14:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 --2- 172.21.15.137:0/462255489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c9010d8a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 --2- 172.21.15.137:0/462255489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 0x7f5c90100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 --2- 172.21.15.137:0/462255489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c901069e0 0x7f5c90106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 -- 172.21.15.137:0/462255489 >> 172.21.15.137:0/462255489 conn(0x7f5c900fc6f0 msgr2=0x7f5c900feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 -- 172.21.15.137:0/462255489 shutdown_connections 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.460+0000 7f5c9779f640 1 -- 172.21.15.137:0/462255489 wait complete. 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 Processor -- start 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 -- start start 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 0x7f5c901991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c90199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c901069e0 0x7f5c90199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90110e20 con 0x7f5c90101370 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90110ca0 con 0x7f5c901069e0 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9779f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90110fa0 con 0x7f5c901009d0 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9553b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 0x7f5c901991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c94d3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c90199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9553b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 0x7f5c901991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33136/0 (socket says 172.21.15.137:33136) 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c9553b640 1 -- 172.21.15.137:0/4180425496 learned_addr learned my addr 172.21.15.137:0/4180425496 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c90199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53960/0 (socket says 172.21.15.137:53960) 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.461+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c901069e0 0x7f5c90199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c94d3a640 1 -- 172.21.15.137:0/4180425496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 msgr2=0x7f5c901991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 0x7f5c901991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c94d3a640 1 -- 172.21.15.137:0/4180425496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c901069e0 msgr2=0x7f5c90199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c901069e0 0x7f5c90199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c94d3a640 1 -- 172.21.15.137:0/4180425496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c901a45f0 con 0x7f5c90101370 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c9553b640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 0x7f5c901991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c90199730 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e9f0 tx=0x7f5c8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c867fc640 1 -- 172.21.15.137:0/4180425496 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000ccc0 con 0x7f5c90101370 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c901a48e0 con 0x7f5c90101370 2024-09-12T12:14:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c867fc640 1 -- 172.21.15.137:0/4180425496 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c80005020 con 0x7f5c90101370 2024-09-12T12:14:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c867fc640 1 -- 172.21.15.137:0/4180425496 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800049e0 con 0x7f5c90101370 2024-09-12T12:14:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.462+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c901a4e30 con 0x7f5c90101370 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.465+0000 7f5c9a61b740 1 Processor -- start 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.465+0000 7f5c9a61b740 1 -- start start 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.465+0000 7f5c867fc640 1 -- 172.21.15.137:0/4180425496 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7f5c80005560 con 0x7f5c90101370 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564074503af0 0x564074503ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x56407450e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56407450ecb0 0x5640745110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5640744682f0 con 0x564074503af0 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564074410270 con 0x56407450ecb0 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640744642b0 con 0x564074504410 2024-09-12T12:14:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c95d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564074503af0 0x564074503ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9553b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x56407450e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c95d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564074503af0 0x564074503ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53972/0 (socket says 172.21.15.137:53972) 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c95d3c640 1 -- 172.21.15.137:0/4065693061 learned_addr learned my addr 172.21.15.137:0/4065693061 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56407450ecb0 0x5640745110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9553b640 1 -- 172.21.15.137:0/4065693061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56407450ecb0 msgr2=0x5640745110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9553b640 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56407450ecb0 0x5640745110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9553b640 1 -- 172.21.15.137:0/4065693061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564074503af0 msgr2=0x564074503ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9553b640 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564074503af0 0x564074503ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9553b640 1 -- 172.21.15.137:0/4065693061 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56407440f340 con 0x564074504410 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564074503af0 0x564074503ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c9553b640 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x56407450e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c011440 tx=0x7f5c8c011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11b670db7aa7542 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c867fc640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5c7c0a2ec0 0x7f5c7c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5c7c0a2ec0 0x7f5c7c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c7b7fe640 1 -- 172.21.15.137:0/4065693061 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004030 con 0x564074504410 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c867fc640 1 -- 172.21.15.137:0/4180425496 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f5c800cf610 con 0x7f5c90101370 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.466+0000 7f5c7b7fe640 1 -- 172.21.15.137:0/4065693061 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c0041d0 con 0x564074504410 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c7b7fe640 1 -- 172.21.15.137:0/4065693061 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00f930 con 0x564074504410 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 -- 172.21.15.137:0/4065693061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 msgr2=0x56407450e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x56407450e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c011440 tx=0x7f5c8c011900 comp rx=0 tx=0).stop 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5c7c0a2ec0 0x7f5c7c0a5380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c88004ab0 tx=0x7f5c88004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 -- 172.21.15.137:0/4065693061 shutdown_connections 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56407450ecb0 0x5640745110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x56407450e770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/4065693061 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564074503af0 0x564074503ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 -- 172.21.15.137:0/4065693061 >> 172.21.15.137:0/4065693061 conn(0x564074500060 msgr2=0x564074500440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 -- 172.21.15.137:0/4065693061 shutdown_connections 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.467+0000 7f5c9a61b740 1 -- 172.21.15.137:0/4065693061 wait complete. 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 -- start start 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 Processor -- start 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x56407450bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x56407450c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564074511880 con 0x56407450ecb0 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5640743f1290 con 0x564074504410 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564074511a00 con 0x564074503af0 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9a61b740 10 client.0 _ll_register_callbacks cb 0x564074422140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c95d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x56407450c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9553b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x56407450bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9553b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x56407450bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33160/0 (socket says 172.21.15.137:33160) 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9553b640 1 -- 172.21.15.137:0/2128923679 learned_addr learned my addr 172.21.15.137:0/2128923679 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9553b640 1 -- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 msgr2=0x56407450c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9553b640 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9553b640 1 -- 172.21.15.137:0/2128923679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 msgr2=0x56407450c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.468+0000 7f5c9553b640 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x56407450c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9553b640 1 -- 172.21.15.137:0/2128923679 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56407417b210 con 0x564074503af0 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x56407450c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9553b640 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x56407450bbe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c01b040 tx=0x7f5c8c003dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5de736f476adc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c78ff9640 1 -- 172.21.15.137:0/2128923679 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c027020 con 0x564074503af0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c8c0041d0 con 0x564074503af0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c78ff9640 1 -- 172.21.15.137:0/2128923679 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f5c8c016040 con 0x564074503af0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c78ff9640 1 -- 172.21.15.137:0/2128923679 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c8c032050 con 0x564074503af0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 10 client.25075 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564074247620 con 0x564074503af0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c78ff9640 1 -- 172.21.15.137:0/2128923679 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c0041d0 con 0x564074503af0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c78ff9640 1 -- 172.21.15.137:0/2128923679 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5c8c00faf0 con 0x564074503af0 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 10 client.25075 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 10 client.25075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 20 client.25075 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56407451ebf0 0x564074520fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 4 client.25075 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:14:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.469+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564074475a20 con 0x56407451ebf0 2024-09-12T12:14:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.470+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56407451ebf0 0x564074520fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.470+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56407451ebf0 0x564074520fe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508a8381a956a125 server_cookie=f6fbcbe6be23f541 in_seq=0 out_seq=0 2024-09-12T12:14:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.470+0000 7f5c78ff9640 10 client.25075 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c78ff9640 1 -- 172.21.15.137:0/2128923679 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564074475a20 con 0x56407451ebf0 2024-09-12T12:14:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c78ff9640 10 client.25075 handle_command_reply: tid=0 2024-09-12T12:14:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 client.25075 shutdown 2024-09-12T12:14:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 msgr2=0x56407450bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x56407450bbe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c01b040 tx=0x7f5c8c003dd0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 shutdown_connections 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56407451ebf0 0x564074520fe0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x56407450c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c78ff9640 0 client.25075 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2128923679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x56407450bbe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.472+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 >> 172.21.15.137:0/2128923679 conn(0x564074500060 msgr2=0x564074505dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.473+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 shutdown_connections 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.473+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2128923679 wait complete. 2024-09-12T12:14:34.480 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.473+0000 7f5c9a61b740 20 client.25075 trim_cache size 0 max 16384 2024-09-12T12:14:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 Processor -- start 2024-09-12T12:14:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 -- start start 2024-09-12T12:14:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x564074506560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074526700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901aff80 con 0x56407450ecb0 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901ad850 con 0x564074504410 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901ab6a0 con 0x564074503af0 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074526700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074526700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53984/0 (socket says 172.21.15.137:53984) 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 -- 172.21.15.137:0/2335972484 learned_addr learned my addr 172.21.15.137:0/2335972484 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c9553b640 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x564074506560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c94d3a640 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 -- 172.21.15.137:0/2335972484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 msgr2=0x564074506560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x564074506560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 -- 172.21.15.137:0/2335972484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 msgr2=0x56407450c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 -- 172.21.15.137:0/2335972484 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56407450e730 con 0x56407450ecb0 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.487+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074526700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5c88035000 tx=0x7f5c88044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d2fe9a454713b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c7affd640 1 -- 172.21.15.137:0/2335972484 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88004040 con 0x56407450ecb0 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c7affd640 1 -- 172.21.15.137:0/2335972484 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c88056070 con 0x56407450ecb0 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c7affd640 1 -- 172.21.15.137:0/2335972484 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88030dd0 con 0x56407450ecb0 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2335972484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 msgr2=0x564074526700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074526700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5c88035000 tx=0x7f5c88044000 comp rx=0 tx=0).stop 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2335972484 shutdown_connections 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074526700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074504410 0x56407450c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/2335972484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074503af0 0x564074506560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2335972484 >> 172.21.15.137:0/2335972484 conn(0x564074500060 msgr2=0x56407450dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2335972484 shutdown_connections 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.488+0000 7f5c9a61b740 1 -- 172.21.15.137:0/2335972484 wait complete. 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 -- start start 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 Processor -- start 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074503af0 0x564074517430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x5640745089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074508f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901abd10 con 0x56407450ecb0 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901ad180 con 0x564074503af0 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c901ac3e0 con 0x564074504410 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9a61b740 10 client.0 _ll_register_callbacks cb 0x7f5c80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c95d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074508f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c9553b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074503af0 0x564074517430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c95d3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074508f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53996/0 (socket says 172.21.15.137:53996) 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c95d3c640 1 -- 172.21.15.137:0/214519792 learned_addr learned my addr 172.21.15.137:0/214519792 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c94d3a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x5640745089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c95d3c640 1 -- 172.21.15.137:0/214519792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 msgr2=0x5640745089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x5640745089b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.489+0000 7f5c95d3c640 1 -- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074503af0 msgr2=0x564074517430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074503af0 0x564074517430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c95d3c640 1 -- 172.21.15.137:0/214519792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640745097a0 con 0x56407450ecb0 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9553b640 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074503af0 0x564074517430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c95d3c640 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074508f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c88042040 tx=0x7f5c88005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6393e749781f336 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c78ff9640 1 -- 172.21.15.137:0/214519792 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88061070 con 0x56407450ecb0 2024-09-12T12:14:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c78ff9640 1 -- 172.21.15.137:0/214519792 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8440+0+0 (secure 0 0 0) 0x7f5c88003fb0 con 0x56407450ecb0 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c88056070 con 0x56407450ecb0 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c78ff9640 1 -- 172.21.15.137:0/214519792 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5c880041b0 con 0x56407450ecb0 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 10 client.15519 fetch_fsmap learned FSMap version 24 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564074589760 con 0x56407450ecb0 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c78ff9640 1 -- 172.21.15.137:0/214519792 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c88056070 con 0x56407450ecb0 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c78ff9640 1 -- 172.21.15.137:0/214519792 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f5c88030070 con 0x56407450ecb0 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 10 client.15519 fetch_fsmap finished waiting for FSMap version 24 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 20 client.15519 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564074589e50 0x56407458c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 4 client.15519 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.490+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56407446fd70 con 0x564074589e50 2024-09-12T12:14:34.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.491+0000 7f5c9553b640 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564074589e50 0x56407458c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.491+0000 7f5c78ff9640 10 client.15519 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:34.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.491+0000 7f5c9553b640 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564074589e50 0x56407458c270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7bdd0633eb56f2 server_cookie=f92183e62a32e5d0 in_seq=0 out_seq=0 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.618+0000 7f5c78ff9640 1 -- 172.21.15.137:0/214519792 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56407446fd70 con 0x564074589e50 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.618+0000 7f5c78ff9640 10 client.15519 handle_command_reply: tid=0 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.618+0000 7f5c9a61b740 1 client.15519 shutdown 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 msgr2=0x564074508f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074508f70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5c88042040 tx=0x7f5c88005fa0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 shutdown_connections 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564074589e50 0x56407458c270 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56407450ecb0 0x564074508f70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564074504410 0x5640745089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 --2- 172.21.15.137:0/214519792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564074503af0 0x564074517430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c78ff9640 0 client.15519 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 >> 172.21.15.137:0/214519792 conn(0x564074500060 msgr2=0x5640744fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 shutdown_connections 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 1 -- 172.21.15.137:0/214519792 wait complete. 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.619+0000 7f5c9a61b740 20 client.15519 trim_cache size 0 max 16384 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:14:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:14:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5c7c0a2ec0 msgr2=0x7f5c7c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5c7c0a2ec0 0x7f5c7c0a5380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c88004ab0 tx=0x7f5c88004a40 comp rx=0 tx=0).stop 2024-09-12T12:14:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 msgr2=0x7f5c90199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c90199730 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e9f0 tx=0x7f5c8000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 shutdown_connections 2024-09-12T12:14:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5c7c0a2ec0 0x7f5c7c0a5380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c901069e0 0x7f5c90199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5c90101370 0x7f5c90199730 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 --2- 172.21.15.137:0/4180425496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c901009d0 0x7f5c901991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 >> 172.21.15.137:0/4180425496 conn(0x7f5c900fc6f0 msgr2=0x7f5c900fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 shutdown_connections 2024-09-12T12:14:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:34.620+0000 7f5c9779f640 1 -- 172.21.15.137:0/4180425496 wait complete. 2024-09-12T12:14:34.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 29475 19.55 MB/sec warmup 9 sec latency 70.406 ms 2024-09-12T12:14:35.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:34 smithi137 ceph-mon[30950]: pgmap v43: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 613 KiB/s wr, 73 op/s 2024-09-12T12:14:35.083 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:34 smithi189 ceph-mon[35475]: pgmap v43: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 613 KiB/s wr, 73 op/s 2024-09-12T12:14:35.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:34 smithi171 ceph-mon[35098]: pgmap v43: 97 pgs: 97 active+clean; 7.7 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 613 KiB/s wr, 73 op/s 2024-09-12T12:14:35.341 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:14:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:14:35.341 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:14:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:14:35.341 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:14:35 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/3400197952 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640310fb860 con 0x5640310d9400 2024-09-12T12:14:35.341 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:14:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:14:35.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 32723 19.35 MB/sec warmup 10 sec latency 34.348 ms 2024-09-12T12:14:36.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:35 smithi137 ceph-mon[30950]: daemon mds.f restarted 2024-09-12T12:14:36.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:35 smithi189 ceph-mon[35475]: daemon mds.f restarted 2024-09-12T12:14:36.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:35 smithi171 ceph-mon[35098]: daemon mds.f restarted 2024-09-12T12:14:36.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 36169 19.30 MB/sec warmup 11 sec latency 10.203 ms 2024-09-12T12:14:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:36 smithi137 ceph-mon[30950]: pgmap v44: 97 pgs: 97 active+clean; 15 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-09-12T12:14:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:36 smithi137 ceph-mon[30950]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T12:14:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:36 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/3680297976,v1:172.21.15.189:6835/3680297976] up:boot 2024-09-12T12:14:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:36 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:14:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:36 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:14:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:36 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:14:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:36 smithi189 ceph-mon[35475]: pgmap v44: 97 pgs: 97 active+clean; 15 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-09-12T12:14:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:36 smithi189 ceph-mon[35475]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T12:14:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:36 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/3680297976,v1:172.21.15.189:6835/3680297976] up:boot 2024-09-12T12:14:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:36 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:14:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:36 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:14:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:36 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:14:37.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:36 smithi171 ceph-mon[35098]: pgmap v44: 97 pgs: 97 active+clean; 15 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-09-12T12:14:37.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:36 smithi171 ceph-mon[35098]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T12:14:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:36 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/3680297976,v1:172.21.15.189:6835/3680297976] up:boot 2024-09-12T12:14:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:36 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:14:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:36 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:14:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:36 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:14:37.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 39470 19.28 MB/sec warmup 12 sec latency 14.916 ms 2024-09-12T12:14:38.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 42732 19.16 MB/sec warmup 13 sec latency 15.401 ms 2024-09-12T12:14:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:38 smithi137 ceph-mon[30950]: pgmap v46: 97 pgs: 97 active+clean; 15 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-12T12:14:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:38 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:14:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:38 smithi189 ceph-mon[35475]: pgmap v46: 97 pgs: 97 active+clean; 15 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-12T12:14:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:38 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:14:39.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:38 smithi171 ceph-mon[35098]: pgmap v46: 97 pgs: 97 active+clean; 15 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-12T12:14:39.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:38 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:14:39.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 45570 18.79 MB/sec warmup 14 sec latency 15.227 ms 2024-09-12T12:14:40.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 48576 18.45 MB/sec warmup 15 sec latency 13.304 ms 2024-09-12T12:14:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:40 smithi137 ceph-mon[30950]: pgmap v47: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:40 smithi189 ceph-mon[35475]: pgmap v47: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:41.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:40 smithi171 ceph-mon[35098]: pgmap v47: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:41.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 51385 18.37 MB/sec warmup 16 sec latency 15.216 ms 2024-09-12T12:14:42.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 53981 18.13 MB/sec warmup 17 sec latency 14.914 ms 2024-09-12T12:14:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:42 smithi137 ceph-mon[30950]: pgmap v48: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:42 smithi189 ceph-mon[35475]: pgmap v48: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:43.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:42 smithi171 ceph-mon[35098]: pgmap v48: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:43.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 57143 18.16 MB/sec warmup 18 sec latency 10.226 ms 2024-09-12T12:14:44.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 59800 17.87 MB/sec warmup 19 sec latency 10.322 ms 2024-09-12T12:14:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:44 smithi137 ceph-mon[30950]: pgmap v49: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:45.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:44 smithi189 ceph-mon[35475]: pgmap v49: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:44 smithi171 ceph-mon[35098]: pgmap v49: 97 pgs: 97 active+clean; 14 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 403 op/s 2024-09-12T12:14:45.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 61859 17.56 MB/sec warmup 20 sec latency 14.930 ms 2024-09-12T12:14:46.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 64475 17.39 MB/sec warmup 21 sec latency 18.410 ms 2024-09-12T12:14:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:46 smithi189 ceph-mon[35475]: pgmap v50: 97 pgs: 97 active+clean; 21 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.8 MiB/s wr, 422 op/s 2024-09-12T12:14:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:46 smithi189 ceph-mon[35475]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-12T12:14:47.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:46 smithi171 ceph-mon[35098]: pgmap v50: 97 pgs: 97 active+clean; 21 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.8 MiB/s wr, 422 op/s 2024-09-12T12:14:47.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:46 smithi171 ceph-mon[35098]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-12T12:14:47.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:46 smithi137 ceph-mon[30950]: pgmap v50: 97 pgs: 97 active+clean; 21 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 1.8 MiB/s wr, 422 op/s 2024-09-12T12:14:47.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:46 smithi137 ceph-mon[30950]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-09-12T12:14:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2317770684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf981069e0 msgr2=0x7fcf98106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2317770684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf981069e0 0x7fcf98106dc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009930 tx=0x7fcf9402a420 comp rx=0 tx=0).stop 2024-09-12T12:14:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2317770684 shutdown_connections 2024-09-12T12:14:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2317770684 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf98101370 0x7fcf9810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2317770684 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf981009d0 0x7fcf98100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2317770684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf981069e0 0x7fcf98106dc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2317770684 >> 172.21.15.137:0/2317770684 conn(0x7fcf980fc6f0 msgr2=0x7fcf980feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2317770684 shutdown_connections 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.710+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2317770684 wait complete. 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 Processor -- start 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 -- start start 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf981009d0 0x7fcf9810e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf98101370 0x7fcf9810e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf981069e0 0x7fcf9810edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98111f60 con 0x7fcf98101370 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98111de0 con 0x7fcf981069e0 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9fcf8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981120e0 con 0x7fcf981009d0 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9d293640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf98101370 0x7fcf9810e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9da94640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf981009d0 0x7fcf9810e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9da94640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf981009d0 0x7fcf9810e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:36028/0 (socket says 172.21.15.137:36028) 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9da94640 1 -- 172.21.15.137:0/2558096649 learned_addr learned my addr 172.21.15.137:0/2558096649 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9e295640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf981069e0 0x7fcf9810edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9d293640 1 -- 172.21.15.137:0/2558096649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf981009d0 msgr2=0x7fcf9810e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9d293640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf981009d0 0x7fcf9810e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9d293640 1 -- 172.21.15.137:0/2558096649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf981069e0 msgr2=0x7fcf9810edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9d293640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf981069e0 0x7fcf9810edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.711+0000 7fcf9d293640 1 -- 172.21.15.137:0/2558096649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf981a6850 con 0x7fcf98101370 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.712+0000 7fcf9d293640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf98101370 0x7fcf9810e870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00e9f0 tx=0x7fcf8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.712+0000 7fcf8affd640 1 -- 172.21.15.137:0/2558096649 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00ccc0 con 0x7fcf98101370 2024-09-12T12:14:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.712+0000 7fcf8affd640 1 -- 172.21.15.137:0/2558096649 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8c010910 con 0x7fcf98101370 2024-09-12T12:14:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.712+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981a6b40 con 0x7fcf98101370 2024-09-12T12:14:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.712+0000 7fcf8affd640 1 -- 172.21.15.137:0/2558096649 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c018960 con 0x7fcf98101370 2024-09-12T12:14:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.712+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf981a7090 con 0x7fcf98101370 2024-09-12T12:14:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.714+0000 7fcf8affd640 1 -- 172.21.15.137:0/2558096649 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcf8c010ac0 con 0x7fcf98101370 2024-09-12T12:14:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 Processor -- start 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 -- start start 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72e7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c72f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5615c724c2f0 con 0x5615c72f2cb0 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5615c71f4270 con 0x5615c72e8410 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5615c72482b0 con 0x5615c72e7af0 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.715+0000 7fcf8affd640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcf840a2da0 0x7fcf840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72e7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9d293640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c72f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72e7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:36044/0 (socket says 172.21.15.137:36044) 2024-09-12T12:14:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 -- 172.21.15.137:0/3502967220 learned_addr learned my addr 172.21.15.137:0/3502967220 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9da94640 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcf840a2da0 0x7fcf840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf8affd640 1 -- 172.21.15.137:0/2558096649 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fcf8c0cf770 con 0x7fcf98101370 2024-09-12T12:14:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 -- 172.21.15.137:0/3502967220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 msgr2=0x5615c72f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 -- 172.21.15.137:0/3502967220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 msgr2=0x5615c72f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c72f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 -- 172.21.15.137:0/3502967220 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c71f3340 con 0x5615c72e7af0 2024-09-12T12:14:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72e7ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005f40 tx=0x7fcf9000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929f13ad490fda98 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf9e295640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcf840a2da0 0x7fcf840a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000fc90 tx=0x7fcf90018000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf83fff640 1 -- 172.21.15.137:0/3502967220 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90022070 con 0x5615c72e7af0 2024-09-12T12:14:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf83fff640 1 -- 172.21.15.137:0/3502967220 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf9000ec50 con 0x5615c72e7af0 2024-09-12T12:14:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.716+0000 7fcf83fff640 1 -- 172.21.15.137:0/3502967220 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004050 con 0x5615c72e7af0 2024-09-12T12:14:47.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 67680 17.45 MB/sec warmup 22 sec latency 10.743 ms 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3502967220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 msgr2=0x5615c72e7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72e7ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf90005f40 tx=0x7fcf9000f790 comp rx=0 tx=0).stop 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3502967220 shutdown_connections 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c72f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3502967220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72e7ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3502967220 >> 172.21.15.137:0/3502967220 conn(0x5615c72e4060 msgr2=0x5615c72e4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3502967220 shutdown_connections 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3502967220 wait complete. 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.717+0000 7fcfa2a95740 1 -- start start 2024-09-12T12:14:47.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 1 Processor -- start 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72f2cb0 0x5615c72f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5615c72f5880 con 0x5615c72e7af0 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5615c71d5290 con 0x5615c72e8410 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5615c72f5a00 con 0x5615c72f2cb0 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcfa2a95740 10 client.0 _ll_register_callbacks cb 0x5615c7206140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72efbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44528/0 (socket says 172.21.15.137:44528) 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 -- 172.21.15.137:0/3617107207 learned_addr learned my addr 172.21.15.137:0/3617107207 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9e295640 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72f2cb0 0x5615c72f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9d293640 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 -- 172.21.15.137:0/3617107207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72f2cb0 msgr2=0x5615c72f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72f2cb0 0x5615c72f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 -- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 msgr2=0x5615c72f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.718+0000 7fcf9da94640 1 -- 172.21.15.137:0/3617107207 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c6f5f210 con 0x5615c72e7af0 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf9da94640 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72efbd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcf940043b0 tx=0x7fcf940043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797797bdcf39dbc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf817fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf817fa640 1 -- 172.21.15.137:0/3617107207 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94035040 con 0x5615c72e7af0 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf817fa640 1 -- 172.21.15.137:0/3617107207 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fcf94005450 con 0x5615c72e7af0 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf9000ec50 con 0x5615c72e7af0 2024-09-12T12:14:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf817fa640 1 -- 172.21.15.137:0/3617107207 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf94005650 con 0x5615c72e7af0 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 10 client.15537 fetch_fsmap learned FSMap version 27 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615c702b620 con 0x5615c72e7af0 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf817fa640 1 -- 172.21.15.137:0/3617107207 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf9000ec50 con 0x5615c72e7af0 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf817fa640 1 -- 172.21.15.137:0/3617107207 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcf94031af0 con 0x5615c72e7af0 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 10 client.15537 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 20 client.15537 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c7302bf0 0x5615c7305020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 4 client.15537 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615c7259a20 con 0x5615c7302bf0 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.719+0000 7fcf9d293640 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c7302bf0 0x5615c7305020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.720+0000 7fcf817fa640 10 client.15537 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:47.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.720+0000 7fcf9d293640 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c7302bf0 0x5615c7305020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af70aaf55e0fcc9d server_cookie=3bfe0452d3a479d4 in_seq=0 out_seq=0 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.720+0000 7fcf817fa640 1 -- 172.21.15.137:0/3617107207 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615c7259a20 con 0x5615c7302bf0 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.720+0000 7fcf817fa640 10 client.15537 handle_command_reply: tid=0 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.720+0000 7fcfa2a95740 1 client.15537 shutdown 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 msgr2=0x5615c72efbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72efbd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcf940043b0 tx=0x7fcf940043e0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 shutdown_connections 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72f2cb0 0x5615c72f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c7302bf0 0x5615c7305020 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3617107207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72efbd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcf817fa640 0 client.15537 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 >> 172.21.15.137:0/3617107207 conn(0x5615c72e4060 msgr2=0x5615c72e31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 shutdown_connections 2024-09-12T12:14:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3617107207 wait complete. 2024-09-12T12:14:47.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.721+0000 7fcfa2a95740 20 client.15537 trim_cache size 0 max 16384 2024-09-12T12:14:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 Processor -- start 2024-09-12T12:14:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 -- start start 2024-09-12T12:14:47.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c730a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981b21d0 con 0x5615c72f2cb0 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981afa40 con 0x5615c72e8410 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ad890 con 0x5615c72e7af0 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcf9e295640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c730a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcf9da94640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcf9d293640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcf9e295640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c730a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44540/0 (socket says 172.21.15.137:44540) 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.735+0000 7fcf9e295640 1 -- 172.21.15.137:0/3300232377 learned_addr learned my addr 172.21.15.137:0/3300232377 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf9e295640 1 -- 172.21.15.137:0/3300232377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 msgr2=0x5615c72ea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf9e295640 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72ea5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf9e295640 1 -- 172.21.15.137:0/3300232377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 msgr2=0x5615c72f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf9e295640 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf9e295640 1 -- 172.21.15.137:0/3300232377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c72e4d50 con 0x5615c72f2cb0 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf9d293640 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf9e295640 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c730a930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcf90003b90 tx=0x7fcf90003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edc17b00bbe8bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf837fe640 1 -- 172.21.15.137:0/3300232377 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90022040 con 0x5615c72f2cb0 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf837fe640 1 -- 172.21.15.137:0/3300232377 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90003e30 con 0x5615c72f2cb0 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.736+0000 7fcf837fe640 1 -- 172.21.15.137:0/3300232377 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90005890 con 0x5615c72f2cb0 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3300232377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 msgr2=0x5615c730a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c730a930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcf90003b90 tx=0x7fcf90003bc0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3300232377 shutdown_connections 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72f2cb0 0x5615c730a930 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72e8410 0x5615c72f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/3300232377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e7af0 0x5615c72ea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3300232377 >> 172.21.15.137:0/3300232377 conn(0x5615c72e4060 msgr2=0x5615c72f1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3300232377 shutdown_connections 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 -- 172.21.15.137:0/3300232377 wait complete. 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 -- start start 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.737+0000 7fcfa2a95740 1 Processor -- start 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72fb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e8410 0x5615c72ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72f2cb0 0x5615c72ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981adf00 con 0x5615c72e7af0 2024-09-12T12:14:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981af370 con 0x5615c72f2cb0 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf981ae5d0 con 0x5615c72e8410 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 10 client.0 _ll_register_callbacks cb 0x7fcf94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9e295640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72f2cb0 0x5615c72ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72fb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44552/0 (socket says 172.21.15.137:44552) 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 -- 172.21.15.137:0/2212669919 learned_addr learned my addr 172.21.15.137:0/2212669919 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9d293640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e8410 0x5615c72ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 -- 172.21.15.137:0/2212669919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e8410 msgr2=0x5615c72ec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e8410 0x5615c72ec9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 -- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72f2cb0 msgr2=0x5615c72ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72f2cb0 0x5615c72ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 -- 172.21.15.137:0/2212669919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c72ed750 con 0x5615c72e7af0 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9e295640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72f2cb0 0x5615c72ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9d293640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e8410 0x5615c72ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf9da94640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72fb450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcf94031550 tx=0x7fcf94003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bc9fbf3045a18d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf817fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf817fa640 1 -- 172.21.15.137:0/2212669919 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94035070 con 0x5615c72e7af0 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf90003e30 con 0x5615c72e7af0 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.738+0000 7fcf817fa640 1 -- 172.21.15.137:0/2212669919 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fcf940317b0 con 0x5615c72e7af0 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcf817fa640 1 -- 172.21.15.137:0/2212669919 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcf940319b0 con 0x5615c72e7af0 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 10 client.15543 fetch_fsmap learned FSMap version 27 2024-09-12T12:14:47.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615c736d9a0 con 0x5615c72e7af0 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcf817fa640 1 -- 172.21.15.137:0/2212669919 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf90003e30 con 0x5615c72e7af0 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcf817fa640 1 -- 172.21.15.137:0/2212669919 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcf94004ba0 con 0x5615c72e7af0 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 10 client.15543 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 20 client.15543 populate_metadata read hostname 'smithi137' 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c736e0f0 0x5615c7370530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 4 client.15543 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615c7253d70 con 0x5615c736e0f0 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcf9d293640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c736e0f0 0x5615c7370530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcf817fa640 10 client.15543 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.739+0000 7fcf9d293640 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c736e0f0 0x5615c7370530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ede0e03955c20343 server_cookie=ca9ab49a3b2662d3 in_seq=0 out_seq=0 2024-09-12T12:14:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcf817fa640 1 -- 172.21.15.137:0/2212669919 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615c7253d70 con 0x5615c736e0f0 2024-09-12T12:14:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcf817fa640 10 client.15543 handle_command_reply: tid=0 2024-09-12T12:14:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 client.15543 shutdown 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 msgr2=0x5615c72fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72fb450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcf94031550 tx=0x7fcf94003c20 comp rx=0 tx=0).stop 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 shutdown_connections 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5615c72f2cb0 0x5615c72ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615c72e8410 0x5615c72ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5615c736e0f0 0x5615c7370530 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 --2- 172.21.15.137:0/2212669919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5615c72e7af0 0x5615c72fb450 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcf817fa640 0 client.15543 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 >> 172.21.15.137:0/2212669919 conn(0x5615c72e4060 msgr2=0x5615c72e29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 shutdown_connections 2024-09-12T12:14:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 1 -- 172.21.15.137:0/2212669919 wait complete. 2024-09-12T12:14:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.818+0000 7fcfa2a95740 20 client.15543 trim_cache size 0 max 16384 2024-09-12T12:14:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:14:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:14:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:14:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcf840a2da0 msgr2=0x7fcf840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcf840a2da0 0x7fcf840a5260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000fc90 tx=0x7fcf90018000 comp rx=0 tx=0).stop 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf98101370 msgr2=0x7fcf9810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf98101370 0x7fcf9810e870 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00e9f0 tx=0x7fcf8c00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 shutdown_connections 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcf840a2da0 0x7fcf840a5260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf981069e0 0x7fcf9810edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf98101370 0x7fcf9810e870 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 --2- 172.21.15.137:0/2558096649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf981009d0 0x7fcf9810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:14:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 >> 172.21.15.137:0/2558096649 conn(0x7fcf980fc6f0 msgr2=0x7fcf980fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:14:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 shutdown_connections 2024-09-12T12:14:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:14:47.819+0000 7fcf9fcf8640 1 -- 172.21.15.137:0/2558096649 wait complete. 2024-09-12T12:14:48.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 69861 17.02 MB/sec warmup 23 sec latency 69.134 ms 2024-09-12T12:14:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:48 smithi189 ceph-mon[35475]: pgmap v51: 97 pgs: 97 active+clean; 21 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.6 MiB/s wr, 363 op/s 2024-09-12T12:14:49.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:48 smithi171 ceph-mon[35098]: pgmap v51: 97 pgs: 97 active+clean; 21 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.6 MiB/s wr, 363 op/s 2024-09-12T12:14:49.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:48 smithi137 ceph-mon[30950]: pgmap v51: 97 pgs: 97 active+clean; 21 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.6 MiB/s wr, 363 op/s 2024-09-12T12:14:49.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 72083 16.89 MB/sec warmup 24 sec latency 11.284 ms 2024-09-12T12:14:50.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 74967 16.90 MB/sec warmup 25 sec latency 23.537 ms 2024-09-12T12:14:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:50 smithi189 ceph-mon[35475]: pgmap v52: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 535 op/s 2024-09-12T12:14:51.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:50 smithi171 ceph-mon[35098]: pgmap v52: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 535 op/s 2024-09-12T12:14:51.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:50 smithi137 ceph-mon[30950]: pgmap v52: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 535 op/s 2024-09-12T12:14:51.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 78052 16.92 MB/sec warmup 26 sec latency 13.894 ms 2024-09-12T12:14:52.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 80150 16.57 MB/sec warmup 27 sec latency 13.956 ms 2024-09-12T12:14:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:52 smithi189 ceph-mon[35475]: pgmap v53: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:53.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:52 smithi171 ceph-mon[35098]: pgmap v53: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:53.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:52 smithi137 ceph-mon[30950]: pgmap v53: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:53.716 INFO:tasks.workunit.client.0.smithi137.stdout: 1 82385 16.47 MB/sec warmup 28 sec latency 10.056 ms 2024-09-12T12:14:54.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 85505 16.54 MB/sec warmup 29 sec latency 11.050 ms 2024-09-12T12:14:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:54 smithi189 ceph-mon[35475]: pgmap v54: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:55.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:54 smithi171 ceph-mon[35098]: pgmap v54: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:55.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:54 smithi137 ceph-mon[30950]: pgmap v54: 97 pgs: 97 active+clean; 19 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:55.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 88480 16.48 MB/sec warmup 30 sec latency 11.833 ms 2024-09-12T12:14:56.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 89517 16.11 MB/sec warmup 31 sec latency 184.705 ms 2024-09-12T12:14:57.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:56 smithi189 ceph-mon[35475]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 411 op/s 2024-09-12T12:14:57.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:56 smithi171 ceph-mon[35098]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 411 op/s 2024-09-12T12:14:57.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:56 smithi137 ceph-mon[30950]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 411 op/s 2024-09-12T12:14:57.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 91716 15.99 MB/sec warmup 32 sec latency 272.974 ms 2024-09-12T12:14:58.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 93449 15.77 MB/sec warmup 33 sec latency 28.103 ms 2024-09-12T12:14:59.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:14:58 smithi189 ceph-mon[35475]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:59.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:14:58 smithi171 ceph-mon[35098]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:59.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:14:58 smithi137 ceph-mon[30950]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.3 MiB/s wr, 297 op/s 2024-09-12T12:14:59.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 96240 15.80 MB/sec warmup 34 sec latency 14.733 ms 2024-09-12T12:15:00.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 99191 15.77 MB/sec warmup 35 sec latency 12.780 ms 2024-09-12T12:15:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 -- 172.21.15.137:0/2519285282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 msgr2=0x7f949c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/2519285282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c100e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f948c009930 tx=0x7f948c02a420 comp rx=0 tx=0).stop 2024-09-12T12:15:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 -- 172.21.15.137:0/2519285282 shutdown_connections 2024-09-12T12:15:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/2519285282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f949c101350 0x7f949c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/2519285282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c100e10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/2519285282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f949c1069c0 0x7f949c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 -- 172.21.15.137:0/2519285282 >> 172.21.15.137:0/2519285282 conn(0x7f949c077990 msgr2=0x7f949c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 -- 172.21.15.137:0/2519285282 shutdown_connections 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.908+0000 7f94a3ce6640 1 -- 172.21.15.137:0/2519285282 wait complete. 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 Processor -- start 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 -- start start 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f949c101350 0x7f949c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f949c1069c0 0x7f949c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f949c110e00 con 0x7f949c1009b0 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f949c110c80 con 0x7f949c101350 2024-09-12T12:15:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a3ce6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f949c110f80 con 0x7f949c1069c0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a1a82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a1a82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51338/0 (socket says 172.21.15.137:51338) 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a1a82640 1 -- 172.21.15.137:0/1922334164 learned_addr learned my addr 172.21.15.137:0/1922334164 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a1281640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f949c101350 0x7f949c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a2283640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f949c1069c0 0x7f949c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a1a82640 1 -- 172.21.15.137:0/1922334164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f949c1069c0 msgr2=0x7f949c19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a1a82640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f949c1069c0 0x7f949c19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.909+0000 7f94a1a82640 1 -- 172.21.15.137:0/1922334164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f949c101350 msgr2=0x7f949c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f94a1a82640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f949c101350 0x7f949c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f94a1a82640 1 -- 172.21.15.137:0/1922334164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c19c2e0 con 0x7f949c1009b0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f94a1a82640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c1972f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f949800eb00 tx=0x7f949800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f9492ffd640 1 -- 172.21.15.137:0/1922334164 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949800bc30 con 0x7f949c1009b0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949c19c5d0 con 0x7f949c1009b0 2024-09-12T12:15:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f9492ffd640 1 -- 172.21.15.137:0/1922334164 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949800bdd0 con 0x7f949c1009b0 2024-09-12T12:15:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f9492ffd640 1 -- 172.21.15.137:0/1922334164 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498017410 con 0x7f949c1009b0 2024-09-12T12:15:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.910+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949c0698a0 con 0x7f949c1009b0 2024-09-12T12:15:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.912+0000 7f9492ffd640 1 -- 172.21.15.137:0/1922334164 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9498005020 con 0x7f949c1009b0 2024-09-12T12:15:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.912+0000 7f94a6a7f740 1 Processor -- start 2024-09-12T12:15:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a6a7f740 1 -- start start 2024-09-12T12:15:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b04338dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b04347b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b0434a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555b0429d2f0 con 0x555b043480c0 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555b04245270 con 0x555b04339310 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555b042992b0 con 0x555b043389f0 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f9492ffd640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94880a2da0 0x7f94880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a1a82640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b04347b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a2283640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b04338dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a1281640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b0434a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a2283640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b04338dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:51002/0 (socket says 172.21.15.137:51002) 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a2283640 1 -- 172.21.15.137:0/2515955146 learned_addr learned my addr 172.21.15.137:0/2515955146 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f94a2283640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94880a2da0 0x7f94880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.913+0000 7f9492ffd640 1 -- 172.21.15.137:0/1922334164 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f94980d0c90 con 0x7f949c1009b0 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a2283640 1 -- 172.21.15.137:0/2515955146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 msgr2=0x555b04347b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a2283640 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b04347b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a2283640 1 -- 172.21.15.137:0/2515955146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 msgr2=0x555b0434a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a2283640 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b0434a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a2283640 1 -- 172.21.15.137:0/2515955146 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b04244340 con 0x555b043389f0 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a1a82640 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b04347b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a2283640 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b04338dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9494011e00 tx=0x7f949400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9fecae7457e3b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f94a2283640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94880a2da0 0x7f94880a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f949400fc90 tx=0x7f9494018000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f9487fff640 1 -- 172.21.15.137:0/2515955146 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494022070 con 0x555b043389f0 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f9487fff640 1 -- 172.21.15.137:0/2515955146 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949400ed10 con 0x555b043389f0 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.914+0000 7f9487fff640 1 -- 172.21.15.137:0/2515955146 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494005850 con 0x555b043389f0 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2515955146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 msgr2=0x555b04338dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b04338dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9494011e00 tx=0x7f949400f790 comp rx=0 tx=0).stop 2024-09-12T12:15:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2515955146 shutdown_connections 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b0434a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b04347b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2515955146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b04338dd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2515955146 >> 172.21.15.137:0/2515955146 conn(0x555b04335060 msgr2=0x555b04335440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2515955146 shutdown_connections 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.915+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2515955146 wait complete. 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 -- start start 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 Processor -- start 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04345030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b04345570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04345ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x555b0434acc0 con 0x555b043389f0 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555b04226290 con 0x555b043480c0 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555b0434ae40 con 0x555b04339310 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a6a7f740 10 client.0 _ll_register_callbacks cb 0x555b04257140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a1a82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04345030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a1281640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b04345570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a2283640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04345ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a1a82640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04345030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51356/0 (socket says 172.21.15.137:51356) 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a1281640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b04345570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:51018/0 (socket says 172.21.15.137:51018) 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a2283640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04345ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55120/0 (socket says 172.21.15.137:55120) 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.916+0000 7f94a1a82640 1 -- 172.21.15.137:0/976602337 learned_addr learned my addr 172.21.15.137:0/976602337 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a1a82640 1 -- 172.21.15.137:0/976602337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 msgr2=0x555b04345570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a1a82640 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b04345570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a1a82640 1 -- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 msgr2=0x555b04345ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a1a82640 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04345ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a1a82640 1 -- 172.21.15.137:0/976602337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b03fb0210 con 0x555b043389f0 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a2283640 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04345ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a1a82640 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04345030 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f949800b750 tx=0x7f949806c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd094b18e628427 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94857fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94857fa640 1 -- 172.21.15.137:0/976602337 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949809d250 con 0x555b043389f0 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94857fa640 1 -- 172.21.15.137:0/976602337 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f94980a1170 con 0x555b043389f0 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f949400ed10 con 0x555b043389f0 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94857fa640 1 -- 172.21.15.137:0/976602337 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94980a1550 con 0x555b043389f0 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a6a7f740 10 client.15555 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b0407c620 con 0x555b043389f0 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.917+0000 7f94857fa640 1 -- 172.21.15.137:0/976602337 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949400ed10 con 0x555b043389f0 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94857fa640 1 -- 172.21.15.137:0/976602337 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f94980a5980 con 0x555b043389f0 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a6a7f740 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a6a7f740 10 client.15555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a6a7f740 20 client.15555 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043580c0 0x555b0435a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a6a7f740 4 client.15555 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b042aaa20 con 0x555b043580c0 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a1281640 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043580c0 0x555b0435a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94857fa640 10 client.15555 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.918+0000 7f94a1281640 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043580c0 0x555b0435a4b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1191707b0099c83a server_cookie=453db102956abfd8 in_seq=0 out_seq=0 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94857fa640 1 -- 172.21.15.137:0/976602337 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b042aaa20 con 0x555b043580c0 2024-09-12T12:15:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94857fa640 10 client.15555 handle_command_reply: tid=0 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 client.15555 shutdown 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 msgr2=0x555b04345030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04345030 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f949800b750 tx=0x7f949806c180 comp rx=0 tx=0).stop 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 shutdown_connections 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04345ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b04345570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043580c0 0x555b0435a4b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/976602337 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04345030 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94857fa640 0 client.15555 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 >> 172.21.15.137:0/976602337 conn(0x555b04335060 msgr2=0x555b0433ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 shutdown_connections 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 1 -- 172.21.15.137:0/976602337 wait complete. 2024-09-12T12:15:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.919+0000 7f94a6a7f740 20 client.15555 trim_cache size 0 max 16384 2024-09-12T12:15:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.938+0000 7f94a6a7f740 1 Processor -- start 2024-09-12T12:15:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.938+0000 7f94a6a7f740 1 -- start start 2024-09-12T12:15:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b0435fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b043602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b04342290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1addb0 con 0x555b043480c0 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1ab620 con 0x555b04339310 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1a9470 con 0x555b043389f0 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a2283640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b04342290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b043602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1a82640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b0435fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b043602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55130/0 (socket says 172.21.15.137:55130) 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a2283640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b04342290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51360/0 (socket says 172.21.15.137:51360) 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 -- 172.21.15.137:0/2038507876 learned_addr learned my addr 172.21.15.137:0/2038507876 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 -- 172.21.15.137:0/2038507876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 msgr2=0x555b0435fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b0435fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 -- 172.21.15.137:0/2038507876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 msgr2=0x555b04342290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b04342290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 -- 172.21.15.137:0/2038507876 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0435a820 con 0x555b04339310 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a2283640 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b04342290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.939+0000 7f94a1281640 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b043602e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f948c02a940 tx=0x7f948c031900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87995629a96c7b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94877fe640 1 -- 172.21.15.137:0/2038507876 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c035040 con 0x555b04339310 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94877fe640 1 -- 172.21.15.137:0/2038507876 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948c005020 con 0x555b04339310 2024-09-12T12:15:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94877fe640 1 -- 172.21.15.137:0/2038507876 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c004a90 con 0x555b04339310 2024-09-12T12:15:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2038507876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 msgr2=0x555b043602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b043602e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f948c02a940 tx=0x7f948c031900 comp rx=0 tx=0).stop 2024-09-12T12:15:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2038507876 shutdown_connections 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043480c0 0x555b04342290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b04339310 0x555b043602e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/2038507876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b043389f0 0x555b0435fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2038507876 >> 172.21.15.137:0/2038507876 conn(0x555b04335060 msgr2=0x555b043470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.940+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2038507876 shutdown_connections 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.941+0000 7f94a6a7f740 1 -- 172.21.15.137:0/2038507876 wait complete. 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.941+0000 7f94a6a7f740 1 -- start start 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.941+0000 7f94a6a7f740 1 Processor -- start 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.941+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04342170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.941+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b043426b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a6a7f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04342bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1a9ae0 con 0x555b043389f0 2024-09-12T12:15:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b043426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1aaf50 con 0x555b043480c0 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a6a7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f949c1aa1b0 con 0x555b04339310 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b043426b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:51034/0 (socket says 172.21.15.137:51034) 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 -- 172.21.15.137:0/4050239253 learned_addr learned my addr 172.21.15.137:0/4050239253 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a2283640 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04342bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1a82640 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04342170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a6a7f740 10 client.0 _ll_register_callbacks cb 0x7f9498009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 -- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 msgr2=0x555b04342bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04342bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 -- 172.21.15.137:0/4050239253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 msgr2=0x555b04342170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04342170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 -- 172.21.15.137:0/4050239253 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b043c1dc0 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94a1281640 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b043426b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f948c006010 tx=0x7f948c0045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd09eea5da0150cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94857fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.942+0000 7f94857fa640 1 -- 172.21.15.137:0/4050239253 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c035040 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f948c005020 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94857fa640 1 -- 172.21.15.137:0/4050239253 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f948c0051d0 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94857fa640 1 -- 172.21.15.137:0/4050239253 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f948c05e020 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 10 client.25123 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555b043c3090 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94857fa640 1 -- 172.21.15.137:0/4050239253 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948c005020 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94857fa640 1 -- 172.21.15.137:0/4050239253 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f948c0035f0 con 0x555b04339310 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 10 client.25123 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 10 client.25123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 20 client.25123 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043c3880 0x555b043c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 4 client.25123 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:15:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b042a4d70 con 0x555b043c3880 2024-09-12T12:15:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.943+0000 7f94a1a82640 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043c3880 0x555b043c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.944+0000 7f94857fa640 10 client.25123 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.944+0000 7f94a1a82640 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043c3880 0x555b043c5ca0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bce2bbce9fcb695 server_cookie=7b9e46b239ef8e25 in_seq=0 out_seq=0 2024-09-12T12:15:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94857fa640 1 -- 172.21.15.137:0/4050239253 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b042a4d70 con 0x555b043c3880 2024-09-12T12:15:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94857fa640 10 client.25123 handle_command_reply: tid=0 2024-09-12T12:15:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 client.25123 shutdown 2024-09-12T12:15:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 msgr2=0x555b043426b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b043426b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f948c006010 tx=0x7f948c0045c0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 shutdown_connections 2024-09-12T12:15:00.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b043480c0 0x555b04342bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b04339310 0x555b043426b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x555b043c3880 0x555b043c5ca0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 --2- 172.21.15.137:0/4050239253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555b043389f0 0x555b04342170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 >> 172.21.15.137:0/4050239253 conn(0x555b04335060 msgr2=0x555b04344ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94857fa640 0 client.25123 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 shutdown_connections 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 1 -- 172.21.15.137:0/4050239253 wait complete. 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.995+0000 7f94a6a7f740 20 client.25123 trim_cache size 0 max 16384 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:15:00.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:15:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:15:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:15:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94880a2da0 msgr2=0x7f94880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.998 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94880a2da0 0x7f94880a5260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f949400fc90 tx=0x7f9494018000 comp rx=0 tx=0).stop 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 msgr2=0x7f949c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c1972f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f949800eb00 tx=0x7f949800ef00 comp rx=0 tx=0).stop 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 shutdown_connections 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f94880a2da0 0x7f94880a5260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f949c1069c0 0x7f949c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f949c101350 0x7f949c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 --2- 172.21.15.137:0/1922334164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f949c1009b0 0x7f949c1972f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.996+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 >> 172.21.15.137:0/1922334164 conn(0x7f949c077990 msgr2=0x7f949c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.997+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 shutdown_connections 2024-09-12T12:15:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:00.997+0000 7f94a3ce6640 1 -- 172.21.15.137:0/1922334164 wait complete. 2024-09-12T12:15:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:00 smithi189 ceph-mon[35475]: pgmap v57: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.5 MiB/s wr, 613 op/s 2024-09-12T12:15:01.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:00 smithi171 ceph-mon[35098]: pgmap v57: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.5 MiB/s wr, 613 op/s 2024-09-12T12:15:01.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:00 smithi137 ceph-mon[30950]: pgmap v57: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.5 MiB/s wr, 613 op/s 2024-09-12T12:15:01.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 100390 15.50 MB/sec warmup 36 sec latency 31.043 ms 2024-09-12T12:15:02.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 103160 15.55 MB/sec warmup 37 sec latency 24.997 ms 2024-09-12T12:15:02.913 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T12:15:02.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:15:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:02 smithi189 ceph-mon[35475]: pgmap v58: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 429 op/s 2024-09-12T12:15:03.179 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:15:03.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:02 smithi171 ceph-mon[35098]: pgmap v58: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 429 op/s 2024-09-12T12:15:03.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:02 smithi137 ceph-mon[30950]: pgmap v58: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 429 op/s 2024-09-12T12:15:03.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 105378 15.35 MB/sec warmup 38 sec latency 15.424 ms 2024-09-12T12:15:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.712+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1683411613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34100e20 msgr2=0x7f3c34101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.712+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1683411613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34100e20 0x7f3c34101200 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009900 tx=0x7f3c2c02a480 comp rx=0 tx=0).stop 2024-09-12T12:15:03.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1683411613 shutdown_connections 2024-09-12T12:15:03.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1683411613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c34102ef0 0x7f3c3410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1683411613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c34102570 0x7f3c341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1683411613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34100e20 0x7f3c34101200 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1683411613 >> 172.21.15.137:0/1683411613 conn(0x7f3c340fc7d0 msgr2=0x7f3c340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1683411613 shutdown_connections 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1683411613 wait complete. 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.713+0000 7f3c38f7d640 1 Processor -- start 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c38f7d640 1 -- start start 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c38f7d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c34100e20 0x7f3c341984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c38f7d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34102570 0x7f3c34198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c38f7d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c34102ef0 0x7f3c3419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c38f7d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34110f80 con 0x7f3c34102570 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c38f7d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34110e00 con 0x7f3c34102ef0 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c38f7d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34111100 con 0x7f3c34100e20 2024-09-12T12:15:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34102570 0x7f3c34198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34102570 0x7f3c34198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51374/0 (socket says 172.21.15.137:51374) 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 -- 172.21.15.137:0/1442405427 learned_addr learned my addr 172.21.15.137:0/1442405427 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c3259c640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c34100e20 0x7f3c341984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c32d9d640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c34102ef0 0x7f3c3419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 -- 172.21.15.137:0/1442405427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c34100e20 msgr2=0x7f3c341984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c34100e20 0x7f3c341984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 -- 172.21.15.137:0/1442405427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c34102ef0 msgr2=0x7f3c3419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c34102ef0 0x7f3c3419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 -- 172.21.15.137:0/1442405427 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3419d490 con 0x7f3c34102570 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.714+0000 7f3c31d9b640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34102570 0x7f3c34198a20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00eab0 tx=0x7f3c1c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.715+0000 7f3c2b7fe640 1 -- 172.21.15.137:0/1442405427 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00ce40 con 0x7f3c34102570 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.715+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3419d780 con 0x7f3c34102570 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.715+0000 7f3c2b7fe640 1 -- 172.21.15.137:0/1442405427 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c011950 con 0x7f3c34102570 2024-09-12T12:15:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.715+0000 7f3c2b7fe640 1 -- 172.21.15.137:0/1442405427 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c010a80 con 0x7f3c34102570 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.715+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c341a5060 con 0x7f3c34102570 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.716+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c34101200 con 0x7f3c34102570 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.717+0000 7f3c2b7fe640 1 -- 172.21.15.137:0/1442405427 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3c1c011b00 con 0x7f3c34102570 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.718+0000 7f3c2b7fe640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3c140a2cf0 0x7f3c140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.718+0000 7f3c3259c640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3c140a2cf0 0x7f3c140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.718+0000 7f3c2b7fe640 1 -- 172.21.15.137:0/1442405427 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f3c1c0cfa00 con 0x7f3c34102570 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.718+0000 7f3c3259c640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3c140a2cf0 0x7f3c140a51b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009230 tx=0x7f3c2c0058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:03.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.722+0000 7f3c2b7fe640 1 -- 172.21.15.137:0/1442405427 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c1c016030 con 0x7f3c34102570 2024-09-12T12:15:03.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.935+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c341a52c0 con 0x7f3c34102570 2024-09-12T12:15:03.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.936+0000 7f3c2b7fe640 1 -- 172.21.15.137:0/1442405427 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f3c1c09c670 con 0x7f3c34102570 2024-09-12T12:15:03.941 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:15:03.941 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":27,"btime":"2024-09-12T12:14:37:790728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:14:36.964904+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 27 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3c140a2cf0 msgr2=0x7f3c140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3c140a2cf0 0x7f3c140a51b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009230 tx=0x7f3c2c0058d0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34102570 msgr2=0x7f3c34198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34102570 0x7f3c34198a20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00eab0 tx=0x7f3c1c00ef70 comp rx=0 tx=0).stop 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 shutdown_connections 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3c140a2cf0 0x7f3c140a51b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c34102ef0 0x7f3c3419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c34102570 0x7f3c34198a20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 --2- 172.21.15.137:0/1442405427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c34100e20 0x7f3c341984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.938+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 >> 172.21.15.137:0/1442405427 conn(0x7f3c340fc7d0 msgr2=0x7f3c3406a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:03.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.939+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 shutdown_connections 2024-09-12T12:15:03.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:03.939+0000 7f3c38f7d640 1 -- 172.21.15.137:0/1442405427 wait complete. 2024-09-12T12:15:04.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 107334 15.27 MB/sec warmup 39 sec latency 11.153 ms 2024-09-12T12:15:04.806 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T12:15:04.807 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T12:15:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:04 smithi137 ceph-mon[30950]: pgmap v59: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 429 op/s 2024-09-12T12:15:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:04 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1442405427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:05.063 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:15:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:04 smithi189 ceph-mon[35475]: pgmap v59: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 429 op/s 2024-09-12T12:15:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:04 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1442405427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:05.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:04 smithi171 ceph-mon[35098]: pgmap v59: 97 pgs: 97 active+clean; 38 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.8 MiB/s wr, 429 op/s 2024-09-12T12:15:05.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:04 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1442405427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:05.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.550+0000 7f84bf494640 1 -- 172.21.15.137:0/1762795609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81036e0 msgr2=0x7f84b8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.550+0000 7f84bf494640 1 --2- 172.21.15.137:0/1762795609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81036e0 0x7f84b8107b90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009930 tx=0x7f84ac02a420 comp rx=0 tx=0).stop 2024-09-12T12:15:05.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 -- 172.21.15.137:0/1762795609 shutdown_connections 2024-09-12T12:15:05.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 --2- 172.21.15.137:0/1762795609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84b81082a0 0x7f84b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 --2- 172.21.15.137:0/1762795609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81036e0 0x7f84b8107b90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 --2- 172.21.15.137:0/1762795609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b8102d30 0x7f84b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 -- 172.21.15.137:0/1762795609 >> 172.21.15.137:0/1762795609 conn(0x7f84b80fc7d0 msgr2=0x7f84b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 -- 172.21.15.137:0/1762795609 shutdown_connections 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 -- 172.21.15.137:0/1762795609 wait complete. 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 Processor -- start 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.551+0000 7f84bf494640 1 -- start start 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bf494640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84b8102d30 0x7f84b8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bf494640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b81036e0 0x7f84b8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bf494640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81082a0 0x7f84b819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bf494640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81120a0 con 0x7f84b81082a0 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bf494640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8111f20 con 0x7f84b81036e0 2024-09-12T12:15:05.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bf494640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84b8112220 con 0x7f84b8102d30 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bda31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81082a0 0x7f84b819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bd230640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84b8102d30 0x7f84b8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bda31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81082a0 0x7f84b819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51384/0 (socket says 172.21.15.137:51384) 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bda31640 1 -- 172.21.15.137:0/1865718363 learned_addr learned my addr 172.21.15.137:0/1865718363 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bca2f640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b81036e0 0x7f84b8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bd230640 1 -- 172.21.15.137:0/1865718363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b81036e0 msgr2=0x7f84b8198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bd230640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b81036e0 0x7f84b8198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bd230640 1 -- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81082a0 msgr2=0x7f84b819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bd230640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81082a0 0x7f84b819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bd230640 1 -- 172.21.15.137:0/1865718363 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b819d5f0 con 0x7f84b8102d30 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.552+0000 7f84bd230640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84b8102d30 0x7f84b8198640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.553+0000 7f84b67fc640 1 -- 172.21.15.137:0/1865718363 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b000ccc0 con 0x7f84b8102d30 2024-09-12T12:15:05.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.553+0000 7f84b67fc640 1 -- 172.21.15.137:0/1865718363 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84b0010910 con 0x7f84b8102d30 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.553+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b819d8e0 con 0x7f84b8102d30 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.553+0000 7f84b67fc640 1 -- 172.21.15.137:0/1865718363 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0018960 con 0x7f84b8102d30 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.553+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b819dd30 con 0x7f84b8102d30 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.555+0000 7f84b67fc640 1 -- 172.21.15.137:0/1865718363 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f84b0018b00 con 0x7f84b8102d30 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 Processor -- start 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 -- start start 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1645b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e1650400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e1652d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5622e15ab2a0 con 0x5622e1645760 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5622e1552b10 con 0x5622e1646080 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84c186d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622e15a7260 con 0x5622e1650940 2024-09-12T12:15:05.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84bda31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1645b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84bda31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1645b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51398/0 (socket says 172.21.15.137:51398) 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84bda31640 1 -- 172.21.15.137:0/1158721756 learned_addr learned my addr 172.21.15.137:0/1158721756 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84bca2f640 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e1652d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.556+0000 7f84bd230640 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e1650400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84b67fc640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f84a00a2cf0 0x7f84a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bda31640 1 -- 172.21.15.137:0/1158721756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 msgr2=0x5622e1652d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bda31640 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e1652d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bda31640 1 -- 172.21.15.137:0/1158721756 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 msgr2=0x5622e1650400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bda31640 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e1650400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bda31640 1 -- 172.21.15.137:0/1158721756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e131cb90 con 0x5622e1645760 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bd230640 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e1650400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bca2f640 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e1652d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bda31640 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1645b40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f84a8011470 tx=0x7f84a8011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0c44c718be5591 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:05.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84b67fc640 1 -- 172.21.15.137:0/1865718363 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f84b00cf910 con 0x7f84b8102d30 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f84bda31640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f84a00a2cf0 0x7f84a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f849f7fe640 1 -- 172.21.15.137:0/1158721756 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a801d020 con 0x5622e1645760 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f849f7fe640 1 -- 172.21.15.137:0/1158721756 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a8019410 con 0x5622e1645760 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.557+0000 7f849f7fe640 1 -- 172.21.15.137:0/1158721756 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a80186d0 con 0x5622e1645760 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 -- 172.21.15.137:0/1158721756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 msgr2=0x5622e1645b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84bda31640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f84a00a2cf0 0x7f84a00a51b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84a801b000 tx=0x7f84a800ffd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1645b40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f84a8011470 tx=0x7f84a8011930 comp rx=0 tx=0).stop 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 -- 172.21.15.137:0/1158721756 shutdown_connections 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e1652d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e1650400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 --2- 172.21.15.137:0/1158721756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1645b40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 -- 172.21.15.137:0/1158721756 >> 172.21.15.137:0/1158721756 conn(0x5622e1641cf0 msgr2=0x5622e16420d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:05.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 -- 172.21.15.137:0/1158721756 shutdown_connections 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 -- 172.21.15.137:0/1158721756 wait complete. 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.558+0000 7f84c186d740 1 -- start start 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 1 Processor -- start 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1645760 0x5622e16503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e164d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1650940 0x5622e164de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5622e1653510 con 0x5622e1650940 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5622e15336e0 con 0x5622e1645760 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5622e1653690 con 0x5622e1646080 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84c186d740 10 client.0 _ll_register_callbacks cb 0x5622e1557a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84bd230640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1645760 0x5622e16503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84bca2f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e164d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84bda31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1650940 0x5622e164de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84bd230640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1645760 0x5622e16503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55178/0 (socket says 172.21.15.137:55178) 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84bca2f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e164d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:51100/0 (socket says 172.21.15.137:51100) 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84bd230640 1 -- 172.21.15.137:0/522145069 learned_addr learned my addr 172.21.15.137:0/522145069 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.559+0000 7f84bd230640 1 -- 172.21.15.137:0/522145069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 msgr2=0x5622e164d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84bd230640 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e164d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84bd230640 1 -- 172.21.15.137:0/522145069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1650940 msgr2=0x5622e164de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84bd230640 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1650940 0x5622e164de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84bd230640 1 -- 172.21.15.137:0/522145069 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e165c980 con 0x5622e1645760 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84bda31640 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1650940 0x5622e164de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:05.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84bd230640 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1645760 0x5622e16503f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c6a0 tx=0x7f84b0010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3311d3cd628a1996 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f849cff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f849cff9640 1 -- 172.21.15.137:0/522145069 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b00d1e60 con 0x5622e1645760 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f849cff9640 1 -- 172.21.15.137:0/522145069 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f84b009b270 con 0x5622e1645760 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a8019410 con 0x5622e1645760 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f849cff9640 1 -- 172.21.15.137:0/522145069 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84b00cd4d0 con 0x5622e1645760 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84c186d740 10 client.25076 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622e165f210 con 0x5622e1645760 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f849cff9640 1 -- 172.21.15.137:0/522145069 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a8019410 con 0x5622e1645760 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.560+0000 7f849cff9640 1 -- 172.21.15.137:0/522145069 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f84b0097b70 con 0x5622e1645760 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84c186d740 10 client.25076 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84c186d740 10 client.25076 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84c186d740 20 client.25076 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84c186d740 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16606d0 0x5622e1662b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84c186d740 4 client.25076 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622e15b89d0 con 0x5622e16606d0 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84bca2f640 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16606d0 0x5622e1662b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f849cff9640 10 client.25076 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.561+0000 7f84bca2f640 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16606d0 0x5622e1662b20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff3c365d173d6779 server_cookie=c96c644530a7aa0b in_seq=0 out_seq=0 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f849cff9640 1 -- 172.21.15.137:0/522145069 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622e15b89d0 con 0x5622e16606d0 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f849cff9640 10 client.25076 handle_command_reply: tid=0 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 client.25076 shutdown 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1645760 msgr2=0x5622e16503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1645760 0x5622e16503f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c6a0 tx=0x7f84b0010ee0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 shutdown_connections 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16606d0 0x5622e1662b20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1650940 0x5622e164de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e164d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 --2- 172.21.15.137:0/522145069 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1645760 0x5622e16503f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 >> 172.21.15.137:0/522145069 conn(0x5622e1641cf0 msgr2=0x5622e1647a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:05.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f849cff9640 0 client.25076 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:05.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 shutdown_connections 2024-09-12T12:15:05.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 1 -- 172.21.15.137:0/522145069 wait complete. 2024-09-12T12:15:05.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.562+0000 7f84c186d740 20 client.25076 trim_cache size 0 max 16384 2024-09-12T12:15:05.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 Processor -- start 2024-09-12T12:15:05.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 -- start start 2024-09-12T12:15:05.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e16489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e164f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e16683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81b01f0 con 0x5622e1645760 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81ada60 con 0x5622e1646080 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84c186d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81ab8b0 con 0x5622e1650940 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bd230640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e16489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bd230640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e16489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51420/0 (socket says 172.21.15.137:51420) 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bca2f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e164f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bd230640 1 -- 172.21.15.137:0/208571516 learned_addr learned my addr 172.21.15.137:0/208571516 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bda31640 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e16683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bca2f640 1 -- 172.21.15.137:0/208571516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 msgr2=0x5622e16683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bca2f640 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e16683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bca2f640 1 -- 172.21.15.137:0/208571516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 msgr2=0x5622e16489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.581+0000 7f84bca2f640 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e16489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.582+0000 7f84bca2f640 1 -- 172.21.15.137:0/208571516 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622e164c650 con 0x5622e1646080 2024-09-12T12:15:05.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.582+0000 7f84bd230640 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e16489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.582+0000 7f84bca2f640 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e164f610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84ac02a940 tx=0x7f84ac005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17b9b648e432263 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.582+0000 7f849effd640 1 -- 172.21.15.137:0/208571516 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac04a070 con 0x5622e1646080 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.582+0000 7f849effd640 1 -- 172.21.15.137:0/208571516 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac005520 con 0x5622e1646080 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.582+0000 7f849effd640 1 -- 172.21.15.137:0/208571516 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac035040 con 0x5622e1646080 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 -- 172.21.15.137:0/208571516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 msgr2=0x5622e164f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e164f610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84ac02a940 tx=0x7f84ac005be0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 -- 172.21.15.137:0/208571516 shutdown_connections 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1650940 0x5622e16683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1646080 0x5622e164f610 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 --2- 172.21.15.137:0/208571516 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e16489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 -- 172.21.15.137:0/208571516 >> 172.21.15.137:0/208571516 conn(0x5622e1641cf0 msgr2=0x5622e164ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 -- 172.21.15.137:0/208571516 shutdown_connections 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 -- 172.21.15.137:0/208571516 wait complete. 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 -- start start 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.583+0000 7f84c186d740 1 Processor -- start 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84c186d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1658dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84c186d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e16538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84c186d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1650940 0x5622e1653de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84c186d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81abf20 con 0x5622e1645760 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84c186d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81ad390 con 0x5622e1650940 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84c186d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84b81ac5f0 con 0x5622e1646080 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84c186d740 10 client.0 _ll_register_callbacks cb 0x7f84b000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bd230640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1658dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bd230640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1658dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51430/0 (socket says 172.21.15.137:51430) 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bd230640 1 -- 172.21.15.137:0/78760874 learned_addr learned my addr 172.21.15.137:0/78760874 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bca2f640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e16538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bda31640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1650940 0x5622e1653de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bca2f640 1 -- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1650940 msgr2=0x5622e1653de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bca2f640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1650940 0x5622e1653de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bca2f640 1 -- 172.21.15.137:0/78760874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 msgr2=0x5622e1658dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bca2f640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1658dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bca2f640 1 -- 172.21.15.137:0/78760874 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622e164e1a0 con 0x5622e1646080 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.584+0000 7f84bd230640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1658dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f84bca2f640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e16538a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009230 tx=0x7f84ac032f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c45a755d0c5ccb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f849cff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f849cff9640 1 -- 172.21.15.137:0/78760874 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac04a040 con 0x5622e1646080 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f849cff9640 1 -- 172.21.15.137:0/78760874 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f84ac004300 con 0x5622e1646080 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ac005520 con 0x5622e1646080 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f849cff9640 1 -- 172.21.15.137:0/78760874 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f84ac035040 con 0x5622e1646080 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f84c186d740 10 client.25147 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5622e16d0370 con 0x5622e1646080 2024-09-12T12:15:05.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f849cff9640 1 -- 172.21.15.137:0/78760874 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac005520 con 0x5622e1646080 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f849cff9640 1 -- 172.21.15.137:0/78760874 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f84ac0317e0 con 0x5622e1646080 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f84c186d740 10 client.25147 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f84c186d740 10 client.25147 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.585+0000 7f84c186d740 20 client.25147 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.586+0000 7f84c186d740 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16d0b70 0x5622e16d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.586+0000 7f84c186d740 4 client.25147 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.586+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x5622e15b2d20 con 0x5622e16d0b70 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.586+0000 7f84bd230640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16d0b70 0x5622e16d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.586+0000 7f849cff9640 10 client.25147 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:05.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.586+0000 7f84bd230640 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16d0b70 0x5622e16d2f90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3241d98c395c7ef7 server_cookie=ca35a519670125c7 in_seq=0 out_seq=0 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f849cff9640 1 -- 172.21.15.137:0/78760874 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5622e15b2d20 con 0x5622e16d0b70 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f849cff9640 10 client.25147 handle_command_reply: tid=0 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 client.25147 shutdown 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 msgr2=0x5622e16538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e16538a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009230 tx=0x7f84ac032f10 comp rx=0 tx=0).stop 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 shutdown_connections 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5622e1650940 0x5622e1653de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622e1646080 0x5622e16538a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5622e16d0b70 0x5622e16d2f90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 --2- 172.21.15.137:0/78760874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5622e1645760 0x5622e1658dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f849cff9640 0 client.25147 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 >> 172.21.15.137:0/78760874 conn(0x5622e1641cf0 msgr2=0x5622e164f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 shutdown_connections 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 1 -- 172.21.15.137:0/78760874 wait complete. 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.660+0000 7f84c186d740 20 client.25147 trim_cache size 0 max 16384 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:15:05.662 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 27, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 6, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "4b2e6b88": { 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 6, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 133.4, 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:15:05.663 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:05.664 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f84a00a2cf0 msgr2=0x7f84a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f84a00a2cf0 0x7f84a00a51b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f84a801b000 tx=0x7f84a800ffd0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84b8102d30 msgr2=0x7f84b8198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84b8102d30 0x7f84b8198640 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f84b000e9f0 tx=0x7f84b000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 shutdown_connections 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f84a00a2cf0 0x7f84a00a51b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84b81082a0 0x7f84b819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84b81036e0 0x7f84b8198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 --2- 172.21.15.137:0/1865718363 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84b8102d30 0x7f84b8198640 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:05.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 >> 172.21.15.137:0/1865718363 conn(0x7f84b80fc7d0 msgr2=0x7f84b8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:05.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 shutdown_connections 2024-09-12T12:15:05.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:05.661+0000 7f84bf494640 1 -- 172.21.15.137:0/1865718363 wait complete. 2024-09-12T12:15:05.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 15.31 MB/sec warmup 40 sec latency 57.225 ms 2024-09-12T12:15:06.491 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4b2e6b88', quiesce took 3.6 seconds. Will release after: 9.8 2024-09-12T12:15:06.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 14.94 MB/sec warmup 41 sec latency 1057.289 ms 2024-09-12T12:15:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:06 smithi189 ceph-mon[35475]: pgmap v60: 97 pgs: 97 active+clean; 36 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 614 op/s 2024-09-12T12:15:07.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:06 smithi171 ceph-mon[35098]: pgmap v60: 97 pgs: 97 active+clean; 36 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 614 op/s 2024-09-12T12:15:07.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:06 smithi137 ceph-mon[30950]: pgmap v60: 97 pgs: 97 active+clean; 36 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.6 MiB/s wr, 614 op/s 2024-09-12T12:15:07.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 14.58 MB/sec warmup 42 sec latency 2057.376 ms 2024-09-12T12:15:08.717 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 14.24 MB/sec warmup 43 sec latency 3057.474 ms 2024-09-12T12:15:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:08 smithi189 ceph-mon[35475]: pgmap v61: 97 pgs: 97 active+clean; 36 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 500 op/s 2024-09-12T12:15:09.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:08 smithi171 ceph-mon[35098]: pgmap v61: 97 pgs: 97 active+clean; 36 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 500 op/s 2024-09-12T12:15:09.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:08 smithi137 ceph-mon[30950]: pgmap v61: 97 pgs: 97 active+clean; 36 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.9 MiB/s wr, 500 op/s 2024-09-12T12:15:09.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 13.92 MB/sec warmup 44 sec latency 4057.594 ms 2024-09-12T12:15:10.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 13.61 MB/sec warmup 45 sec latency 5057.690 ms 2024-09-12T12:15:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:10 smithi189 ceph-mon[35475]: pgmap v62: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.1 MiB/s wr, 537 op/s 2024-09-12T12:15:11.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:10 smithi171 ceph-mon[35098]: pgmap v62: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.1 MiB/s wr, 537 op/s 2024-09-12T12:15:11.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:10 smithi137 ceph-mon[30950]: pgmap v62: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.1 MiB/s wr, 537 op/s 2024-09-12T12:15:11.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 13.31 MB/sec warmup 46 sec latency 6057.762 ms 2024-09-12T12:15:12.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 13.03 MB/sec warmup 47 sec latency 7057.853 ms 2024-09-12T12:15:13.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:12 smithi171 ceph-mon[35098]: pgmap v63: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 958 KiB/s wr, 221 op/s 2024-09-12T12:15:13.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:12 smithi137 ceph-mon[30950]: pgmap v63: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 958 KiB/s wr, 221 op/s 2024-09-12T12:15:13.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:12 smithi189 ceph-mon[35475]: pgmap v63: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 958 KiB/s wr, 221 op/s 2024-09-12T12:15:13.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 12.76 MB/sec warmup 48 sec latency 8057.969 ms 2024-09-12T12:15:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 -- 172.21.15.137:0/4224240106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed58101360 msgr2=0x7fed5810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 --2- 172.21.15.137:0/4224240106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed58101360 0x7fed5810d890 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fed50009930 tx=0x7fed5002a420 comp rx=0 tx=0).stop 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 -- 172.21.15.137:0/4224240106 shutdown_connections 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 --2- 172.21.15.137:0/4224240106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed58101360 0x7fed5810d890 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 --2- 172.21.15.137:0/4224240106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed581009c0 0x7fed58100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 --2- 172.21.15.137:0/4224240106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed581069d0 0x7fed58106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 -- 172.21.15.137:0/4224240106 >> 172.21.15.137:0/4224240106 conn(0x7fed580fc700 msgr2=0x7fed580feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 -- 172.21.15.137:0/4224240106 shutdown_connections 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.094+0000 7fed5d711640 1 -- 172.21.15.137:0/4224240106 wait complete. 2024-09-12T12:15:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 Processor -- start 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 -- start start 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed581009c0 0x7fed58197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed58101360 0x7fed58197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed581069d0 0x7fed5819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed58110e10 con 0x7fed581069d0 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed58110c90 con 0x7fed58101360 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed5d711640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed58110f90 con 0x7fed581009c0 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed581069d0 0x7fed5819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed581069d0 0x7fed5819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49492/0 (socket says 172.21.15.137:49492) 2024-09-12T12:15:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed577fe640 1 -- 172.21.15.137:0/501708365 learned_addr learned my addr 172.21.15.137:0/501708365 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed567fc640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed58101360 0x7fed58197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed56ffd640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed581009c0 0x7fed58197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.095+0000 7fed577fe640 1 -- 172.21.15.137:0/501708365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed581009c0 msgr2=0x7fed58197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed577fe640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed581009c0 0x7fed58197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed577fe640 1 -- 172.21.15.137:0/501708365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed58101360 msgr2=0x7fed58197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed577fe640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed58101360 0x7fed58197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed577fe640 1 -- 172.21.15.137:0/501708365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5819c230 con 0x7fed581069d0 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed56ffd640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed581009c0 0x7fed58197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed577fe640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed581069d0 0x7fed5819bb00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fed50005ab0 tx=0x7fed50004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed3ffff640 1 -- 172.21.15.137:0/501708365 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50046070 con 0x7fed581069d0 2024-09-12T12:15:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed3ffff640 1 -- 172.21.15.137:0/501708365 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50004440 con 0x7fed581069d0 2024-09-12T12:15:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5819c4c0 con 0x7fed581069d0 2024-09-12T12:15:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed3ffff640 1 -- 172.21.15.137:0/501708365 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50035050 con 0x7fed581069d0 2024-09-12T12:15:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.096+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5806a340 con 0x7fed581069d0 2024-09-12T12:15:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.098+0000 7fed3ffff640 1 -- 172.21.15.137:0/501708365 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fed50049070 con 0x7fed581069d0 2024-09-12T12:15:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 Processor -- start 2024-09-12T12:15:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 -- start start 2024-09-12T12:15:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed3ffff640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed400a2da0 0x7fed400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af69b0 0x5613a6af6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b084b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5613a6a5b2f0 con 0x5613a6af72d0 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613a6a03270 con 0x5613a6b060c0 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed604a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613a6a572b0 con 0x5613a6af69b0 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed3ffff640 1 -- 172.21.15.137:0/501708365 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fed500f0f00 con 0x7fed581069d0 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed567fc640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed400a2da0 0x7fed400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b05b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49496/0 (socket says 172.21.15.137:49496) 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed567fc640 1 -- 172.21.15.137:0/678706891 learned_addr learned my addr 172.21.15.137:0/678706891 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed577fe640 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.099+0000 7fed56ffd640 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af69b0 0x5613a6af6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed567fc640 1 -- 172.21.15.137:0/678706891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af69b0 msgr2=0x5613a6af6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed567fc640 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af69b0 0x5613a6af6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed567fc640 1 -- 172.21.15.137:0/678706891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 msgr2=0x5613a6b084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed567fc640 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed567fc640 1 -- 172.21.15.137:0/678706891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613a6a02340 con 0x5613a6af72d0 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed56ffd640 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af69b0 0x5613a6af6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed567fc640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed400a2da0 0x7fed400a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0049b0 tx=0x7fed4c009f10 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed567fc640 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b05b80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fed4c010fd0 tx=0x7fed4c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732f745b94443358 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed3d7fa640 1 -- 172.21.15.137:0/678706891 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c024070 con 0x5613a6af72d0 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed3d7fa640 1 -- 172.21.15.137:0/678706891 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c00dbe0 con 0x5613a6af72d0 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.100+0000 7fed3d7fa640 1 -- 172.21.15.137:0/678706891 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c005720 con 0x5613a6af72d0 2024-09-12T12:15:14.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- 172.21.15.137:0/678706891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 msgr2=0x5613a6b05b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b05b80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fed4c010fd0 tx=0x7fed4c01c000 comp rx=0 tx=0).stop 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- 172.21.15.137:0/678706891 shutdown_connections 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b05b80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 --2- 172.21.15.137:0/678706891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af69b0 0x5613a6af6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- 172.21.15.137:0/678706891 >> 172.21.15.137:0/678706891 conn(0x5613a6af3060 msgr2=0x5613a6af3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- 172.21.15.137:0/678706891 shutdown_connections 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- 172.21.15.137:0/678706891 wait complete. 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- start start 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 Processor -- start 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6af69b0 0x5613a6b030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b035f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6b060c0 0x5613a6b03b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5613a6b08c30 con 0x5613a6af72d0 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613a69e4290 con 0x5613a6af69b0 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.101+0000 7fed604a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613a6b08db0 con 0x5613a6b060c0 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed604a9740 10 client.0 _ll_register_callbacks cb 0x5613a6a15140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49502/0 (socket says 172.21.15.137:49502) 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 -- 172.21.15.137:0/1157136464 learned_addr learned my addr 172.21.15.137:0/1157136464 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed56ffd640 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6af69b0 0x5613a6b030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed577fe640 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6b060c0 0x5613a6b03b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 -- 172.21.15.137:0/1157136464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6b060c0 msgr2=0x5613a6b03b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6b060c0 0x5613a6b03b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 -- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6af69b0 msgr2=0x5613a6b030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6af69b0 0x5613a6b030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 -- 172.21.15.137:0/1157136464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613a676e210 con 0x5613a6af72d0 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed567fc640 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b035f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005320 tx=0x7fed4c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819781fa7d511262 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed3aff5640 1 -- 172.21.15.137:0/1157136464 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c02e070 con 0x5613a6af72d0 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed4c00dbe0 con 0x5613a6af72d0 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed3aff5640 1 -- 172.21.15.137:0/1157136464 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fed4c0049e0 con 0x5613a6af72d0 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed3aff5640 1 -- 172.21.15.137:0/1157136464 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed4c039050 con 0x5613a6af72d0 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed604a9740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.102+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613a683a620 con 0x5613a6af72d0 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed3aff5640 1 -- 172.21.15.137:0/1157136464 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed4c00dbe0 con 0x5613a6af72d0 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed3aff5640 1 -- 172.21.15.137:0/1157136464 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed4c01f070 con 0x5613a6af72d0 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed604a9740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed604a9740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed604a9740 20 client.15597 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed604a9740 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b16000 0x5613a6b183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed604a9740 4 client.15597 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613a6a68a20 con 0x5613a6b16000 2024-09-12T12:15:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.103+0000 7fed56ffd640 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b16000 0x5613a6b183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.104+0000 7fed3aff5640 10 client.15597 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.104+0000 7fed56ffd640 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b16000 0x5613a6b183f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601ac9937ce62850 server_cookie=44015a7159ad4404 in_seq=0 out_seq=0 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.104+0000 7fed3aff5640 1 -- 172.21.15.137:0/1157136464 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613a6a68a20 con 0x5613a6b16000 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.104+0000 7fed3aff5640 10 client.15597 handle_command_reply: tid=0 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.104+0000 7fed604a9740 1 client.15597 shutdown 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 msgr2=0x5613a6b035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b035f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005320 tx=0x7fed4c005a20 comp rx=0 tx=0).stop 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 shutdown_connections 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6b060c0 0x5613a6b03b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b16000 0x5613a6b183f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af72d0 0x5613a6b035f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 --2- 172.21.15.137:0/1157136464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6af69b0 0x5613a6b030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed3aff5640 0 client.15597 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 >> 172.21.15.137:0/1157136464 conn(0x5613a6af3060 msgr2=0x5613a6af8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 shutdown_connections 2024-09-12T12:15:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 1 -- 172.21.15.137:0/1157136464 wait complete. 2024-09-12T12:15:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.105+0000 7fed604a9740 20 client.15597 trim_cache size 0 max 16384 2024-09-12T12:15:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 Processor -- start 2024-09-12T12:15:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 -- start start 2024-09-12T12:15:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b1dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b1e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed581adc60 con 0x5613a6af69b0 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed581ab4d0 con 0x5613a6b060c0 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed604a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed581a9340 con 0x5613a6af72d0 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed56ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b1dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed56ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b1dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49506/0 (socket says 172.21.15.137:49506) 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed56ffd640 1 -- 172.21.15.137:0/80665023 learned_addr learned my addr 172.21.15.137:0/80665023 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.119+0000 7fed577fe640 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed567fc640 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b1e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed56ffd640 1 -- 172.21.15.137:0/80665023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 msgr2=0x5613a6b1e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed56ffd640 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b1e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed56ffd640 1 -- 172.21.15.137:0/80665023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 msgr2=0x5613a6b002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed56ffd640 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed56ffd640 1 -- 172.21.15.137:0/80665023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613a6b02040 con 0x5613a6af69b0 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed577fe640 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed56ffd640 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b1dc00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5613a6b18b40 tx=0x7fed480128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd22cd639b2cac2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed3cff9640 1 -- 172.21.15.137:0/80665023 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48023030 con 0x5613a6af69b0 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed3cff9640 1 -- 172.21.15.137:0/80665023 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed48005500 con 0x5613a6af69b0 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed3cff9640 1 -- 172.21.15.137:0/80665023 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4801e750 con 0x5613a6af69b0 2024-09-12T12:15:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed604a9740 1 -- 172.21.15.137:0/80665023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 msgr2=0x5613a6b1dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.120+0000 7fed604a9740 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b1dc00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5613a6b18b40 tx=0x7fed480128e0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- 172.21.15.137:0/80665023 shutdown_connections 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b1e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 --2- 172.21.15.137:0/80665023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b1dc00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- 172.21.15.137:0/80665023 >> 172.21.15.137:0/80665023 conn(0x5613a6af3060 msgr2=0x5613a6b03890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- 172.21.15.137:0/80665023 shutdown_connections 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- 172.21.15.137:0/80665023 wait complete. 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- start start 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 Processor -- start 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b0a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed581a9990 con 0x5613a6af69b0 2024-09-12T12:15:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed581aae00 con 0x5613a6b060c0 2024-09-12T12:15:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed581aa060 con 0x5613a6af72d0 2024-09-12T12:15:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed604a9740 10 client.0 _ll_register_callbacks cb 0x7fed50005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed56ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed56ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b09e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49508/0 (socket says 172.21.15.137:49508) 2024-09-12T12:15:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed56ffd640 1 -- 172.21.15.137:0/2014725028 learned_addr learned my addr 172.21.15.137:0/2014725028 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed567fc640 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b0a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.121+0000 7fed577fe640 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed56ffd640 1 -- 172.21.15.137:0/2014725028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 msgr2=0x5613a6b0a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed56ffd640 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b0a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed56ffd640 1 -- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 msgr2=0x5613a6b0a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed56ffd640 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b0a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed56ffd640 1 -- 172.21.15.137:0/2014725028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613a6b18760 con 0x5613a6af69b0 2024-09-12T12:15:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed56ffd640 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b09e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fed48009b70 tx=0x7fed48004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11790fd4001862db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed3aff5640 1 -- 172.21.15.137:0/2014725028 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4802b070 con 0x5613a6af69b0 2024-09-12T12:15:14.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed3aff5640 1 -- 172.21.15.137:0/2014725028 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fed480048c0 con 0x5613a6af69b0 2024-09-12T12:15:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed48005500 con 0x5613a6af69b0 2024-09-12T12:15:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed3aff5640 1 -- 172.21.15.137:0/2014725028 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fed48037050 con 0x5613a6af69b0 2024-09-12T12:15:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 10 client.15603 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5613a6b80d80 con 0x5613a6af69b0 2024-09-12T12:15:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed3aff5640 1 -- 172.21.15.137:0/2014725028 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed48005500 con 0x5613a6af69b0 2024-09-12T12:15:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed3aff5640 1 -- 172.21.15.137:0/2014725028 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fed48011560 con 0x5613a6af69b0 2024-09-12T12:15:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 20 client.15603 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b81470 0x5613a6b83890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 4 client.15603 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:15:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.122+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613a6a62d70 con 0x5613a6b81470 2024-09-12T12:15:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.123+0000 7fed567fc640 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b81470 0x5613a6b83890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.123+0000 7fed567fc640 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b81470 0x5613a6b83890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c149682a6faf683 server_cookie=c982890f8be1ec2b in_seq=0 out_seq=0 2024-09-12T12:15:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.123+0000 7fed3aff5640 10 client.15603 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:14.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.157+0000 7fed3aff5640 1 -- 172.21.15.137:0/2014725028 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613a6a62d70 con 0x5613a6b81470 2024-09-12T12:15:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.157+0000 7fed3aff5640 10 client.15603 handle_command_reply: tid=0 2024-09-12T12:15:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.157+0000 7fed604a9740 1 client.15603 shutdown 2024-09-12T12:15:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.157+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 msgr2=0x5613a6b09e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.157+0000 7fed604a9740 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b09e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fed48009b70 tx=0x7fed48004f60 comp rx=0 tx=0).stop 2024-09-12T12:15:14.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 shutdown_connections 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613a6b060c0 0x5613a6b0a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613a6af72d0 0x5613a6b0a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613a6b81470 0x5613a6b83890 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 --2- 172.21.15.137:0/2014725028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613a6af69b0 0x5613a6b09e70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed3aff5640 0 client.15603 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 >> 172.21.15.137:0/2014725028 conn(0x5613a6af3060 msgr2=0x5613a6af84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 shutdown_connections 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 1 -- 172.21.15.137:0/2014725028 wait complete. 2024-09-12T12:15:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed604a9740 20 client.15603 trim_cache size 0 max 16384 2024-09-12T12:15:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:15:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:15:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:15:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:15:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed400a2da0 msgr2=0x7fed400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed400a2da0 0x7fed400a5260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0049b0 tx=0x7fed4c009f10 comp rx=0 tx=0).stop 2024-09-12T12:15:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed581069d0 msgr2=0x7fed5819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed581069d0 0x7fed5819bb00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fed50005ab0 tx=0x7fed50004040 comp rx=0 tx=0).stop 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 shutdown_connections 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed400a2da0 0x7fed400a5260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed581069d0 0x7fed5819bb00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed58101360 0x7fed58197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 --2- 172.21.15.137:0/501708365 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed581009c0 0x7fed58197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.158+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 >> 172.21.15.137:0/501708365 conn(0x7fed580fc700 msgr2=0x7fed580fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.159+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 shutdown_connections 2024-09-12T12:15:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:14.159+0000 7fed5d711640 1 -- 172.21.15.137:0/501708365 wait complete. 2024-09-12T12:15:14.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 12.50 MB/sec warmup 49 sec latency 9058.041 ms 2024-09-12T12:15:15.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:14 smithi171 ceph-mon[35098]: pgmap v64: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 958 KiB/s wr, 221 op/s 2024-09-12T12:15:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:14 smithi137 ceph-mon[30950]: pgmap v64: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 958 KiB/s wr, 221 op/s 2024-09-12T12:15:15.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:14 smithi189 ceph-mon[35475]: pgmap v64: 97 pgs: 97 active+clean; 38 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 958 KiB/s wr, 221 op/s 2024-09-12T12:15:15.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 12.25 MB/sec warmup 50 sec latency 10058.140 ms 2024-09-12T12:15:16.292 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:15:16.552 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:15:16.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 12.01 MB/sec warmup 51 sec latency 11058.172 ms 2024-09-12T12:15:16.995 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:16 smithi137 ceph-mon[30950]: pgmap v65: 97 pgs: 97 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 249 op/s 2024-09-12T12:15:17.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 -- 172.21.15.137:0/1929520640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68100a70 msgr2=0x7fba68100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/1929520640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68100a70 0x7fba68100ed0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009930 tx=0x7fba5c02a420 comp rx=0 tx=0).stop 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 -- 172.21.15.137:0/1929520640 shutdown_connections 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/1929520640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba68101410 0x7fba6810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/1929520640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68100a70 0x7fba68100ed0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/1929520640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68106a70 0x7fba68106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 -- 172.21.15.137:0/1929520640 >> 172.21.15.137:0/1929520640 conn(0x7fba680fc790 msgr2=0x7fba680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 -- 172.21.15.137:0/1929520640 shutdown_connections 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 -- 172.21.15.137:0/1929520640 wait complete. 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 Processor -- start 2024-09-12T12:15:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.075+0000 7fba6d3a8640 1 -- start start 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba6d3a8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68100a70 0x7fba68197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba6d3a8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba68101410 0x7fba68197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba6d3a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68106a70 0x7fba68197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba6d3a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fba68110f10 con 0x7fba68106a70 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba6d3a8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba68110d90 con 0x7fba68101410 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba6d3a8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba68111090 con 0x7fba68100a70 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68106a70 0x7fba68197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68106a70 0x7fba68197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49532/0 (socket says 172.21.15.137:49532) 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 -- 172.21.15.137:0/2819889770 learned_addr learned my addr 172.21.15.137:0/2819889770 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba667fc640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba68101410 0x7fba68197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:17.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba66ffd640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68100a70 0x7fba68197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 -- 172.21.15.137:0/2819889770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68100a70 msgr2=0x7fba68197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68100a70 0x7fba68197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 -- 172.21.15.137:0/2819889770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba68101410 msgr2=0x7fba68197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba68101410 0x7fba68197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 -- 172.21.15.137:0/2819889770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba681a6ac0 con 0x7fba68106a70 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba66ffd640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68100a70 0x7fba68197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba677fe640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68106a70 0x7fba68197cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba60009fd0 tx=0x7fba6000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba4ffff640 1 -- 172.21.15.137:0/2819889770 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60011830 con 0x7fba68106a70 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba681a6db0 con 0x7fba68106a70 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba4ffff640 1 -- 172.21.15.137:0/2819889770 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba60016070 con 0x7fba68106a70 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.076+0000 7fba4ffff640 1 -- 172.21.15.137:0/2819889770 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6000f2e0 con 0x7fba68106a70 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.077+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba681a7270 con 0x7fba68106a70 2024-09-12T12:15:17.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.077+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba68100f50 con 0x7fba68106a70 2024-09-12T12:15:17.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.079+0000 7fba4ffff640 1 -- 172.21.15.137:0/2819889770 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fba600119d0 con 0x7fba68106a70 2024-09-12T12:15:17.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.079+0000 7fba4ffff640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fba500a2cf0 0x7fba500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:17.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.079+0000 7fba66ffd640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fba500a2cf0 0x7fba500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:17.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.079+0000 7fba4ffff640 1 -- 172.21.15.137:0/2819889770 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fba600cf0e0 con 0x7fba68106a70 2024-09-12T12:15:17.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.080+0000 7fba66ffd640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fba500a2cf0 0x7fba500a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fba58007eb0 tx=0x7fba580073c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:17.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.083+0000 7fba4ffff640 1 -- 172.21.15.137:0/2819889770 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba6009be60 con 0x7fba68106a70 2024-09-12T12:15:17.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:16 smithi171 ceph-mon[35098]: pgmap v65: 97 pgs: 97 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 249 op/s 2024-09-12T12:15:17.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.285+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba6802d7c0 con 0x7fba68106a70 2024-09-12T12:15:17.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.286+0000 7fba4ffff640 1 -- 172.21.15.137:0/2819889770 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7fba6009b760 con 0x7fba68106a70 2024-09-12T12:15:17.287 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:15:17.287 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":27,"btime":"2024-09-12T12:14:37:790728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:14:36.964904+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:15:17.287 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 27 2024-09-12T12:15:17.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.288+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fba500a2cf0 msgr2=0x7fba500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:17.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.288+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fba500a2cf0 0x7fba500a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fba58007eb0 tx=0x7fba580073c0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.288+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68106a70 msgr2=0x7fba68197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:17.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.288+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68106a70 0x7fba68197cd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fba60009fd0 tx=0x7fba6000b8e0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 shutdown_connections 2024-09-12T12:15:17.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fba500a2cf0 0x7fba500a51b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fba68106a70 0x7fba68197cd0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba68101410 0x7fba68197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 --2- 172.21.15.137:0/2819889770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68100a70 0x7fba68197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:17.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 >> 172.21.15.137:0/2819889770 conn(0x7fba680fc790 msgr2=0x7fba680fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:17.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 shutdown_connections 2024-09-12T12:15:17.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:17.289+0000 7fba6d3a8640 1 -- 172.21.15.137:0/2819889770 wait complete. 2024-09-12T12:15:17.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:16 smithi189 ceph-mon[35475]: pgmap v65: 97 pgs: 97 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 249 op/s 2024-09-12T12:15:17.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 11.78 MB/sec warmup 52 sec latency 12058.207 ms 2024-09-12T12:15:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:17 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2819889770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:18.135 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 4b2e6b88 --release --await' 2024-09-12T12:15:18.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 4b2e6b88 --release --await 2024-09-12T12:15:18.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:17 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2819889770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:18.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:17 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2819889770' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:18.389 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:15:18.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 110081 11.55 MB/sec warmup 53 sec latency 13058.233 ms 2024-09-12T12:15:18.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 -- 172.21.15.137:0/2935486077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4102d90 msgr2=0x7f4cb4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:18.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 --2- 172.21.15.137:0/2935486077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4102d90 0x7f4cb4103170 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4009930 tx=0x7f4ca402a420 comp rx=0 tx=0).stop 2024-09-12T12:15:18.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 -- 172.21.15.137:0/2935486077 shutdown_connections 2024-09-12T12:15:18.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 --2- 172.21.15.137:0/2935486077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cb4108290 0x7f4cb410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 --2- 172.21.15.137:0/2935486077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cb4103740 0x7f4cb4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 --2- 172.21.15.137:0/2935486077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4102d90 0x7f4cb4103170 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 -- 172.21.15.137:0/2935486077 >> 172.21.15.137:0/2935486077 conn(0x7f4cb40fc7d0 msgr2=0x7f4cb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:18.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 -- 172.21.15.137:0/2935486077 shutdown_connections 2024-09-12T12:15:18.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 -- 172.21.15.137:0/2935486077 wait complete. 2024-09-12T12:15:18.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 Processor -- start 2024-09-12T12:15:18.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.934+0000 7f4cba585640 1 -- start start 2024-09-12T12:15:18.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cba585640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cb4102d90 0x7f4cb4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cba585640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cb4103740 0x7f4cb4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cba585640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4108290 0x7f4cb419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cba585640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4112090 con 0x7f4cb4108290 2024-09-12T12:15:18.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cba585640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4111f10 con 0x7f4cb4102d90 2024-09-12T12:15:18.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cba585640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4112210 con 0x7f4cb4103740 2024-09-12T12:15:18.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb8b22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4108290 0x7f4cb419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb3fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cb4102d90 0x7f4cb4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb8b22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4108290 0x7f4cb419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49550/0 (socket says 172.21.15.137:49550) 2024-09-12T12:15:18.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb8b22640 1 -- 172.21.15.137:0/1452857052 learned_addr learned my addr 172.21.15.137:0/1452857052 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:18.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cb4103740 0x7f4cb4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb3fff640 1 -- 172.21.15.137:0/1452857052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cb4103740 msgr2=0x7f4cb4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:18.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb3fff640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cb4103740 0x7f4cb4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb3fff640 1 -- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4108290 msgr2=0x7f4cb419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:18.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb3fff640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4108290 0x7f4cb419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb3fff640 1 -- 172.21.15.137:0/1452857052 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb419d540 con 0x7f4cb4102d90 2024-09-12T12:15:18.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4108290 0x7f4cb419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:18.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.935+0000 7f4cb3fff640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cb4102d90 0x7f4cb4198590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4005940 tx=0x7f4ca4031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:18.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.936+0000 7f4cb17fa640 1 -- 172.21.15.137:0/1452857052 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca403f070 con 0x7f4cb4102d90 2024-09-12T12:15:18.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.936+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb419d7d0 con 0x7f4cb4102d90 2024-09-12T12:15:18.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.936+0000 7f4cb17fa640 1 -- 172.21.15.137:0/1452857052 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ca4003ad0 con 0x7f4cb4102d90 2024-09-12T12:15:18.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.936+0000 7f4cb17fa640 1 -- 172.21.15.137:0/1452857052 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4035070 con 0x7f4cb4102d90 2024-09-12T12:15:18.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.936+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb41a5110 con 0x7f4cb4102d90 2024-09-12T12:15:18.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.938+0000 7f4cbc95e740 1 Processor -- start 2024-09-12T12:15:18.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cbc95e740 1 -- start start 2024-09-12T12:15:18.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457d9d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457daa630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f457d02620 con 0x55f457da8240 2024-09-12T12:15:18.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f457ca9e50 con 0x55f457d9d060 2024-09-12T12:15:18.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f457cfe5e0 con 0x55f457d9d980 2024-09-12T12:15:18.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb17fa640 1 -- 172.21.15.137:0/1452857052 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4ca40035f0 con 0x7f4cb4102d90 2024-09-12T12:15:18.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457d9d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457daa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457d9d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:42330/0 (socket says 172.21.15.137:42330) 2024-09-12T12:15:18.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 -- 172.21.15.137:0/2643355699 learned_addr learned my addr 172.21.15.137:0/2643355699 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:18.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb3fff640 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 -- 172.21.15.137:0/2643355699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 msgr2=0x55f457da7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 -- 172.21.15.137:0/2643355699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 msgr2=0x55f457daa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457daa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb8b22640 1 -- 172.21.15.137:0/2643355699 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f457c9ad40 con 0x55f457d9d060 2024-09-12T12:15:18.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.939+0000 7f4cb3fff640 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:15:18.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.940+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457d9d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4cac011850 tx=0x7f4cac011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e0b454ad3e07ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:18.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.940+0000 7f4cb17fa640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4ca00a2cf0 0x7f4ca00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.940+0000 7f4caa7fc640 1 -- 172.21.15.137:0/2643355699 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac011330 con 0x55f457d9d060 2024-09-12T12:15:18.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.940+0000 7f4caa7fc640 1 -- 172.21.15.137:0/2643355699 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cac00fe80 con 0x55f457d9d060 2024-09-12T12:15:18.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.940+0000 7f4caa7fc640 1 -- 172.21.15.137:0/2643355699 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0193e0 con 0x55f457d9d060 2024-09-12T12:15:18.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.940+0000 7f4cb3fff640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4ca00a2cf0 0x7f4ca00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.940+0000 7f4cb17fa640 1 -- 172.21.15.137:0/1452857052 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f4ca40ef860 con 0x7f4cb4102d90 2024-09-12T12:15:18.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 -- 172.21.15.137:0/2643355699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 msgr2=0x55f457d9d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:18.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cb3fff640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4ca00a2cf0 0x7f4ca00a51b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40ed0b0 tx=0x7f4ca402a420 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:18.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457d9d440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4cac011850 tx=0x7f4cac011d10 comp rx=0 tx=0).stop 2024-09-12T12:15:18.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 -- 172.21.15.137:0/2643355699 shutdown_connections 2024-09-12T12:15:18.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457daa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/2643355699 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457d9d440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 -- 172.21.15.137:0/2643355699 >> 172.21.15.137:0/2643355699 conn(0x55f457d995f0 msgr2=0x55f457d999d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:18.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 -- 172.21.15.137:0/2643355699 shutdown_connections 2024-09-12T12:15:18.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 -- 172.21.15.137:0/2643355699 wait complete. 2024-09-12T12:15:18.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 -- start start 2024-09-12T12:15:18.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.941+0000 7f4cbc95e740 1 Processor -- start 2024-09-12T12:15:18.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d060 0x55f457da5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457da8240 0x55f457da5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f457daae10 con 0x55f457d9d060 2024-09-12T12:15:18.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f457c8af10 con 0x55f457da8240 2024-09-12T12:15:18.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f457daaf90 con 0x55f457d9d980 2024-09-12T12:15:18.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 10 client.0 _ll_register_callbacks cb 0x55f457caedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:18.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb8b22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d060 0x55f457da5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb8b22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d060 0x55f457da5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49572/0 (socket says 172.21.15.137:49572) 2024-09-12T12:15:18.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb8b22640 1 -- 172.21.15.137:0/1281083480 learned_addr learned my addr 172.21.15.137:0/1281083480 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:18.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457da8240 0x55f457da5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb37fe640 1 -- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457da8240 msgr2=0x55f457da5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457da8240 0x55f457da5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb37fe640 1 -- 172.21.15.137:0/1281083480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d060 msgr2=0x55f457da5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d060 0x55f457da5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb37fe640 1 -- 172.21.15.137:0/1281083480 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f457a11580 con 0x55f457d9d980 2024-09-12T12:15:18.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d060 0x55f457da5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da5560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40fedf0 tx=0x7f4c9c009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc8f0a44339dda80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4c937fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4c937fe640 1 -- 172.21.15.137:0/1281083480 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c021020 con 0x55f457d9d980 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f457d0fb90 con 0x55f457d9d980 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.942+0000 7f4c937fe640 1 -- 172.21.15.137:0/1281083480 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f4c9c01d400 con 0x55f457d9d980 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4c937fe640 1 -- 172.21.15.137:0/1281083480 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c9c02c050 con 0x55f457d9d980 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 10 client.25174 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f457adb960 con 0x55f457d9d980 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4c937fe640 1 -- 172.21.15.137:0/1281083480 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f457d0fb90 con 0x55f457d9d980 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4c937fe640 1 -- 172.21.15.137:0/1281083480 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4c9c025530 con 0x55f457d9d980 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 10 client.25174 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 10 client.25174 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:15:18.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 20 client.25174 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457db7c30 0x55f457dba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 4 client.25174 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f457d0a0a0 con 0x55f457db7c30 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.943+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457db7c30 0x55f457dba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.944+0000 7f4c937fe640 10 client.25174 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.944+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457db7c30 0x55f457dba0b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ac9968b8677559 server_cookie=9aa87ed8eda1eb8 in_seq=0 out_seq=0 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.944+0000 7f4c937fe640 1 -- 172.21.15.137:0/1281083480 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f457d0a0a0 con 0x55f457db7c30 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.944+0000 7f4c937fe640 10 client.25174 handle_command_reply: tid=0 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.944+0000 7f4cbc95e740 1 client.25174 shutdown 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 msgr2=0x55f457da5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da5560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40fedf0 tx=0x7f4c9c009eb0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 shutdown_connections 2024-09-12T12:15:18.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457da8240 0x55f457da5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d980 0x55f457da5560 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457db7c30 0x55f457dba0b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1281083480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d060 0x55f457da5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 >> 172.21.15.137:0/1281083480 conn(0x55f457d995f0 msgr2=0x55f457d98750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:18.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4c937fe640 0 client.25174 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:18.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 shutdown_connections 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1281083480 wait complete. 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.945+0000 7f4cbc95e740 20 client.25174 trim_cache size 0 max 16384 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.961+0000 7f4cbc95e740 1 Processor -- start 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cbc95e740 1 -- start start 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457dbfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d980 0x55f457dbff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457da8240 0x55f457da22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41adea0 con 0x55f457d9d980 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41ab710 con 0x55f457d9d060 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41a9560 con 0x55f457da8240 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d980 0x55f457dbff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d980 0x55f457dbff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49582/0 (socket says 172.21.15.137:49582) 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 -- 172.21.15.137:0/1005953207 learned_addr learned my addr 172.21.15.137:0/1005953207 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457dbfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457da8240 0x55f457da22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 -- 172.21.15.137:0/1005953207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457da8240 msgr2=0x55f457da22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457da8240 0x55f457da22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 -- 172.21.15.137:0/1005953207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 msgr2=0x55f457dbfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457dbfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.962+0000 7f4cb37fe640 1 -- 172.21.15.137:0/1005953207 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f457dba730 con 0x55f457d9d980 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.963+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d980 0x55f457dbff50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c025b90 tx=0x7f4c9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8bde6c294352f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.963+0000 7f4ca9ffb640 1 -- 172.21.15.137:0/1005953207 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c035070 con 0x55f457d9d980 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.963+0000 7f4ca9ffb640 1 -- 172.21.15.137:0/1005953207 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9c025e10 con 0x55f457d9d980 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.963+0000 7f4ca9ffb640 1 -- 172.21.15.137:0/1005953207 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c021070 con 0x55f457d9d980 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.963+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1005953207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d980 msgr2=0x55f457dbff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d980 0x55f457dbff50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c025b90 tx=0x7f4c9c00ee80 comp rx=0 tx=0).stop 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1005953207 shutdown_connections 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457da8240 0x55f457da22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457d9d980 0x55f457dbff50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/1005953207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d060 0x55f457dbfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1005953207 >> 172.21.15.137:0/1005953207 conn(0x55f457d995f0 msgr2=0x55f457da6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:18.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1005953207 shutdown_connections 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 -- 172.21.15.137:0/1005953207 wait complete. 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 -- start start 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 Processor -- start 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.964+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d060 0x55f457db03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d980 0x55f457db0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cbc95e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457da1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41a9bd0 con 0x55f457da8240 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41ab040 con 0x55f457d9d980 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cbc95e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41aa2a0 con 0x55f457d9d060 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cbc95e740 10 client.0 _ll_register_callbacks cb 0x7f4c9c013db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d980 0x55f457db0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457da1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb37fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d980 0x55f457db0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:42362/0 (socket says 172.21.15.137:42362) 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb37fe640 1 -- 172.21.15.137:0/4281583178 learned_addr learned my addr 172.21.15.137:0/4281583178 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d060 0x55f457db03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 -- 172.21.15.137:0/4281583178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d060 msgr2=0x55f457db03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d060 0x55f457db03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 -- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d980 msgr2=0x55f457db0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d980 0x55f457db0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 -- 172.21.15.137:0/4281583178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f457d99420 con 0x55f457da8240 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d980 0x55f457db0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cb8b22640 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457da1290 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55f457dba9b0 tx=0x7f4cac011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e8dca5563be025 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4c937fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4c937fe640 1 -- 172.21.15.137:0/4281583178 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac01f040 con 0x55f457da8240 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4c937fe640 1 -- 172.21.15.137:0/4281583178 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f4cac004bd0 con 0x55f457da8240 2024-09-12T12:15:18.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.965+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c9c025e10 con 0x55f457da8240 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4c937fe640 1 -- 172.21.15.137:0/4281583178 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4cac03b020 con 0x55f457da8240 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f457da2ac0 con 0x55f457da8240 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4c937fe640 1 -- 172.21.15.137:0/4281583178 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9c025e10 con 0x55f457da8240 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4c937fe640 1 -- 172.21.15.137:0/4281583178 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f4cac018070 con 0x55f457da8240 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 10 client.15627 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 20 client.15627 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457e239f0 0x55f457e25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 4 client.15627 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4b2e6b88", "release": true, "await": true}] 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4b2e6b88", "release": true, "await": true}) -- 0x55f457c865a0 con 0x55f457e239f0 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.966+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457e239f0 0x55f457e25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.967+0000 7f4c937fe640 10 client.15627 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:18.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.967+0000 7f4cb37fe640 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457e239f0 0x55f457e25de0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74eea57a6bab4519 server_cookie=91788231b7775791 in_seq=0 out_seq=0 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4c937fe640 1 -- 172.21.15.137:0/4281583178 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f457c865a0 con 0x55f457e239f0 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4c937fe640 10 client.15627 handle_command_reply: tid=0 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 client.15627 shutdown 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 msgr2=0x55f457da1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457da1290 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55f457dba9b0 tx=0x7f4cac011d10 comp rx=0 tx=0).stop 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 shutdown_connections 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f457e239f0 0x55f457e25de0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f457da8240 0x55f457da1290 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4c937fe640 0 client.15627 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f457d9d980 0x55f457db0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 --2- 172.21.15.137:0/4281583178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f457d9d060 0x55f457db03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 >> 172.21.15.137:0/4281583178 conn(0x55f457d995f0 msgr2=0x55f457da4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:18.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 shutdown_connections 2024-09-12T12:15:18.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.974+0000 7f4cbc95e740 1 -- 172.21.15.137:0/4281583178 wait complete. 2024-09-12T12:15:18.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cbc95e740 20 client.15627 trim_cache size 0 max 16384 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 27, 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 8, 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "4b2e6b88": { 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 8, 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:15:18.999 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 133.4, 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:19.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:19.001 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:15:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4ca00a2cf0 msgr2=0x7f4ca00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4ca00a2cf0 0x7f4ca00a51b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40ed0b0 tx=0x7f4ca402a420 comp rx=0 tx=0).stop 2024-09-12T12:15:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cb4102d90 msgr2=0x7f4cb4198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cb4102d90 0x7f4cb4198590 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4005940 tx=0x7f4ca4031b50 comp rx=0 tx=0).stop 2024-09-12T12:15:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 shutdown_connections 2024-09-12T12:15:19.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4ca00a2cf0 0x7f4ca00a51b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:19.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cb4108290 0x7f4cb419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:19.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cb4103740 0x7f4cb4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:19.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 --2- 172.21.15.137:0/1452857052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cb4102d90 0x7f4cb4198590 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:19.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 >> 172.21.15.137:0/1452857052 conn(0x7f4cb40fc7d0 msgr2=0x7f4cb4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:19.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 shutdown_connections 2024-09-12T12:15:19.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:18.975+0000 7f4cba585640 1 -- 172.21.15.137:0/1452857052 wait complete. 2024-09-12T12:15:19.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:18 smithi171 ceph-mon[35098]: pgmap v66: 97 pgs: 97 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 428 KiB/s wr, 65 op/s 2024-09-12T12:15:19.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:18 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:15:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:18 smithi137 ceph-mon[30950]: pgmap v66: 97 pgs: 97 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 428 KiB/s wr, 65 op/s 2024-09-12T12:15:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:18 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:15:19.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:18 smithi189 ceph-mon[35475]: pgmap v66: 97 pgs: 97 active+clean; 40 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 428 KiB/s wr, 65 op/s 2024-09-12T12:15:19.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:18 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:15:19.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 111170 11.40 MB/sec warmup 54 sec latency 13316.086 ms 2024-09-12T12:15:19.866 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4b2e6b88', total seconds elapsed: 17.0 2024-09-12T12:15:20.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:15:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:15:20.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:15:20.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:15:20.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:15:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:15:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:15:20.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 113943 11.51 MB/sec warmup 55 sec latency 9.916 ms 2024-09-12T12:15:21.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:21 smithi171 ceph-mon[35098]: pgmap v67: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 465 KiB/s wr, 77 op/s 2024-09-12T12:15:21.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:21 smithi137 ceph-mon[30950]: pgmap v67: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 465 KiB/s wr, 77 op/s 2024-09-12T12:15:21.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:21 smithi189 ceph-mon[35475]: pgmap v67: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 465 KiB/s wr, 77 op/s 2024-09-12T12:15:21.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 116711 11.54 MB/sec warmup 56 sec latency 13.989 ms 2024-09-12T12:15:22.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 118500 11.53 MB/sec warmup 57 sec latency 13.342 ms 2024-09-12T12:15:23.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:23 smithi171 ceph-mon[35098]: pgmap v68: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 260 KiB/s wr, 40 op/s 2024-09-12T12:15:23.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:23 smithi137 ceph-mon[30950]: pgmap v68: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 260 KiB/s wr, 40 op/s 2024-09-12T12:15:23.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:23 smithi189 ceph-mon[35475]: pgmap v68: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 260 KiB/s wr, 40 op/s 2024-09-12T12:15:23.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 121025 11.57 MB/sec warmup 58 sec latency 21.382 ms 2024-09-12T12:15:24.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 123564 11.59 MB/sec warmup 59 sec latency 15.946 ms 2024-09-12T12:15:25.392 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:25 smithi189 ceph-mon[35475]: pgmap v69: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 260 KiB/s wr, 40 op/s 2024-09-12T12:15:25.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:25 smithi171 ceph-mon[35098]: pgmap v69: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 260 KiB/s wr, 40 op/s 2024-09-12T12:15:25.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:25 smithi137 ceph-mon[30950]: pgmap v69: 97 pgs: 97 active+clean; 41 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 260 KiB/s wr, 40 op/s 2024-09-12T12:15:25.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 125617 11.59 MB/sec warmup 60 sec latency 12.101 ms 2024-09-12T12:15:26.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 128177 11.63 MB/sec warmup 61 sec latency 9.992 ms 2024-09-12T12:15:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 -- 172.21.15.137:0/2970010773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 msgr2=0x7f9530100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 --2- 172.21.15.137:0/2970010773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530100e10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f951c009930 tx=0x7f951c02a420 comp rx=0 tx=0).stop 2024-09-12T12:15:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 -- 172.21.15.137:0/2970010773 shutdown_connections 2024-09-12T12:15:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 --2- 172.21.15.137:0/2970010773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9530101350 0x7f953010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 --2- 172.21.15.137:0/2970010773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530100e10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 --2- 172.21.15.137:0/2970010773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95301069c0 0x7f9530106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 -- 172.21.15.137:0/2970010773 >> 172.21.15.137:0/2970010773 conn(0x7f9530077990 msgr2=0x7f95300fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 -- 172.21.15.137:0/2970010773 shutdown_connections 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.248+0000 7f9536d8d640 1 -- 172.21.15.137:0/2970010773 wait complete. 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 Processor -- start 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 -- start start 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9530101350 0x7f9530198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95301069c0 0x7f953019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9530110e00 con 0x7f95301009b0 2024-09-12T12:15:27.252 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9530110c80 con 0x7f95301069c0 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9536d8d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9530110f80 con 0x7f9530101350 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39982/0 (socket says 172.21.15.137:39982) 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 -- 172.21.15.137:0/1645289098 learned_addr learned my addr 172.21.15.137:0/1645289098 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f952ffff640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9530101350 0x7f9530198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f953532a640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95301069c0 0x7f953019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 -- 172.21.15.137:0/1645289098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9530101350 msgr2=0x7f9530198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9530101350 0x7f9530198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 -- 172.21.15.137:0/1645289098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95301069c0 msgr2=0x7f953019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95301069c0 0x7f953019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 -- 172.21.15.137:0/1645289098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953019d2f0 con 0x7f95301009b0 2024-09-12T12:15:27.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.249+0000 7f9534b29640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530198300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.250+0000 7f952dffb640 1 -- 172.21.15.137:0/1645289098 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952400cd80 con 0x7f95301009b0 2024-09-12T12:15:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.250+0000 7f952dffb640 1 -- 172.21.15.137:0/1645289098 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9524010910 con 0x7f95301009b0 2024-09-12T12:15:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.250+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953019d5e0 con 0x7f95301009b0 2024-09-12T12:15:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.250+0000 7f952dffb640 1 -- 172.21.15.137:0/1645289098 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524018990 con 0x7f95301009b0 2024-09-12T12:15:27.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.250+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95301a4e70 con 0x7f95301009b0 2024-09-12T12:15:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.252+0000 7f952dffb640 1 -- 172.21.15.137:0/1645289098 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9524010ac0 con 0x7f95301009b0 2024-09-12T12:15:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f952dffb640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f95200a2da0 0x7f95200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 Processor -- start 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f952dffb640 1 -- 172.21.15.137:0/1645289098 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f95240cf740 con 0x7f95301009b0 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f952ffff640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f95200a2da0 0x7f95200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 -- start start 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e799f0 0x55b4e3e79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3e88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e8b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b4e3dde2f0 con 0x55b4e3e7a310 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b4e3d86270 con 0x55b4e3e799f0 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.253+0000 7f9539c03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4e3dda2b0 con 0x55b4e3e890c0 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e799f0 0x55b4e3e79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e799f0 0x55b4e3e79dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:60640/0 (socket says 172.21.15.137:60640) 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 -- 172.21.15.137:0/1738003034 learned_addr learned my addr 172.21.15.137:0/1738003034 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:27.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f953532a640 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e8b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f952ffff640 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3e88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f952ffff640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f95200a2da0 0x7f95200a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f951c009900 tx=0x7f951c0058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 -- 172.21.15.137:0/1738003034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 msgr2=0x55b4e3e8b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e8b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 -- 172.21.15.137:0/1738003034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 msgr2=0x55b4e3e88b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3e88b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 -- 172.21.15.137:0/1738003034 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e3d85340 con 0x55b4e3e799f0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f953532a640 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e8b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f9534b29640 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e799f0 0x55b4e3e79dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9524098680 tx=0x7f952409c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43f4a099d319d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f951affd640 1 -- 172.21.15.137:0/1738003034 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240a4920 con 0x55b4e3e799f0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f951affd640 1 -- 172.21.15.137:0/1738003034 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9524099fb0 con 0x55b4e3e799f0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.254+0000 7f951affd640 1 -- 172.21.15.137:0/1738003034 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952409b100 con 0x55b4e3e799f0 2024-09-12T12:15:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 -- 172.21.15.137:0/1738003034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e799f0 msgr2=0x55b4e3e79dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e799f0 0x55b4e3e79dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9524098680 tx=0x7f952409c470 comp rx=0 tx=0).stop 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 -- 172.21.15.137:0/1738003034 shutdown_connections 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e8b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3e88b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 --2- 172.21.15.137:0/1738003034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e799f0 0x55b4e3e79dd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 -- 172.21.15.137:0/1738003034 >> 172.21.15.137:0/1738003034 conn(0x55b4e3e76060 msgr2=0x55b4e3e76440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 -- 172.21.15.137:0/1738003034 shutdown_connections 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 -- 172.21.15.137:0/1738003034 wait complete. 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.255+0000 7f9539c03740 1 -- start start 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 1 Processor -- start 2024-09-12T12:15:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e85f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e7a310 0x55b4e3e864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3e86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b4e3e8bcc0 con 0x55b4e3e799f0 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b4e3d67290 con 0x55b4e3e890c0 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4e3e8be40 con 0x55b4e3e7a310 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9539c03740 10 client.0 _ll_register_callbacks cb 0x55b4e3d98140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9534b29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e85f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f953532a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3e86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9534b29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e85f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40004/0 (socket says 172.21.15.137:40004) 2024-09-12T12:15:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f952ffff640 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e7a310 0x55b4e3e864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f9534b29640 1 -- 172.21.15.137:0/1807204105 learned_addr learned my addr 172.21.15.137:0/1807204105 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f953532a640 1 -- 172.21.15.137:0/1807204105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e7a310 msgr2=0x55b4e3e864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f953532a640 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e7a310 0x55b4e3e864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f953532a640 1 -- 172.21.15.137:0/1807204105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 msgr2=0x55b4e3e85f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f953532a640 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e85f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.256+0000 7f953532a640 1 -- 172.21.15.137:0/1807204105 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e3af1210 con 0x55b4e3e890c0 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f9534b29640 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e85f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f953532a640 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3e86a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9528005fd0 tx=0x7f9528012d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683f246df1e684f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f950ffff640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f950ffff640 1 -- 172.21.15.137:0/1807204105 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9528010e80 con 0x55b4e3e890c0 2024-09-12T12:15:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f950ffff640 1 -- 172.21.15.137:0/1807204105 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f9528004b20 con 0x55b4e3e890c0 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9524099fb0 con 0x55b4e3e890c0 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f950ffff640 1 -- 172.21.15.137:0/1807204105 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9528004d20 con 0x55b4e3e890c0 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f9539c03740 10 client.25133 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4e3bbd620 con 0x55b4e3e890c0 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f950ffff640 1 -- 172.21.15.137:0/1807204105 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9524099fb0 con 0x55b4e3e890c0 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.257+0000 7f950ffff640 1 -- 172.21.15.137:0/1807204105 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9528019030 con 0x55b4e3e890c0 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9539c03740 10 client.25133 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9539c03740 10 client.25133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9539c03740 20 client.25133 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9539c03740 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3e99150 0x55b4e3e9b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9539c03740 4 client.25133 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4e3deba20 con 0x55b4e3e99150 2024-09-12T12:15:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9534b29640 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3e99150 0x55b4e3e9b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f9534b29640 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3e99150 0x55b4e3e9b540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d18ee9b6b6ccb0 server_cookie=51eb87c4348fe548 in_seq=0 out_seq=0 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.258+0000 7f950ffff640 10 client.25133 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f950ffff640 1 -- 172.21.15.137:0/1807204105 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4e3deba20 con 0x55b4e3e99150 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f950ffff640 10 client.25133 handle_command_reply: tid=0 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 client.25133 shutdown 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 msgr2=0x55b4e3e86a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3e86a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9528005fd0 tx=0x7f9528012d10 comp rx=0 tx=0).stop 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 shutdown_connections 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3e86a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e7a310 0x55b4e3e864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3e99150 0x55b4e3e9b540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 --2- 172.21.15.137:0/1807204105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e85f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f950ffff640 0 client.25133 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 >> 172.21.15.137:0/1807204105 conn(0x55b4e3e76060 msgr2=0x55b4e3e7bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 shutdown_connections 2024-09-12T12:15:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 1 -- 172.21.15.137:0/1807204105 wait complete. 2024-09-12T12:15:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.259+0000 7f9539c03740 20 client.25133 trim_cache size 0 max 16384 2024-09-12T12:15:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.273+0000 7f9539c03740 1 Processor -- start 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9539c03740 1 -- start start 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9539c03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e799f0 0x55b4e3e87f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9539c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3ea0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9539c03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3ea1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9539c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95301ade40 con 0x55b4e3e7a310 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9539c03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f95301ab6b0 con 0x55b4e3e890c0 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9539c03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95301a9500 con 0x55b4e3e799f0 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3ea0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3ea0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40018/0 (socket says 172.21.15.137:40018) 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 -- 172.21.15.137:0/3263103596 learned_addr learned my addr 172.21.15.137:0/3263103596 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f953532a640 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3ea1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f9534b29640 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e799f0 0x55b4e3e87f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 -- 172.21.15.137:0/3263103596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e799f0 msgr2=0x55b4e3e87f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e799f0 0x55b4e3e87f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 -- 172.21.15.137:0/3263103596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 msgr2=0x55b4e3ea1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3ea1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 -- 172.21.15.137:0/3263103596 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e3e75e90 con 0x55b4e3e7a310 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.274+0000 7f952ffff640 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3ea0e50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4c8fc059b4d6be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f951a7fc640 1 -- 172.21.15.137:0/3263103596 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c035040 con 0x55b4e3e7a310 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f951a7fc640 1 -- 172.21.15.137:0/3263103596 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c003ad0 con 0x55b4e3e7a310 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f951a7fc640 1 -- 172.21.15.137:0/3263103596 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c03cc70 con 0x55b4e3e7a310 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 -- 172.21.15.137:0/3263103596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 msgr2=0x55b4e3ea0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3ea0e50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f951c006010 tx=0x7f951c03d000 comp rx=0 tx=0).stop 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 -- 172.21.15.137:0/3263103596 shutdown_connections 2024-09-12T12:15:27.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e890c0 0x55b4e3ea1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e7a310 0x55b4e3ea0e50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 --2- 172.21.15.137:0/3263103596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e799f0 0x55b4e3e87f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 -- 172.21.15.137:0/3263103596 >> 172.21.15.137:0/3263103596 conn(0x55b4e3e76060 msgr2=0x55b4e3e877f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 -- 172.21.15.137:0/3263103596 shutdown_connections 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.275+0000 7f9539c03740 1 -- 172.21.15.137:0/3263103596 wait complete. 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 -- start start 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 Processor -- start 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e91970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e7a310 0x55b4e3e82e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f95301a9b70 con 0x55b4e3e799f0 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f95301aafe0 con 0x55b4e3e7a310 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95301aa240 con 0x55b4e3e890c0 2024-09-12T12:15:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9539c03740 10 client.0 _ll_register_callbacks cb 0x7f9528009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9534b29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e91970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f952ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e7a310 0x55b4e3e82e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9534b29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e91970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40028/0 (socket says 172.21.15.137:40028) 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f9534b29640 1 -- 172.21.15.137:0/1979263489 learned_addr learned my addr 172.21.15.137:0/1979263489 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.276+0000 7f953532a640 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9534b29640 1 -- 172.21.15.137:0/1979263489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 msgr2=0x55b4e3e835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9534b29640 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9534b29640 1 -- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e7a310 msgr2=0x55b4e3e82e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9534b29640 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e7a310 0x55b4e3e82e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9534b29640 1 -- 172.21.15.137:0/1979263489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e3e76fc0 con 0x55b4e3e799f0 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f952ffff640 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e7a310 0x55b4e3e82e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9534b29640 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e91970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f952400c6a0 tx=0x7f9524018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5418d02101de1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f950ffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f950ffff640 1 -- 172.21.15.137:0/1979263489 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240983a0 con 0x55b4e3e799f0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f950ffff640 1 -- 172.21.15.137:0/1979263489 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f952409b100 con 0x55b4e3e799f0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f951c003ad0 con 0x55b4e3e799f0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f950ffff640 1 -- 172.21.15.137:0/1979263489 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95240ab050 con 0x55b4e3e799f0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 10 client.15645 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4e3f03db0 con 0x55b4e3e799f0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f950ffff640 1 -- 172.21.15.137:0/1979263489 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f951c003ad0 con 0x55b4e3e799f0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f950ffff640 1 -- 172.21.15.137:0/1979263489 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9524097a00 con 0x55b4e3e799f0 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 10 client.15645 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 20 client.15645 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3f040a0 0x55b4e3f06490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 4 client.15645 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:15:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.277+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4e3de5d70 con 0x55b4e3f040a0 2024-09-12T12:15:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.278+0000 7f952ffff640 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3f040a0 0x55b4e3f06490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.278+0000 7f950ffff640 10 client.15645 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.278+0000 7f952ffff640 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3f040a0 0x55b4e3f06490 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1487eb9b7d040bf server_cookie=bc4f520105e705a3 in_seq=0 out_seq=0 2024-09-12T12:15:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f950ffff640 1 -- 172.21.15.137:0/1979263489 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4e3de5d70 con 0x55b4e3f040a0 2024-09-12T12:15:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f950ffff640 10 client.15645 handle_command_reply: tid=0 2024-09-12T12:15:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 client.15645 shutdown 2024-09-12T12:15:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 msgr2=0x55b4e3e91970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e91970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f952400c6a0 tx=0x7f9524018420 comp rx=0 tx=0).stop 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 shutdown_connections 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4e3e890c0 0x55b4e3e835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b4e3e7a310 0x55b4e3e82e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b4e3f040a0 0x55b4e3f06490 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 --2- 172.21.15.137:0/1979263489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b4e3e799f0 0x55b4e3e91970 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f950ffff640 0 client.15645 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 >> 172.21.15.137:0/1979263489 conn(0x55b4e3e76060 msgr2=0x55b4e3e751c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 shutdown_connections 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 1 -- 172.21.15.137:0/1979263489 wait complete. 2024-09-12T12:15:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.333+0000 7f9539c03740 20 client.15645 trim_cache size 0 max 16384 2024-09-12T12:15:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:15:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:15:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:15:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f95200a2da0 msgr2=0x7f95200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f95200a2da0 0x7f95200a5260 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f951c009900 tx=0x7f951c0058d0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 msgr2=0x7f9530198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530198300 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 shutdown_connections 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f95200a2da0 0x7f95200a5260 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95301069c0 0x7f953019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9530101350 0x7f9530198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 --2- 172.21.15.137:0/1645289098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f95301009b0 0x7f9530198300 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 >> 172.21.15.137:0/1645289098 conn(0x7f9530077990 msgr2=0x7f95300699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 shutdown_connections 2024-09-12T12:15:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:27.334+0000 7f9536d8d640 1 -- 172.21.15.137:0/1645289098 wait complete. 2024-09-12T12:15:27.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:27 smithi171 ceph-mon[35098]: pgmap v70: 97 pgs: 97 active+clean; 47 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 970 KiB/s wr, 154 op/s 2024-09-12T12:15:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:27 smithi137 ceph-mon[30950]: pgmap v70: 97 pgs: 97 active+clean; 47 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 970 KiB/s wr, 154 op/s 2024-09-12T12:15:27.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:27 smithi189 ceph-mon[35475]: pgmap v70: 97 pgs: 97 active+clean; 47 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 970 KiB/s wr, 154 op/s 2024-09-12T12:15:27.718 INFO:tasks.workunit.client.0.smithi137.stdout: 1 130719 11.64 MB/sec warmup 62 sec latency 41.194 ms 2024-09-12T12:15:28.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 132769 11.64 MB/sec warmup 63 sec latency 12.444 ms 2024-09-12T12:15:29.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:29 smithi171 ceph-mon[35098]: pgmap v71: 97 pgs: 97 active+clean; 47 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 747 KiB/s wr, 126 op/s 2024-09-12T12:15:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:29 smithi137 ceph-mon[30950]: pgmap v71: 97 pgs: 97 active+clean; 47 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 747 KiB/s wr, 126 op/s 2024-09-12T12:15:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:29 smithi189 ceph-mon[35475]: pgmap v71: 97 pgs: 97 active+clean; 47 MiB data, 594 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 747 KiB/s wr, 126 op/s 2024-09-12T12:15:29.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 135276 11.68 MB/sec warmup 64 sec latency 8.628 ms 2024-09-12T12:15:30.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 137986 11.71 MB/sec warmup 65 sec latency 13.692 ms 2024-09-12T12:15:31.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:31 smithi171 ceph-mon[35098]: pgmap v72: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2024-09-12T12:15:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:31 smithi137 ceph-mon[30950]: pgmap v72: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2024-09-12T12:15:31.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:31 smithi189 ceph-mon[35475]: pgmap v72: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2024-09-12T12:15:31.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 139925 11.70 MB/sec warmup 66 sec latency 12.155 ms 2024-09-12T12:15:32.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 142471 11.73 MB/sec warmup 67 sec latency 10.018 ms 2024-09-12T12:15:33.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:33 smithi171 ceph-mon[35098]: pgmap v73: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-12T12:15:33.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:33 smithi137 ceph-mon[30950]: pgmap v73: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-12T12:15:33.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:33 smithi189 ceph-mon[35475]: pgmap v73: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-12T12:15:33.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 145526 11.82 MB/sec warmup 68 sec latency 18.901 ms 2024-09-12T12:15:34.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 147236 11.74 MB/sec warmup 69 sec latency 16.561 ms 2024-09-12T12:15:35.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:35 smithi171 ceph-mon[35098]: pgmap v74: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-12T12:15:35.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:35 smithi137 ceph-mon[30950]: pgmap v74: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-12T12:15:35.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:35 smithi189 ceph-mon[35475]: pgmap v74: 97 pgs: 97 active+clean; 51 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-12T12:15:35.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 149752 11.78 MB/sec warmup 70 sec latency 11.698 ms 2024-09-12T12:15:36.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 152843 11.87 MB/sec warmup 71 sec latency 17.491 ms 2024-09-12T12:15:37.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:37 smithi171 ceph-mon[35098]: pgmap v75: 97 pgs: 97 active+clean; 56 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.9 MiB/s wr, 414 op/s 2024-09-12T12:15:37.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:37 smithi137 ceph-mon[30950]: pgmap v75: 97 pgs: 97 active+clean; 56 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.9 MiB/s wr, 414 op/s 2024-09-12T12:15:37.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:37 smithi189 ceph-mon[35475]: pgmap v75: 97 pgs: 97 active+clean; 56 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.9 MiB/s wr, 414 op/s 2024-09-12T12:15:37.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 155477 11.89 MB/sec warmup 72 sec latency 11.051 ms 2024-09-12T12:15:38.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 157478 11.88 MB/sec warmup 73 sec latency 24.498 ms 2024-09-12T12:15:39.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:39 smithi171 ceph-mon[35098]: pgmap v76: 97 pgs: 97 active+clean; 56 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 300 op/s 2024-09-12T12:15:39.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:39 smithi137 ceph-mon[30950]: pgmap v76: 97 pgs: 97 active+clean; 56 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 300 op/s 2024-09-12T12:15:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:39 smithi189 ceph-mon[35475]: pgmap v76: 97 pgs: 97 active+clean; 56 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 300 op/s 2024-09-12T12:15:39.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 160090 11.91 MB/sec warmup 74 sec latency 29.278 ms 2024-09-12T12:15:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/2746993773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a441069c0 msgr2=0x7f5a44106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/2746993773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a441069c0 0x7f5a44106da0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009930 tx=0x7f5a4002a420 comp rx=0 tx=0).stop 2024-09-12T12:15:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/2746993773 shutdown_connections 2024-09-12T12:15:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/2746993773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a44101350 0x7f5a4410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.426 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/2746993773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a441009b0 0x7f5a44100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/2746993773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a441069c0 0x7f5a44106da0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/2746993773 >> 172.21.15.137:0/2746993773 conn(0x7f5a44077990 msgr2=0x7f5a440fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/2746993773 shutdown_connections 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/2746993773 wait complete. 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 Processor -- start 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.425+0000 7f5a4bd5f640 1 -- start start 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4bd5f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a441009b0 0x7f5a44198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4bd5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44101350 0x7f5a441988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4bd5f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a441069c0 0x7f5a4419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4bd5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44111eb0 con 0x7f5a44101350 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4bd5f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44111d30 con 0x7f5a441009b0 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4bd5f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44112030 con 0x7f5a441069c0 2024-09-12T12:15:40.427 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44101350 0x7f5a441988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44101350 0x7f5a441988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53962/0 (socket says 172.21.15.137:53962) 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 -- 172.21.15.137:0/1344706401 learned_addr learned my addr 172.21.15.137:0/1344706401 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a49afb640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a441009b0 0x7f5a44198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a441069c0 0x7f5a4419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 -- 172.21.15.137:0/1344706401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a441069c0 msgr2=0x7f5a4419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a441069c0 0x7f5a4419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 -- 172.21.15.137:0/1344706401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a441009b0 msgr2=0x7f5a44198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a441009b0 0x7f5a44198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 -- 172.21.15.137:0/1344706401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4419d350 con 0x7f5a44101350 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a441069c0 0x7f5a4419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a492fa640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44101350 0x7f5a441988a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400ea60 tx=0x7f5a3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.426+0000 7f5a3affd640 1 -- 172.21.15.137:0/1344706401 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400cde0 con 0x7f5a44101350 2024-09-12T12:15:40.428 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.427+0000 7f5a3affd640 1 -- 172.21.15.137:0/1344706401 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a340035f0 con 0x7f5a44101350 2024-09-12T12:15:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.427+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4419d640 con 0x7f5a44101350 2024-09-12T12:15:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.427+0000 7f5a3affd640 1 -- 172.21.15.137:0/1344706401 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34010670 con 0x7f5a44101350 2024-09-12T12:15:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.427+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a441a4ed0 con 0x7f5a44101350 2024-09-12T12:15:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.429+0000 7f5a3affd640 1 -- 172.21.15.137:0/1344706401 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5a340037a0 con 0x7f5a44101350 2024-09-12T12:15:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.429+0000 7f5a3affd640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5a300a2da0 0x7f5a300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.429+0000 7f5a3affd640 1 -- 172.21.15.137:0/1344706401 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f5a340cf700 con 0x7f5a44101350 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.429+0000 7f5a49afb640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5a300a2da0 0x7f5a300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 Processor -- start 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 -- start start 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a49afb640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5a300a2da0 0x7f5a300a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009900 tx=0x7f5a400058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b09af0 0x5573b8b09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b0a410 0x5573b8b14770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5573b8a6e2f0 con 0x5573b8b14cb0 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5573b8a16270 con 0x5573b8b0a410 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5573b8a6a2b0 con 0x5573b8b09af0 2024-09-12T12:15:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4a2fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a492fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b0a410 0x5573b8b14770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a49afb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b09af0 0x5573b8b09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a492fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b0a410 0x5573b8b14770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:38154/0 (socket says 172.21.15.137:38154) 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a49afb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b09af0 0x5573b8b09ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:48834/0 (socket says 172.21.15.137:48834) 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a4a2fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b170a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53974/0 (socket says 172.21.15.137:53974) 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.430+0000 7f5a492fa640 1 -- 172.21.15.137:0/2016856982 learned_addr learned my addr 172.21.15.137:0/2016856982 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a492fa640 1 -- 172.21.15.137:0/2016856982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b09af0 msgr2=0x5573b8b09ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a492fa640 1 --2- 172.21.15.137:0/2016856982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b09af0 0x5573b8b09ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a492fa640 1 -- 172.21.15.137:0/2016856982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 msgr2=0x5573b8b170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a492fa640 1 --2- 172.21.15.137:0/2016856982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a492fa640 1 -- 172.21.15.137:0/2016856982 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b8a15340 con 0x5573b8b0a410 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a492fa640 1 --2- 172.21.15.137:0/2016856982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b0a410 0x5573b8b14770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a34000c00 tx=0x7f5a3409bb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f89376e1507992cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a2ffff640 1 -- 172.21.15.137:0/2016856982 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3409ffc0 con 0x5573b8b0a410 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a2ffff640 1 -- 172.21.15.137:0/2016856982 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a340a4880 con 0x5573b8b0a410 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.431+0000 7f5a2ffff640 1 -- 172.21.15.137:0/2016856982 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3409b060 con 0x5573b8b0a410 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/2016856982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b0a410 msgr2=0x5573b8b14770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/2016856982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b0a410 0x5573b8b14770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a34000c00 tx=0x7f5a3409bb10 comp rx=0 tx=0).stop 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/2016856982 shutdown_connections 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/2016856982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/2016856982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b0a410 0x5573b8b14770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/2016856982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b09af0 0x5573b8b09ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/2016856982 >> 172.21.15.137:0/2016856982 conn(0x5573b8b06060 msgr2=0x5573b8b06440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/2016856982 shutdown_connections 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/2016856982 wait complete. 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 -- start start 2024-09-12T12:15:40.434 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.432+0000 7f5a4ebd7740 1 Processor -- start 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b11c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b12190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5573b8b17880 con 0x5573b8b14cb0 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5573b89f7290 con 0x5573b8b09af0 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5573b8b17a00 con 0x5573b8b0a410 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 10 client.0 _ll_register_callbacks cb 0x5573b8a28140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b126d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53980/0 (socket says 172.21.15.137:53980) 2024-09-12T12:15:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/3041212825 learned_addr learned my addr 172.21.15.137:0/3041212825 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a49afb640 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b11c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a492fa640 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b12190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/3041212825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 msgr2=0x5573b8b12190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b12190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 msgr2=0x5573b8b11c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b11c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/3041212825 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b8781210 con 0x5573b8b14cb0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b126d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00fe20 tx=0x7f5a3c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cb2a90cd833c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/3041212825 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c003970 con 0x5573b8b14cb0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a340a4880 con 0x5573b8b14cb0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/3041212825 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f5a3c003b10 con 0x5573b8b14cb0 2024-09-12T12:15:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/3041212825 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c003d10 con 0x5573b8b14cb0 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 10 client.15657 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.433+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573b884d620 con 0x5573b8b14cb0 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/3041212825 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a340a4880 con 0x5573b8b14cb0 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/3041212825 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5a3c026750 con 0x5573b8b14cb0 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a4ebd7740 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a4ebd7740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a4ebd7740 20 client.15657 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b24dc0 0x5573b8b271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a4ebd7740 4 client.15657 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b8a7ba20 con 0x5573b8b24dc0 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a49afb640 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b24dc0 0x5573b8b271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a2d7fa640 10 client.15657 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:40.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.434+0000 7f5a49afb640 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b24dc0 0x5573b8b271d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27c1f5503e998534 server_cookie=7f31816f998d8517 in_seq=0 out_seq=0 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/3041212825 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573b8a7ba20 con 0x5573b8b24dc0 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a2d7fa640 10 client.15657 handle_command_reply: tid=0 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 client.15657 shutdown 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 msgr2=0x5573b8b126d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b126d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00fe20 tx=0x7f5a3c012820 comp rx=0 tx=0).stop 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 shutdown_connections 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b24dc0 0x5573b8b271d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b126d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b12190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a2d7fa640 0 client.15657 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/3041212825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b11c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 >> 172.21.15.137:0/3041212825 conn(0x5573b8b06060 msgr2=0x5573b8b0bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:40.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 shutdown_connections 2024-09-12T12:15:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.435+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/3041212825 wait complete. 2024-09-12T12:15:40.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.436+0000 7f5a4ebd7740 20 client.15657 trim_cache size 0 max 16384 2024-09-12T12:15:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 Processor -- start 2024-09-12T12:15:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 -- start start 2024-09-12T12:15:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b12500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b2cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441b00b0 con 0x5573b8b14cb0 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ad920 con 0x5573b8b09af0 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ab770 con 0x5573b8b0a410 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b2cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b2cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53984/0 (socket says 172.21.15.137:53984) 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/638380832 learned_addr learned my addr 172.21.15.137:0/638380832 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a492fa640 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a49afb640 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b12500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/638380832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 msgr2=0x5573b8b2caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b2caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/638380832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 msgr2=0x5573b8b12500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b12500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 -- 172.21.15.137:0/638380832 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b8b147a0 con 0x5573b8b14cb0 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a49afb640 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b12500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.450+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b2cfe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00f970 tx=0x7f5a3c026f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9464f6890f59c25c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a2f7fe640 1 -- 172.21.15.137:0/638380832 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c02c040 con 0x5573b8b14cb0 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a2f7fe640 1 -- 172.21.15.137:0/638380832 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c003970 con 0x5573b8b14cb0 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a2f7fe640 1 -- 172.21.15.137:0/638380832 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c01cb70 con 0x5573b8b14cb0 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/638380832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 msgr2=0x5573b8b2cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b2cfe0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c00f970 tx=0x7f5a3c026f50 comp rx=0 tx=0).stop 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/638380832 shutdown_connections 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b14cb0 0x5573b8b2cfe0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b0a410 0x5573b8b2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/638380832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b12500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/638380832 >> 172.21.15.137:0/638380832 conn(0x5573b8b06060 msgr2=0x5573b8b13cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/638380832 shutdown_connections 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.451+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/638380832 wait complete. 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 -- start start 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 Processor -- start 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b1d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b0a410 0x5573b8b0ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b14cb0 0x5573b8b0f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441abde0 con 0x5573b8b0a410 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ad250 con 0x5573b8b09af0 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ac4b0 con 0x5573b8b14cb0 2024-09-12T12:15:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4ebd7740 10 client.0 _ll_register_callbacks cb 0x7f5a3c00f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a492fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b0a410 0x5573b8b0ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a492fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b0a410 0x5573b8b0ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53996/0 (socket says 172.21.15.137:53996) 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a49afb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b1d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a492fa640 1 -- 172.21.15.137:0/1959393173 learned_addr learned my addr 172.21.15.137:0/1959393173 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.452+0000 7f5a4a2fc640 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b14cb0 0x5573b8b0f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a492fa640 1 -- 172.21.15.137:0/1959393173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b14cb0 msgr2=0x5573b8b0f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a492fa640 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b14cb0 0x5573b8b0f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a492fa640 1 -- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 msgr2=0x5573b8b1d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a492fa640 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b1d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a492fa640 1 -- 172.21.15.137:0/1959393173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b8b0f840 con 0x5573b8b0a410 2024-09-12T12:15:40.458 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a49afb640 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b1d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a492fa640 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b0a410 0x5573b8b0ea30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409bba0 tx=0x7f5a34003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1852d3cf6abe6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/1959393173 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3407f060 con 0x5573b8b0a410 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/1959393173 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f5a340cd0c0 con 0x5573b8b0a410 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3c003970 con 0x5573b8b0a410 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/1959393173 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a340a4880 con 0x5573b8b0a410 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 10 client.15663 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5573b8b93ed0 con 0x5573b8b0a410 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/1959393173 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c003970 con 0x5573b8b0a410 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/1959393173 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5a3409b060 con 0x5573b8b0a410 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 10 client.15663 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 20 client.15663 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b94200 0x5573b8b965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 4 client.15663 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:15:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.453+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573b8a75d70 con 0x5573b8b94200 2024-09-12T12:15:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.454+0000 7f5a49afb640 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b94200 0x5573b8b965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.454+0000 7f5a2d7fa640 10 client.15663 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:40.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.454+0000 7f5a49afb640 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b94200 0x5573b8b965f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f092ff8ac36b7a7 server_cookie=ace4cda90f63ac63 in_seq=0 out_seq=0 2024-09-12T12:15:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a2d7fa640 1 -- 172.21.15.137:0/1959393173 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573b8a75d70 con 0x5573b8b94200 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a2d7fa640 10 client.15663 handle_command_reply: tid=0 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 client.15663 shutdown 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b0a410 msgr2=0x5573b8b0ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b0a410 0x5573b8b0ea30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5a3409bba0 tx=0x7f5a34003d60 comp rx=0 tx=0).stop 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 shutdown_connections 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573b8b14cb0 0x5573b8b0f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5573b8b94200 0x5573b8b965f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5573b8b0a410 0x5573b8b0ea30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a4ebd7740 1 --2- 172.21.15.137:0/1959393173 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5573b8b09af0 0x5573b8b1d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.518+0000 7f5a2d7fa640 0 client.15663 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 >> 172.21.15.137:0/1959393173 conn(0x5573b8b06060 msgr2=0x5573b8b049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 shutdown_connections 2024-09-12T12:15:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4ebd7740 1 -- 172.21.15.137:0/1959393173 wait complete. 2024-09-12T12:15:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4ebd7740 20 client.15663 trim_cache size 0 max 16384 2024-09-12T12:15:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:15:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:15:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:15:40.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:15:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5a300a2da0 msgr2=0x7f5a300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5a300a2da0 0x7f5a300a5260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009900 tx=0x7f5a400058d0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44101350 msgr2=0x7f5a441988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44101350 0x7f5a441988a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400ea60 tx=0x7f5a3400ef20 comp rx=0 tx=0).stop 2024-09-12T12:15:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 shutdown_connections 2024-09-12T12:15:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5a300a2da0 0x7f5a300a5260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a441069c0 0x7f5a4419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a44101350 0x7f5a441988a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 --2- 172.21.15.137:0/1344706401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5a441009b0 0x7f5a44198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.519+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 >> 172.21.15.137:0/1344706401 conn(0x7f5a44077990 msgr2=0x7f5a44103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.520+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 shutdown_connections 2024-09-12T12:15:40.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:40.520+0000 7f5a4bd5f640 1 -- 172.21.15.137:0/1344706401 wait complete. 2024-09-12T12:15:40.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 162646 11.92 MB/sec warmup 75 sec latency 21.159 ms 2024-09-12T12:15:41.452 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:41 smithi171 ceph-mon[35098]: pgmap v77: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 440 op/s 2024-09-12T12:15:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:41 smithi137 ceph-mon[30950]: pgmap v77: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 440 op/s 2024-09-12T12:15:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:41 smithi189 ceph-mon[35475]: pgmap v77: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.4 MiB/s wr, 440 op/s 2024-09-12T12:15:41.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 164671 11.92 MB/sec warmup 76 sec latency 13.560 ms 2024-09-12T12:15:42.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 167261 11.94 MB/sec warmup 77 sec latency 8.575 ms 2024-09-12T12:15:43.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:43 smithi171 ceph-mon[35098]: pgmap v78: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-09-12T12:15:43.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:43 smithi137 ceph-mon[30950]: pgmap v78: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-09-12T12:15:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:43 smithi189 ceph-mon[35475]: pgmap v78: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-09-12T12:15:43.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 170363 12.02 MB/sec warmup 78 sec latency 13.155 ms 2024-09-12T12:15:44.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 172099 11.95 MB/sec warmup 79 sec latency 25.195 ms 2024-09-12T12:15:44.753 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T12:15:44.753 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:15:45.019 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:15:45.449 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:45 smithi137 ceph-mon[30950]: pgmap v79: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-09-12T12:15:45.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:45 smithi171 ceph-mon[35098]: pgmap v79: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-09-12T12:15:45.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 -- 172.21.15.137:0/1070655139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f547410c180 msgr2=0x7f547410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:45.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 --2- 172.21.15.137:0/1070655139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f547410c180 0x7f547410e600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f546c009900 tx=0x7f546c02a480 comp rx=0 tx=0).stop 2024-09-12T12:15:45.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 -- 172.21.15.137:0/1070655139 shutdown_connections 2024-09-12T12:15:45.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 --2- 172.21.15.137:0/1070655139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f547410c180 0x7f547410e600 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 --2- 172.21.15.137:0/1070655139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54741036c0 0x7f547410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 --2- 172.21.15.137:0/1070655139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5474102da0 0x7f5474103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 -- 172.21.15.137:0/1070655139 >> 172.21.15.137:0/1070655139 conn(0x7f54740fc7a0 msgr2=0x7f54740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 -- 172.21.15.137:0/1070655139 shutdown_connections 2024-09-12T12:15:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.536+0000 7f5479797640 1 -- 172.21.15.137:0/1070655139 wait complete. 2024-09-12T12:15:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 Processor -- start 2024-09-12T12:15:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 -- start start 2024-09-12T12:15:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5474102da0 0x7f5474100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54741036c0 0x7f5474100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f547410c180 0x7f5474101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5474111f60 con 0x7f54741036c0 2024-09-12T12:15:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5474111de0 con 0x7f5474102da0 2024-09-12T12:15:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5479797640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54741120e0 con 0x7f547410c180 2024-09-12T12:15:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f54727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54741036c0 0x7f5474100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f54727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54741036c0 0x7f5474100f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54024/0 (socket says 172.21.15.137:54024) 2024-09-12T12:15:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f54727fc640 1 -- 172.21.15.137:0/2044843683 learned_addr learned my addr 172.21.15.137:0/2044843683 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.537+0000 7f5472ffd640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5474102da0 0x7f5474100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f54737fe640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f547410c180 0x7f5474101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:45.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f54727fc640 1 -- 172.21.15.137:0/2044843683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f547410c180 msgr2=0x7f5474101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:45.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f54727fc640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f547410c180 0x7f5474101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f54727fc640 1 -- 172.21.15.137:0/2044843683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5474102da0 msgr2=0x7f5474100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:45.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f54727fc640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5474102da0 0x7f5474100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f54727fc640 1 -- 172.21.15.137:0/2044843683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54741a68e0 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f5472ffd640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5474102da0 0x7f5474100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f54727fc640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54741036c0 0x7f5474100f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f545c00e9f0 tx=0x7f545c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f546bfff640 1 -- 172.21.15.137:0/2044843683 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c00cd80 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54741a6bd0 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f546bfff640 1 -- 172.21.15.137:0/2044843683 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f545c010910 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f546bfff640 1 -- 172.21.15.137:0/2044843683 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c0188f0 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.538+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a7160 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.539+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5474103180 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.540+0000 7f546bfff640 1 -- 172.21.15.137:0/2044843683 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f545c010ac0 con 0x7f54741036c0 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.541+0000 7f546bfff640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54600a2cf0 0x7f54600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:45.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.541+0000 7f5472ffd640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54600a2cf0 0x7f54600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.541+0000 7f546bfff640 1 -- 172.21.15.137:0/2044843683 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f545c0cf740 con 0x7f54741036c0 2024-09-12T12:15:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.541+0000 7f5472ffd640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54600a2cf0 0x7f54600a51b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5464005fd0 tx=0x7f5464009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.545+0000 7f546bfff640 1 -- 172.21.15.137:0/2044843683 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f545c09bd60 con 0x7f54741036c0 2024-09-12T12:15:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:45 smithi189 ceph-mon[35475]: pgmap v79: 97 pgs: 97 active+clean; 62 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 278 op/s 2024-09-12T12:15:45.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 174499 11.98 MB/sec warmup 80 sec latency 11.344 ms 2024-09-12T12:15:45.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.775+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5474107220 con 0x7f54741036c0 2024-09-12T12:15:45.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.776+0000 7f546bfff640 1 -- 172.21.15.137:0/2044843683 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f545c0a0060 con 0x7f54741036c0 2024-09-12T12:15:45.777 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:15:45.777 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":27,"btime":"2024-09-12T12:14:37:790728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:14:36.964904+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:15:45.777 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 27 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54600a2cf0 msgr2=0x7f54600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54600a2cf0 0x7f54600a51b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5464005fd0 tx=0x7f5464009500 comp rx=0 tx=0).stop 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54741036c0 msgr2=0x7f5474100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54741036c0 0x7f5474100f40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f545c00e9f0 tx=0x7f545c00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 shutdown_connections 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54600a2cf0 0x7f54600a51b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f547410c180 0x7f5474101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54741036c0 0x7f5474100f40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 --2- 172.21.15.137:0/2044843683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5474102da0 0x7f5474100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:45.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 >> 172.21.15.137:0/2044843683 conn(0x7f54740fc7a0 msgr2=0x7f54740fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:45.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 shutdown_connections 2024-09-12T12:15:45.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:45.778+0000 7f5479797640 1 -- 172.21.15.137:0/2044843683 wait complete. 2024-09-12T12:15:46.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:46 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2044843683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:46.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:46 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2044843683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:46.593 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:46 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2044843683' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:15:46.636 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T12:15:46.636 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T12:15:46.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 177591 12.06 MB/sec warmup 81 sec latency 12.116 ms 2024-09-12T12:15:46.909 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:15:47.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:47 smithi137 ceph-mon[30950]: pgmap v80: 97 pgs: 97 active+clean; 61 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 442 op/s 2024-09-12T12:15:47.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:47 smithi171 ceph-mon[35098]: pgmap v80: 97 pgs: 97 active+clean; 61 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 442 op/s 2024-09-12T12:15:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.441+0000 7f9eaf62b640 1 -- 172.21.15.137:0/540140641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8102570 msgr2=0x7f9ea81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.441+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/540140641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8102570 0x7f9ea81029b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009900 tx=0x7f9e9802a480 comp rx=0 tx=0).stop 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- 172.21.15.137:0/540140641 shutdown_connections 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/540140641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ea8102ef0 0x7f9ea810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/540140641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8102570 0x7f9ea81029b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/540140641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8100e20 0x7f9ea8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- 172.21.15.137:0/540140641 >> 172.21.15.137:0/540140641 conn(0x7f9ea80fc7d0 msgr2=0x7f9ea80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- 172.21.15.137:0/540140641 shutdown_connections 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- 172.21.15.137:0/540140641 wait complete. 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 Processor -- start 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- start start 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100e20 0x7f9ea80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8102570 0x7f9ea80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ea8102ef0 0x7f9ea80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8110f80 con 0x7f9ea8100e20 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8110e00 con 0x7f9ea8102570 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.442+0000 7f9eaf62b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea8111100 con 0x7f9ea8102ef0 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100e20 0x7f9ea80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100e20 0x7f9ea80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54046/0 (socket says 172.21.15.137:54046) 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2263606228 learned_addr learned my addr 172.21.15.137:0/2263606228 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8102570 0x7f9ea80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2263606228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ea8102ef0 msgr2=0x7f9ea80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ea8102ef0 0x7f9ea80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ea8102ef0 0x7f9ea80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2263606228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8102570 msgr2=0x7f9ea80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8102570 0x7f9ea80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2263606228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea81a6a30 con 0x7f9ea8100e20 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100e20 0x7f9ea80fcc80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400d950 tx=0x7f9ea400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9e9e7fc640 1 -- 172.21.15.137:0/2263606228 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4017030 con 0x7f9ea8100e20 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9e9e7fc640 1 -- 172.21.15.137:0/2263606228 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea4004020 con 0x7f9ea8100e20 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea81a6d20 con 0x7f9ea8100e20 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9e9e7fc640 1 -- 172.21.15.137:0/2263606228 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40047a0 con 0x7f9ea8100e20 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.443+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea81a71d0 con 0x7f9ea8100e20 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.445+0000 7f9e9e7fc640 1 -- 172.21.15.137:0/2263606228 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9ea40041d0 con 0x7f9ea8100e20 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 Processor -- start 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 -- start start 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9e9e7fc640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9e940a2cf0 0x7f9e940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f1fc5492a0 con 0x55f1fc5ee940 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f1fc4f0b10 con 0x55f1fc5e3760 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1fc545260 con 0x55f1fc5e4080 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9e940a2cf0 0x7f9e940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.446+0000 7f9eacbc6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ee400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:48890/0 (socket says 172.21.15.137:48890) 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 -- 172.21.15.137:0/2915460202 learned_addr learned my addr 172.21.15.137:0/2915460202 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:38244/0 (socket says 172.21.15.137:38244) 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9e9e7fc640 1 -- 172.21.15.137:0/2263606228 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f9ea40cedc0 con 0x7f9ea8100e20 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 -- 172.21.15.137:0/2915460202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e3760 msgr2=0x55f1fc5e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 -- 172.21.15.137:0/2915460202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 msgr2=0x55f1fc5f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 -- 172.21.15.137:0/2915460202 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1fc2bab90 con 0x55f1fc5e4080 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9e940a2cf0 0x7f9e940a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005fd0 tx=0x7f9ea0009c00 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ee400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e9802a9a0 tx=0x7f9e980040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ff173832a01a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9e937fe640 1 -- 172.21.15.137:0/2915460202 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98035030 con 0x55f1fc5e4080 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9e937fe640 1 -- 172.21.15.137:0/2915460202 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e98040070 con 0x55f1fc5e4080 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.447+0000 7f9e937fe640 1 -- 172.21.15.137:0/2915460202 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e980039f0 con 0x55f1fc5e4080 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2915460202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 msgr2=0x55f1fc5ee400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ee400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e9802a9a0 tx=0x7f9e980040c0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2915460202 shutdown_connections 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ee400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2915460202 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2915460202 >> 172.21.15.137:0/2915460202 conn(0x55f1fc5dfcf0 msgr2=0x55f1fc5e00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2915460202 shutdown_connections 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2915460202 wait complete. 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.448+0000 7f9eb1a04740 1 -- start start 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 1 Processor -- start 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5ee940 0x55f1fc5ec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f1fc5f1510 con 0x55f1fc5e3760 2024-09-12T12:15:47.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f1fc4d16e0 con 0x55f1fc5ee940 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f1fc5f1690 con 0x55f1fc5e4080 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eb1a04740 10 client.0 _ll_register_callbacks cb 0x55f1fc4f5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5eb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54058/0 (socket says 172.21.15.137:54058) 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eacbc6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eadbc8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5ee940 0x55f1fc5ec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2173662963 learned_addr learned my addr 172.21.15.137:0/2173662963 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2173662963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 msgr2=0x55f1fc5ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5ee940 msgr2=0x55f1fc5ec2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5ee940 0x55f1fc5ec2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9ead3c7640 1 -- 172.21.15.137:0/2173662963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1fc5faa60 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.449+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5ee940 0x55f1fc5ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5eb850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40045f0 tx=0x7f9ea400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b46bfdc96a2f11c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9e90ff9640 1 -- 172.21.15.137:0/2173662963 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea409f6e0 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9e90ff9640 1 -- 172.21.15.137:0/2173662963 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f9ea40979a0 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e98040070 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9e90ff9640 1 -- 172.21.15.137:0/2173662963 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ea40a3fa0 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 10 client.15681 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f1fc5fd450 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9e90ff9640 1 -- 172.21.15.137:0/2173662963 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e98040070 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9e90ff9640 1 -- 172.21.15.137:0/2173662963 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9ea4066260 con 0x55f1fc5e3760 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:47.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 10 client.15681 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 20 client.15681 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc5fe860 0x55f1fc600c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 4 client.15681 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1fc5569d0 con 0x55f1fc5fe860 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.450+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc5fe860 0x55f1fc600c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.451+0000 7f9e90ff9640 10 client.15681 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.451+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc5fe860 0x55f1fc600c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff42f5210d00980 server_cookie=f1714676c1d10d32 in_seq=0 out_seq=0 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.451+0000 7f9e90ff9640 1 -- 172.21.15.137:0/2173662963 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1fc5569d0 con 0x55f1fc5fe860 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.451+0000 7f9e90ff9640 10 client.15681 handle_command_reply: tid=0 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.451+0000 7f9eb1a04740 1 client.15681 shutdown 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 msgr2=0x55f1fc5eb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5eb850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40045f0 tx=0x7f9ea400d8d0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 shutdown_connections 2024-09-12T12:15:47.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5ee940 0x55f1fc5ec2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc5fe860 0x55f1fc600c90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/2173662963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5eb850 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9e90ff9640 0 client.15681 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 >> 172.21.15.137:0/2173662963 conn(0x55f1fc5dfcf0 msgr2=0x55f1fc5dee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 shutdown_connections 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 1 -- 172.21.15.137:0/2173662963 wait complete. 2024-09-12T12:15:47.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.452+0000 7f9eb1a04740 20 client.15681 trim_cache size 0 max 16384 2024-09-12T12:15:47.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 Processor -- start 2024-09-12T12:15:47.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 -- start start 2024-09-12T12:15:47.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ded60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5ee940 0x55f1fc606510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81b2290 con 0x55f1fc5e3760 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81afb00 con 0x55f1fc5e4080 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81ad950 con 0x55f1fc5ee940 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54066/0 (socket says 172.21.15.137:54066) 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 -- 172.21.15.137:0/4176796366 learned_addr learned my addr 172.21.15.137:0/4176796366 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ded60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5ee940 0x55f1fc606510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 -- 172.21.15.137:0/4176796366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5ee940 msgr2=0x55f1fc606510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5ee940 0x55f1fc606510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 -- 172.21.15.137:0/4176796366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 msgr2=0x55f1fc5ded60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ded60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9ead3c7640 1 -- 172.21.15.137:0/4176796366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1fc5ea800 con 0x55f1fc5e3760 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.469+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ded60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.470+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e61c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ccb60 tx=0x7f9ea400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e02982e80b157e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.470+0000 7f9e92ffd640 1 -- 172.21.15.137:0/4176796366 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea409f6e0 con 0x55f1fc5e3760 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.470+0000 7f9e92ffd640 1 -- 172.21.15.137:0/4176796366 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea40662a0 con 0x55f1fc5e3760 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.470+0000 7f9e92ffd640 1 -- 172.21.15.137:0/4176796366 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40ba190 con 0x55f1fc5e3760 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.470+0000 7f9eb1a04740 1 -- 172.21.15.137:0/4176796366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 msgr2=0x55f1fc5e61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.470+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e61c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ccb60 tx=0x7f9ea400df60 comp rx=0 tx=0).stop 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- 172.21.15.137:0/4176796366 shutdown_connections 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5ee940 0x55f1fc606510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc5ded60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/4176796366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5e3760 0x55f1fc5e61c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- 172.21.15.137:0/4176796366 >> 172.21.15.137:0/4176796366 conn(0x55f1fc5dfcf0 msgr2=0x55f1fc5de650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- 172.21.15.137:0/4176796366 shutdown_connections 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- 172.21.15.137:0/4176796366 wait complete. 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- start start 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 Processor -- start 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e3760 0x55f1fc662870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc662db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81adfc0 con 0x55f1fc5ee940 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81af430 con 0x55f1fc5e4080 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea81ae690 con 0x55f1fc5e3760 2024-09-12T12:15:47.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.471+0000 7f9eb1a04740 10 client.0 _ll_register_callbacks cb 0x7f9ea409ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e3760 0x55f1fc662870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e3760 0x55f1fc662870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:48920/0 (socket says 172.21.15.137:48920) 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 -- 172.21.15.137:0/3307971130 learned_addr learned my addr 172.21.15.137:0/3307971130 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc662db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 -- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 msgr2=0x55f1fc662db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc662db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 -- 172.21.15.137:0/3307971130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 msgr2=0x55f1fc5f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 -- 172.21.15.137:0/3307971130 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1fc5f36b0 con 0x55f1fc5e3760 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9eadbc8640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9ead3c7640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e3760 0x55f1fc662870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ba670 tx=0x7f9ea4097c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e30b89f088f4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:47.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9e90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9e90ff9640 1 -- 172.21.15.137:0/3307971130 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40b42f0 con 0x55f1fc5e3760 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9e90ff9640 1 -- 172.21.15.137:0/3307971130 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f9ea409a780 con 0x55f1fc5e3760 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea40662a0 con 0x55f1fc5e3760 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.472+0000 7f9e90ff9640 1 -- 172.21.15.137:0/3307971130 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ea40ba3a0 con 0x55f1fc5e3760 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 10 client.25237 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f1fc5f36b0 con 0x55f1fc5e3760 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9e90ff9640 1 -- 172.21.15.137:0/3307971130 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea40662a0 con 0x55f1fc5e3760 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9e90ff9640 1 -- 172.21.15.137:0/3307971130 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9ea407e020 con 0x55f1fc5e3760 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 10 client.25237 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 10 client.25237 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 20 client.25237 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc66a3c0 0x55f1fc66c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 4 client.25237 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T12:15:47.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x55f1fc550d20 con 0x55f1fc66a3c0 2024-09-12T12:15:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc66a3c0 0x55f1fc66c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9eacbc6640 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc66a3c0 0x55f1fc66c7b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd51f3a3295fe726 server_cookie=dcf50631410356f in_seq=0 out_seq=0 2024-09-12T12:15:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.473+0000 7f9e90ff9640 10 client.25237 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9e90ff9640 1 -- 172.21.15.137:0/3307971130 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f1fc550d20 con 0x55f1fc66a3c0 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9e90ff9640 10 client.25237 handle_command_reply: tid=0 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 client.25237 shutdown 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e3760 msgr2=0x55f1fc662870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e3760 0x55f1fc662870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ba670 tx=0x7f9ea4097c50 comp rx=0 tx=0).stop 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 shutdown_connections 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f1fc66a3c0 0x55f1fc66c7b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f1fc5ee940 0x55f1fc5f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f1fc5e4080 0x55f1fc662db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9e90ff9640 0 client.25237 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:47.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 --2- 172.21.15.137:0/3307971130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f1fc5e3760 0x55f1fc662870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 >> 172.21.15.137:0/3307971130 conn(0x55f1fc5dfcf0 msgr2=0x55f1fc5dee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 shutdown_connections 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.538+0000 7f9eb1a04740 1 -- 172.21.15.137:0/3307971130 wait complete. 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eb1a04740 20 client.25237 trim_cache size 0 max 16384 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 27, 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 10, 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:15:47.541 INFO:teuthology.orchestra.run.smithi137.stdout: "9a1af08d": { 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 10, 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.7, 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:15:47.542 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:15:47.543 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:15:47.543 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:47.543 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:47.543 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:47.543 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:47.543 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:15:47.543 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9e940a2cf0 msgr2=0x7f9e940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9e940a2cf0 0x7f9e940a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005fd0 tx=0x7f9ea0009c00 comp rx=0 tx=0).stop 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100e20 msgr2=0x7f9ea80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100e20 0x7f9ea80fcc80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400d950 tx=0x7f9ea400de10 comp rx=0 tx=0).stop 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 shutdown_connections 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9e940a2cf0 0x7f9e940a51b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ea8102ef0 0x7f9ea80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ea8102570 0x7f9ea80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 --2- 172.21.15.137:0/2263606228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ea8100e20 0x7f9ea80fcc80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 >> 172.21.15.137:0/2263606228 conn(0x7f9ea80fc7d0 msgr2=0x7f9ea806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:47.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 shutdown_connections 2024-09-12T12:15:47.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:15:47.539+0000 7f9eaf62b640 1 -- 172.21.15.137:0/2263606228 wait complete. 2024-09-12T12:15:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:47 smithi189 ceph-mon[35475]: pgmap v80: 97 pgs: 97 active+clean; 61 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 442 op/s 2024-09-12T12:15:47.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.98 MB/sec warmup 82 sec latency 218.357 ms 2024-09-12T12:15:48.453 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9a1af08d', quiesce took 3.7 seconds. Will release after: 21.0 2024-09-12T12:15:48.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.83 MB/sec warmup 83 sec latency 1218.467 ms 2024-09-12T12:15:49.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:49 smithi171 ceph-mon[35098]: pgmap v81: 97 pgs: 97 active+clean; 61 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-09-12T12:15:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:49 smithi137 ceph-mon[30950]: pgmap v81: 97 pgs: 97 active+clean; 61 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-09-12T12:15:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:49 smithi189 ceph-mon[35475]: pgmap v81: 97 pgs: 97 active+clean; 61 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-09-12T12:15:49.719 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.69 MB/sec warmup 84 sec latency 2218.568 ms 2024-09-12T12:15:50.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.55 MB/sec warmup 85 sec latency 3218.686 ms 2024-09-12T12:15:51.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:51 smithi171 ceph-mon[35098]: pgmap v82: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 382 op/s 2024-09-12T12:15:51.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:51 smithi137 ceph-mon[30950]: pgmap v82: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 382 op/s 2024-09-12T12:15:51.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:51 smithi189 ceph-mon[35475]: pgmap v82: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.4 MiB/s wr, 382 op/s 2024-09-12T12:15:51.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.42 MB/sec warmup 86 sec latency 4218.792 ms 2024-09-12T12:15:52.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.29 MB/sec warmup 87 sec latency 5218.856 ms 2024-09-12T12:15:53.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:53 smithi171 ceph-mon[35098]: pgmap v83: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-12T12:15:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:53 smithi137 ceph-mon[30950]: pgmap v83: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-12T12:15:53.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:53 smithi189 ceph-mon[35475]: pgmap v83: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-12T12:15:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 -- 172.21.15.137:0/3727928374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 msgr2=0x7f5268100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 --2- 172.21.15.137:0/3727928374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268100e30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f525c009930 tx=0x7f525c02a420 comp rx=0 tx=0).stop 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 -- 172.21.15.137:0/3727928374 shutdown_connections 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 --2- 172.21.15.137:0/3727928374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5268101370 0x7f526810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 --2- 172.21.15.137:0/3727928374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268100e30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 --2- 172.21.15.137:0/3727928374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52681069e0 0x7f5268106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 -- 172.21.15.137:0/3727928374 >> 172.21.15.137:0/3727928374 conn(0x7f52680fc6f0 msgr2=0x7f52680feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 -- 172.21.15.137:0/3727928374 shutdown_connections 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.613+0000 7f526df78640 1 -- 172.21.15.137:0/3727928374 wait complete. 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 Processor -- start 2024-09-12T12:15:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 -- start start 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5268101370 0x7f52681977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52681069e0 0x7f526819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5268110de0 con 0x7f52681009d0 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5268110c60 con 0x7f52681069e0 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f526df78640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5268110f60 con 0x7f5268101370 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f52677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f52677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41416/0 (socket says 172.21.15.137:41416) 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f52677fe640 1 -- 172.21.15.137:0/3388303608 learned_addr learned my addr 172.21.15.137:0/3388303608 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f5267fff640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52681069e0 0x7f526819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f5266ffd640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5268101370 0x7f52681977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f52677fe640 1 -- 172.21.15.137:0/3388303608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5268101370 msgr2=0x7f52681977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.614+0000 7f52677fe640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5268101370 0x7f52681977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f52677fe640 1 -- 172.21.15.137:0/3388303608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52681069e0 msgr2=0x7f526819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f52677fe640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52681069e0 0x7f526819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f52677fe640 1 -- 172.21.15.137:0/3388303608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526819c260 con 0x7f52681009d0 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f5266ffd640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5268101370 0x7f52681977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f52677fe640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268197270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9f0 tx=0x7f525800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f5264ff9640 1 -- 172.21.15.137:0/3388303608 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525800cd80 con 0x7f52681009d0 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f5264ff9640 1 -- 172.21.15.137:0/3388303608 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258010910 con 0x7f52681009d0 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f5264ff9640 1 -- 172.21.15.137:0/3388303608 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258018990 con 0x7f52681009d0 2024-09-12T12:15:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526819c550 con 0x7f52681009d0 2024-09-12T12:15:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.615+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681a3de0 con 0x7f52681009d0 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.617+0000 7f5264ff9640 1 -- 172.21.15.137:0/3388303608 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5258010ac0 con 0x7f52681009d0 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.617+0000 7f5270df2740 1 Processor -- start 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.617+0000 7f5270df2740 1 -- start start 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5270df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499aaf0 0x562d5499aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5270df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549a5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5270df2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5270df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d548ff2f0 con 0x562d5499b410 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5270df2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562d548a7270 con 0x562d549a5cb0 2024-09-12T12:15:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5270df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d548fb2b0 con 0x562d5499aaf0 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549a5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41426/0 (socket says 172.21.15.137:41426) 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 -- 172.21.15.137:0/1445026800 learned_addr learned my addr 172.21.15.137:0/1445026800 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5267fff640 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499aaf0 0x562d5499aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5266ffd640 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5264ff9640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52500a2da0 0x7f52500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 -- 172.21.15.137:0/1445026800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499aaf0 msgr2=0x562d5499aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499aaf0 0x562d5499aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 -- 172.21.15.137:0/1445026800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 msgr2=0x562d549a80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549a80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 -- 172.21.15.137:0/1445026800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d548a6340 con 0x562d5499b410 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5267fff640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52500a2da0 0x7f52500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f5264ff9640 1 -- 172.21.15.137:0/3388303608 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f52580cf740 con 0x7f52681009d0 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.618+0000 7f52677fe640 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549a5770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f52580cce50 tx=0x7f5258091140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab8c338e5525c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f524dffb640 1 -- 172.21.15.137:0/1445026800 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525809b7f0 con 0x562d5499b410 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5267fff640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52500a2da0 0x7f52500a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f5260004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f524dffb640 1 -- 172.21.15.137:0/1445026800 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258064020 con 0x562d5499b410 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f524dffb640 1 -- 172.21.15.137:0/1445026800 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525806bc40 con 0x562d5499b410 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 -- 172.21.15.137:0/1445026800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 msgr2=0x562d549a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549a5770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f52580cce50 tx=0x7f5258091140 comp rx=0 tx=0).stop 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 -- 172.21.15.137:0/1445026800 shutdown_connections 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549a80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549a5770 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 --2- 172.21.15.137:0/1445026800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499aaf0 0x562d5499aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 -- 172.21.15.137:0/1445026800 >> 172.21.15.137:0/1445026800 conn(0x562d54997060 msgr2=0x562d54997440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.619+0000 7f5270df2740 1 -- 172.21.15.137:0/1445026800 shutdown_connections 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 -- 172.21.15.137:0/1445026800 wait complete. 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 -- start start 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 Processor -- start 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d549a5cb0 0x562d549a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562d549a8880 con 0x562d549a5cb0 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562d54888290 con 0x562d5499aaf0 2024-09-12T12:15:53.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d549a8a00 con 0x562d5499b410 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5270df2740 10 client.0 _ll_register_callbacks cb 0x562d548b9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5267fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d549a5cb0 0x562d549a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5267fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d549a5cb0 0x562d549a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41442/0 (socket says 172.21.15.137:41442) 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.620+0000 7f5267fff640 1 -- 172.21.15.137:0/3184287251 learned_addr learned my addr 172.21.15.137:0/3184287251 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f52677fe640 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5266ffd640 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5267fff640 1 -- 172.21.15.137:0/3184287251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 msgr2=0x562d549a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5267fff640 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5267fff640 1 -- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 msgr2=0x562d549a2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5267fff640 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5267fff640 1 -- 172.21.15.137:0/3184287251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d54612210 con 0x562d549a5cb0 2024-09-12T12:15:53.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5267fff640 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d549a5cb0 0x562d549a3590 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f526000fb70 tx=0x7f5260004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbe956f1f92b4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5246ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5246ffd640 1 -- 172.21.15.137:0/3184287251 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526001eac0 con 0x562d549a5cb0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5246ffd640 1 -- 172.21.15.137:0/3184287251 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f526001ec60 con 0x562d549a5cb0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5258064020 con 0x562d549a5cb0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5246ffd640 1 -- 172.21.15.137:0/3184287251 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f526001ee60 con 0x562d549a5cb0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5270df2740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d546de620 con 0x562d549a5cb0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.621+0000 7f5246ffd640 1 -- 172.21.15.137:0/3184287251 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258064020 con 0x562d549a5cb0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5246ffd640 1 -- 172.21.15.137:0/3184287251 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f52600284b0 con 0x562d549a5cb0 2024-09-12T12:15:53.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5270df2740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5270df2740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5270df2740 20 client.15699 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5270df2740 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d549b5c20 0x562d549b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5270df2740 4 client.15699 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d5490ca20 con 0x562d549b5c20 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f52677fe640 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d549b5c20 0x562d549b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f52677fe640 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d549b5c20 0x562d549b8010 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5377be12670a8b3 server_cookie=904dcba885b677c5 in_seq=0 out_seq=0 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.622+0000 7f5246ffd640 10 client.15699 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5246ffd640 1 -- 172.21.15.137:0/3184287251 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d5490ca20 con 0x562d549b5c20 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5246ffd640 10 client.15699 handle_command_reply: tid=0 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 client.15699 shutdown 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d549a5cb0 msgr2=0x562d549a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d549a5cb0 0x562d549a3590 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f526000fb70 tx=0x7f5260004ab0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 shutdown_connections 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d549b5c20 0x562d549b8010 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d549a5cb0 0x562d549a3590 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5246ffd640 0 client.15699 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 --2- 172.21.15.137:0/3184287251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 >> 172.21.15.137:0/3184287251 conn(0x562d54997060 msgr2=0x562d5499cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 shutdown_connections 2024-09-12T12:15:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.623+0000 7f5270df2740 1 -- 172.21.15.137:0/3184287251 wait complete. 2024-09-12T12:15:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.624+0000 7f5270df2740 20 client.15699 trim_cache size 0 max 16384 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 Processor -- start 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 -- start start 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d549a5cb0 0x562d549ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52681af010 con 0x562d5499b410 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ac880 con 0x562d5499aaf0 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5270df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52681aa6d0 con 0x562d549a5cb0 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549ade80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41456/0 (socket says 172.21.15.137:41456) 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 -- 172.21.15.137:0/4169840083 learned_addr learned my addr 172.21.15.137:0/4169840083 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5267fff640 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d549a5cb0 0x562d549ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f52677fe640 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 -- 172.21.15.137:0/4169840083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d549a5cb0 msgr2=0x562d549ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d549a5cb0 0x562d549ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 -- 172.21.15.137:0/4169840083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 msgr2=0x562d549a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f5266ffd640 1 -- 172.21.15.137:0/4169840083 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d54997cf0 con 0x562d5499b410 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.638+0000 7f52677fe640 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5266ffd640 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549ade80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f525c009930 tx=0x7f525c032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c43f0684fb40b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f524d7fa640 1 -- 172.21.15.137:0/4169840083 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c035040 con 0x562d5499b410 2024-09-12T12:15:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f524d7fa640 1 -- 172.21.15.137:0/4169840083 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525c0317b0 con 0x562d5499b410 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f524d7fa640 1 -- 172.21.15.137:0/4169840083 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c004920 con 0x562d5499b410 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 -- 172.21.15.137:0/4169840083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 msgr2=0x562d549ade80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549ade80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f525c009930 tx=0x7f525c032f00 comp rx=0 tx=0).stop 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 -- 172.21.15.137:0/4169840083 shutdown_connections 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d549a5cb0 0x562d549ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499b410 0x562d549ade80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 --2- 172.21.15.137:0/4169840083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d5499aaf0 0x562d549a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 -- 172.21.15.137:0/4169840083 >> 172.21.15.137:0/4169840083 conn(0x562d54997060 msgr2=0x562d549a4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.639+0000 7f5270df2740 1 -- 172.21.15.137:0/4169840083 shutdown_connections 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 -- 172.21.15.137:0/4169840083 wait complete. 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 -- start start 2024-09-12T12:15:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 Processor -- start 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499aaf0 0x562d549a0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549bd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549bd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52681aad40 con 0x562d5499aaf0 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ac1b0 con 0x562d549a5cb0 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52681ab410 con 0x562d5499b410 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5270df2740 10 client.0 _ll_register_callbacks cb 0x7f5260009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f52677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499aaf0 0x562d549a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f52677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499aaf0 0x562d549a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41460/0 (socket says 172.21.15.137:41460) 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f52677fe640 1 -- 172.21.15.137:0/226174165 learned_addr learned my addr 172.21.15.137:0/226174165 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.640+0000 7f5266ffd640 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549bd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5267fff640 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549bd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f52677fe640 1 -- 172.21.15.137:0/226174165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 msgr2=0x562d549bd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f52677fe640 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549bd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f52677fe640 1 -- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 msgr2=0x562d549bd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:15:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f52677fe640 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549bd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f52677fe640 1 -- 172.21.15.137:0/226174165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d549be150 con 0x562d5499aaf0 2024-09-12T12:15:53.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f52677fe640 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499aaf0 0x562d549a0620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f525809b9b0 tx=0x7f5258018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea64c8f9d3fd872f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:15:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5246ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:15:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5246ffd640 1 -- 172.21.15.137:0/226174165 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525809b610 con 0x562d5499aaf0 2024-09-12T12:15:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5246ffd640 1 -- 172.21.15.137:0/226174165 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f5258099d50 con 0x562d5499aaf0 2024-09-12T12:15:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525c0317b0 con 0x562d5499aaf0 2024-09-12T12:15:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5246ffd640 1 -- 172.21.15.137:0/226174165 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f525809c3b0 con 0x562d5499aaf0 2024-09-12T12:15:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-09-12T12:15:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d54a22870 con 0x562d5499aaf0 2024-09-12T12:15:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5246ffd640 1 -- 172.21.15.137:0/226174165 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525c0317b0 con 0x562d5499aaf0 2024-09-12T12:15:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5246ffd640 1 -- 172.21.15.137:0/226174165 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f52580cd160 con 0x562d5499aaf0 2024-09-12T12:15:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:15:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:15:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 20 client.15705 populate_metadata read hostname 'smithi137' 2024-09-12T12:15:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d54a22bb0 0x562d54a24fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:15:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 4 client.15705 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:15:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.641+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d54906d70 con 0x562d54a22bb0 2024-09-12T12:15:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.642+0000 7f5266ffd640 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d54a22bb0 0x562d54a24fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:15:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.642+0000 7f5246ffd640 10 client.15705 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.642+0000 7f5266ffd640 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d54a22bb0 0x562d54a24fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0be6a2ac762180a server_cookie=5bc69fa12b1b2dec in_seq=0 out_seq=0 2024-09-12T12:15:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.692+0000 7f5246ffd640 1 -- 172.21.15.137:0/226174165 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d54906d70 con 0x562d54a22bb0 2024-09-12T12:15:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.692+0000 7f5246ffd640 10 client.15705 handle_command_reply: tid=0 2024-09-12T12:15:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.692+0000 7f5270df2740 1 client.15705 shutdown 2024-09-12T12:15:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499aaf0 msgr2=0x562d549a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499aaf0 0x562d549a0620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f525809b9b0 tx=0x7f5258018420 comp rx=0 tx=0).stop 2024-09-12T12:15:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 shutdown_connections 2024-09-12T12:15:53.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562d549a5cb0 0x562d549bd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d5499b410 0x562d549bd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562d54a22bb0 0x562d54a24fa0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 --2- 172.21.15.137:0/226174165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562d5499aaf0 0x562d549a0620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5246ffd640 0 client.15705 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 >> 172.21.15.137:0/226174165 conn(0x562d54997060 msgr2=0x562d549959a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 shutdown_connections 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 1 -- 172.21.15.137:0/226174165 wait complete. 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.693+0000 7f5270df2740 20 client.15705 trim_cache size 0 max 16384 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:15:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:15:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:15:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:15:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52500a2da0 msgr2=0x7f52500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52500a2da0 0x7f52500a5260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f5260004a40 comp rx=0 tx=0).stop 2024-09-12T12:15:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 msgr2=0x7f5268197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:15:53.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268197270 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f525800e9f0 tx=0x7f525800eeb0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 shutdown_connections 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52500a2da0 0x7f52500a5260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52681069e0 0x7f526819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5268101370 0x7f52681977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 --2- 172.21.15.137:0/3388303608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52681009d0 0x7f5268197270 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 >> 172.21.15.137:0/3388303608 conn(0x7f52680fc6f0 msgr2=0x7f52680fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 shutdown_connections 2024-09-12T12:15:53.698 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:15:53.694+0000 7f526df78640 1 -- 172.21.15.137:0/3388303608 wait complete. 2024-09-12T12:15:53.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.16 MB/sec warmup 88 sec latency 6218.961 ms 2024-09-12T12:15:54.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 11.04 MB/sec warmup 89 sec latency 7219.085 ms 2024-09-12T12:15:55.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:55 smithi171 ceph-mon[35098]: pgmap v84: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-12T12:15:55.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:55 smithi137 ceph-mon[30950]: pgmap v84: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-12T12:15:55.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:55 smithi189 ceph-mon[35475]: pgmap v84: 97 pgs: 97 active+clean; 64 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-12T12:15:55.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.91 MB/sec warmup 90 sec latency 8219.121 ms 2024-09-12T12:15:56.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.79 MB/sec warmup 91 sec latency 9219.212 ms 2024-09-12T12:15:57.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:57 smithi171 ceph-mon[35098]: pgmap v85: 97 pgs: 97 active+clean; 61 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-09-12T12:15:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:57 smithi137 ceph-mon[30950]: pgmap v85: 97 pgs: 97 active+clean; 61 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-09-12T12:15:57.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:57 smithi189 ceph-mon[35475]: pgmap v85: 97 pgs: 97 active+clean; 61 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 293 op/s 2024-09-12T12:15:57.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.68 MB/sec warmup 92 sec latency 10219.312 ms 2024-09-12T12:15:58.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.56 MB/sec warmup 93 sec latency 11219.358 ms 2024-09-12T12:15:59.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:15:59 smithi171 ceph-mon[35098]: pgmap v86: 97 pgs: 97 active+clean; 61 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-12T12:15:59.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:15:59 smithi137 ceph-mon[30950]: pgmap v86: 97 pgs: 97 active+clean; 61 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-12T12:15:59.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:15:59 smithi189 ceph-mon[35475]: pgmap v86: 97 pgs: 97 active+clean; 61 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-12T12:15:59.720 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.45 MB/sec warmup 94 sec latency 12219.465 ms 2024-09-12T12:16:00.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.34 MB/sec warmup 95 sec latency 13219.598 ms 2024-09-12T12:16:01.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:01 smithi171 ceph-mon[35098]: pgmap v87: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-12T12:16:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:01 smithi137 ceph-mon[30950]: pgmap v87: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-12T12:16:01.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:01 smithi189 ceph-mon[35475]: pgmap v87: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-09-12T12:16:01.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.23 MB/sec warmup 96 sec latency 14219.700 ms 2024-09-12T12:16:02.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.13 MB/sec warmup 97 sec latency 15219.826 ms 2024-09-12T12:16:03.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:03 smithi171 ceph-mon[35098]: pgmap v88: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-12T12:16:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:03 smithi137 ceph-mon[30950]: pgmap v88: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-12T12:16:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:03 smithi189 ceph-mon[35475]: pgmap v88: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-12T12:16:03.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 10.02 MB/sec warmup 98 sec latency 16219.894 ms 2024-09-12T12:16:04.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.92 MB/sec warmup 99 sec latency 17219.978 ms 2024-09-12T12:16:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:05 smithi171 ceph-mon[35098]: pgmap v89: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-12T12:16:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:05 smithi137 ceph-mon[30950]: pgmap v89: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-12T12:16:05.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:05 smithi189 ceph-mon[35475]: pgmap v89: 97 pgs: 97 active+clean; 68 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.5 MiB/s wr, 75 op/s 2024-09-12T12:16:05.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.82 MB/sec warmup 100 sec latency 18220.079 ms 2024-09-12T12:16:06.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.72 MB/sec warmup 101 sec latency 19220.127 ms 2024-09-12T12:16:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 -- 172.21.15.137:0/1257424539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe410101370 msgr2=0x7fe41010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 --2- 172.21.15.137:0/1257424539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe410101370 0x7fe41010d8a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009930 tx=0x7fe40c02a420 comp rx=0 tx=0).stop 2024-09-12T12:16:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 -- 172.21.15.137:0/1257424539 shutdown_connections 2024-09-12T12:16:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 --2- 172.21.15.137:0/1257424539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe410101370 0x7fe41010d8a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 --2- 172.21.15.137:0/1257424539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4101009d0 0x7fe410100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 --2- 172.21.15.137:0/1257424539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4101069e0 0x7fe410106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 -- 172.21.15.137:0/1257424539 >> 172.21.15.137:0/1257424539 conn(0x7fe4100fc6b0 msgr2=0x7fe4100fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 -- 172.21.15.137:0/1257424539 shutdown_connections 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 -- 172.21.15.137:0/1257424539 wait complete. 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 Processor -- start 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.784+0000 7fe417ba6640 1 -- start start 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe417ba6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4101009d0 0x7fe410199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe417ba6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe410101370 0x7fe4101996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe417ba6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4101069e0 0x7fe4101a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe417ba6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110e20 con 0x7fe4101009d0 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe417ba6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110ca0 con 0x7fe4101069e0 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe417ba6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe410110fa0 con 0x7fe410101370 2024-09-12T12:16:06.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4101009d0 0x7fe410199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4101009d0 0x7fe410199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53748/0 (socket says 172.21.15.137:53748) 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 -- 172.21.15.137:0/844829067 learned_addr learned my addr 172.21.15.137:0/844829067 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe416143640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4101069e0 0x7fe4101a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415141640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe410101370 0x7fe4101996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 -- 172.21.15.137:0/844829067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe410101370 msgr2=0x7fe4101996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe410101370 0x7fe4101996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 -- 172.21.15.137:0/844829067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4101069e0 msgr2=0x7fe4101a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4101069e0 0x7fe4101a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 -- 172.21.15.137:0/844829067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4101a4750 con 0x7fe4101009d0 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe416143640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4101069e0 0x7fe4101a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe415942640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4101009d0 0x7fe410199190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe40800cce0 tx=0x7fe40800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.785+0000 7fe406ffd640 1 -- 172.21.15.137:0/844829067 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408004b20 con 0x7fe4101009d0 2024-09-12T12:16:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.786+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4101a4920 con 0x7fe4101009d0 2024-09-12T12:16:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.786+0000 7fe406ffd640 1 -- 172.21.15.137:0/844829067 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe408004cc0 con 0x7fe4101009d0 2024-09-12T12:16:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.786+0000 7fe406ffd640 1 -- 172.21.15.137:0/844829067 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40800f8c0 con 0x7fe4101009d0 2024-09-12T12:16:06.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.786+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4101a4e40 con 0x7fe4101009d0 2024-09-12T12:16:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 Processor -- start 2024-09-12T12:16:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 -- start start 2024-09-12T12:16:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a490215ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.799 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490216410 0x55a490220770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490220cb0 0x55a4902230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a49017a2f0 con 0x55a490215af0 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a490122270 con 0x55a490216410 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a4901762b0 con 0x55a490220cb0 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe406ffd640 1 -- 172.21.15.137:0/844829067 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe40800fa60 con 0x7fe4101009d0 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe416143640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a490215ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe416143640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a490215ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53762/0 (socket says 172.21.15.137:53762) 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe416143640 1 -- 172.21.15.137:0/2762627215 learned_addr learned my addr 172.21.15.137:0/2762627215 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.789+0000 7fe415141640 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490220cb0 0x55a4902230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe415942640 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490216410 0x55a490220770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe416143640 1 -- 172.21.15.137:0/2762627215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490220cb0 msgr2=0x55a4902230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe416143640 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490220cb0 0x55a4902230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe416143640 1 -- 172.21.15.137:0/2762627215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490216410 msgr2=0x55a490220770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe416143640 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490216410 0x55a490220770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe416143640 1 -- 172.21.15.137:0/2762627215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a490121340 con 0x55a490215af0 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe416143640 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a490215ed0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005c40 tx=0x7fe40c032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574f88cf70011aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe3fbfff640 1 -- 172.21.15.137:0/2762627215 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c03f070 con 0x55a490215af0 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe3fbfff640 1 -- 172.21.15.137:0/2762627215 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c0317b0 con 0x55a490215af0 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe3fbfff640 1 -- 172.21.15.137:0/2762627215 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c035070 con 0x55a490215af0 2024-09-12T12:16:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe406ffd640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe3fc0a2da0 0x7fe3fc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.790+0000 7fe415942640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe3fc0a2da0 0x7fe3fc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe406ffd640 1 -- 172.21.15.137:0/844829067 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fe4080cf4c0 con 0x7fe4101009d0 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe415942640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe3fc0a2da0 0x7fe3fc0a5260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55a490212ec0 tx=0x7fe40801fab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 -- 172.21.15.137:0/2762627215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 msgr2=0x55a490215ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a490215ed0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005c40 tx=0x7fe40c032cb0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 -- 172.21.15.137:0/2762627215 shutdown_connections 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490220cb0 0x55a4902230a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490216410 0x55a490220770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/2762627215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a490215ed0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 -- 172.21.15.137:0/2762627215 >> 172.21.15.137:0/2762627215 conn(0x55a490212060 msgr2=0x55a490212440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 -- 172.21.15.137:0/2762627215 shutdown_connections 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.791+0000 7fe41aa1f740 1 -- 172.21.15.137:0/2762627215 wait complete. 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 -- start start 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 Processor -- start 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a49021dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a490223880 con 0x55a490215af0 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a490103290 con 0x55a490220cb0 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a490223a00 con 0x55a490216410 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe41aa1f740 10 client.0 _ll_register_callbacks cb 0x55a490134140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe416143640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a49021dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe416143640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a49021dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53764/0 (socket says 172.21.15.137:53764) 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe416143640 1 -- 172.21.15.137:0/3448010648 learned_addr learned my addr 172.21.15.137:0/3448010648 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.792+0000 7fe415141640 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe416143640 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe416143640 1 -- 172.21.15.137:0/3448010648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 msgr2=0x55a49021e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe416143640 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe416143640 1 -- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 msgr2=0x55a49021e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe416143640 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe416143640 1 -- 172.21.15.137:0/3448010648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a48fe8d210 con 0x55a490215af0 2024-09-12T12:16:06.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe416143640 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a49021dac0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005400 tx=0x7fe40c005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac664860ac9d0b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe3f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe3f97fa640 1 -- 172.21.15.137:0/3448010648 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c050060 con 0x55a490215af0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe40c0317b0 con 0x55a490215af0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe3f97fa640 1 -- 172.21.15.137:0/3448010648 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fe40c004250 con 0x55a490215af0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe3f97fa640 1 -- 172.21.15.137:0/3448010648 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe40c03f070 con 0x55a490215af0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe41aa1f740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a48ff59620 con 0x55a490215af0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.793+0000 7fe3f97fa640 1 -- 172.21.15.137:0/3448010648 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c0317b0 con 0x55a490215af0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe3f97fa640 1 -- 172.21.15.137:0/3448010648 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe40c003680 con 0x55a490215af0 2024-09-12T12:16:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe41aa1f740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe41aa1f740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe41aa1f740 20 client.15717 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4902309e0 0x55a490232dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe41aa1f740 4 client.15717 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a490187a20 con 0x55a4902309e0 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe415141640 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4902309e0 0x55a490232dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe3f97fa640 10 client.15717 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.794+0000 7fe415141640 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4902309e0 0x55a490232dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb741135b45ebb25 server_cookie=b1d855341c2500a5 in_seq=0 out_seq=0 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe3f97fa640 1 -- 172.21.15.137:0/3448010648 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a490187a20 con 0x55a4902309e0 2024-09-12T12:16:06.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe3f97fa640 10 client.15717 handle_command_reply: tid=0 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 client.15717 shutdown 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 msgr2=0x55a49021dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a49021dac0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005400 tx=0x7fe40c005300 comp rx=0 tx=0).stop 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 shutdown_connections 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4902309e0 0x55a490232dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/3448010648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490215af0 0x55a49021dac0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe3f97fa640 0 client.15717 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 >> 172.21.15.137:0/3448010648 conn(0x55a490212060 msgr2=0x55a490217dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 shutdown_connections 2024-09-12T12:16:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.795+0000 7fe41aa1f740 1 -- 172.21.15.137:0/3448010648 wait complete. 2024-09-12T12:16:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.796+0000 7fe41aa1f740 20 client.15717 trim_cache size 0 max 16384 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 Processor -- start 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 -- start start 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490215af0 0x55a49021c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490220cb0 0x55a490238460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101aff90 con 0x55a490220cb0 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101ad800 con 0x55a490215af0 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.818+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101ab650 con 0x55a490216410 2024-09-12T12:16:06.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490220cb0 0x55a490238460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490220cb0 0x55a490238460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53766/0 (socket says 172.21.15.137:53766) 2024-09-12T12:16:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 -- 172.21.15.137:0/798078987 learned_addr learned my addr 172.21.15.137:0/798078987 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe415141640 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490215af0 0x55a49021c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 -- 172.21.15.137:0/798078987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 msgr2=0x55a49021e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 -- 172.21.15.137:0/798078987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490215af0 msgr2=0x55a49021c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490215af0 0x55a49021c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 -- 172.21.15.137:0/798078987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a490211e90 con 0x55a490220cb0 2024-09-12T12:16:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe416143640 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490220cb0 0x55a490238460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005e80 tx=0x7fe40c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9591284283920faf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:06.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe3fb7fe640 1 -- 172.21.15.137:0/798078987 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c035040 con 0x55a490220cb0 2024-09-12T12:16:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe3fb7fe640 1 -- 172.21.15.137:0/798078987 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c0317b0 con 0x55a490220cb0 2024-09-12T12:16:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.819+0000 7fe3fb7fe640 1 -- 172.21.15.137:0/798078987 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40c003850 con 0x55a490220cb0 2024-09-12T12:16:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 -- 172.21.15.137:0/798078987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490220cb0 msgr2=0x55a490238460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490220cb0 0x55a490238460 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005e80 tx=0x7fe40c005730 comp rx=0 tx=0).stop 2024-09-12T12:16:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 -- 172.21.15.137:0/798078987 shutdown_connections 2024-09-12T12:16:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490220cb0 0x55a490238460 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490216410 0x55a49021e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/798078987 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490215af0 0x55a49021c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 -- 172.21.15.137:0/798078987 >> 172.21.15.137:0/798078987 conn(0x55a490212060 msgr2=0x55a49021fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 -- 172.21.15.137:0/798078987 shutdown_connections 2024-09-12T12:16:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 -- 172.21.15.137:0/798078987 wait complete. 2024-09-12T12:16:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 -- start start 2024-09-12T12:16:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.820+0000 7fe41aa1f740 1 Processor -- start 2024-09-12T12:16:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490215af0 0x55a4902293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490216410 0x55a49021a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe41aa1f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101abcc0 con 0x55a490216410 2024-09-12T12:16:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101ad130 con 0x55a490220cb0 2024-09-12T12:16:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe41aa1f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4101ac390 con 0x55a490215af0 2024-09-12T12:16:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe41aa1f740 10 client.0 _ll_register_callbacks cb 0x7fe4000092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:06.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490216410 0x55a49021a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490216410 0x55a49021a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53770/0 (socket says 172.21.15.137:53770) 2024-09-12T12:16:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 -- 172.21.15.137:0/161400664 learned_addr learned my addr 172.21.15.137:0/161400664 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe416143640 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490215af0 0x55a4902293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe416143640 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 -- 172.21.15.137:0/161400664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490215af0 msgr2=0x55a4902293b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490215af0 0x55a4902293b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 -- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 msgr2=0x55a49021ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 -- 172.21.15.137:0/161400664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a49020d2d0 con 0x55a490216410 2024-09-12T12:16:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe415141640 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490216410 0x55a49021a7a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe400009c50 tx=0x7fe40001e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5100171075d575f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:06.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.821+0000 7fe3f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe3f97fa640 1 -- 172.21.15.137:0/161400664 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400005220 con 0x55a490216410 2024-09-12T12:16:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe3f97fa640 1 -- 172.21.15.137:0/161400664 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fe4000053c0 con 0x55a490216410 2024-09-12T12:16:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe40c0317b0 con 0x55a490216410 2024-09-12T12:16:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe3f97fa640 1 -- 172.21.15.137:0/161400664 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4000055c0 con 0x55a490216410 2024-09-12T12:16:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 10 client.15723 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a49029b230 con 0x55a490216410 2024-09-12T12:16:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe3f97fa640 1 -- 172.21.15.137:0/161400664 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40c0317b0 con 0x55a490216410 2024-09-12T12:16:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe3f97fa640 1 -- 172.21.15.137:0/161400664 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe40000f040 con 0x55a490216410 2024-09-12T12:16:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 20 client.15723 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a49029ba40 0x55a49029de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 4 client.15723 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:16:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.822+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a490181d70 con 0x55a49029ba40 2024-09-12T12:16:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.823+0000 7fe416143640 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a49029ba40 0x55a49029de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.823+0000 7fe3f97fa640 10 client.15723 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:06.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.823+0000 7fe416143640 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a49029ba40 0x55a49029de60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7016d5ef1614df4e server_cookie=b0b4fd47b1f1de8 in_seq=0 out_seq=0 2024-09-12T12:16:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe3f97fa640 1 -- 172.21.15.137:0/161400664 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a490181d70 con 0x55a49029ba40 2024-09-12T12:16:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe3f97fa640 10 client.15723 handle_command_reply: tid=0 2024-09-12T12:16:06.862 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 client.15723 shutdown 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490216410 msgr2=0x55a49021a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490216410 0x55a49021a7a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe400009c50 tx=0x7fe40001e7c0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 shutdown_connections 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a490220cb0 0x55a49021ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a49029ba40 0x55a49029de60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a490216410 0x55a49021a7a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 --2- 172.21.15.137:0/161400664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a490215af0 0x55a4902293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe3f97fa640 0 client.15723 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 >> 172.21.15.137:0/161400664 conn(0x55a490212060 msgr2=0x55a49022bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 shutdown_connections 2024-09-12T12:16:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 1 -- 172.21.15.137:0/161400664 wait complete. 2024-09-12T12:16:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.861+0000 7fe41aa1f740 20 client.15723 trim_cache size 0 max 16384 2024-09-12T12:16:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:16:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:16:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:16:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:16:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe3fc0a2da0 msgr2=0x7fe3fc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe3fc0a2da0 0x7fe3fc0a5260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55a490212ec0 tx=0x7fe40801fab0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4101009d0 msgr2=0x7fe410199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4101009d0 0x7fe410199190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe40800cce0 tx=0x7fe40800a620 comp rx=0 tx=0).stop 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 shutdown_connections 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe3fc0a2da0 0x7fe3fc0a5260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4101069e0 0x7fe4101a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe410101370 0x7fe4101996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 --2- 172.21.15.137:0/844829067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4101009d0 0x7fe410199190 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 >> 172.21.15.137:0/844829067 conn(0x7fe4100fc6b0 msgr2=0x7fe4100fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 shutdown_connections 2024-09-12T12:16:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:06.862+0000 7fe417ba6640 1 -- 172.21.15.137:0/844829067 wait complete. 2024-09-12T12:16:07.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:07 smithi171 ceph-mon[35098]: pgmap v90: 97 pgs: 97 active+clean; 67 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.3 MiB/s wr, 126 op/s 2024-09-12T12:16:07.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:07 smithi137 ceph-mon[30950]: pgmap v90: 97 pgs: 97 active+clean; 67 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.3 MiB/s wr, 126 op/s 2024-09-12T12:16:07.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:07 smithi189 ceph-mon[35475]: pgmap v90: 97 pgs: 97 active+clean; 67 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 2.3 MiB/s wr, 126 op/s 2024-09-12T12:16:07.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.63 MB/sec warmup 102 sec latency 20220.208 ms 2024-09-12T12:16:08.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.54 MB/sec warmup 103 sec latency 21220.266 ms 2024-09-12T12:16:09.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:09 smithi171 ceph-mon[35098]: pgmap v91: 97 pgs: 97 active+clean; 67 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-09-12T12:16:09.454 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:16:09.481 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:09 smithi137 ceph-mon[30950]: pgmap v91: 97 pgs: 97 active+clean; 67 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-09-12T12:16:09.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:09 smithi189 ceph-mon[35475]: pgmap v91: 97 pgs: 97 active+clean; 67 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.7 MiB/s wr, 75 op/s 2024-09-12T12:16:09.708 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:16:09.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.44 MB/sec warmup 104 sec latency 22220.295 ms 2024-09-12T12:16:10.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/3002459489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44106a80 msgr2=0x7f3e44106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/3002459489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44106a80 0x7f3e44106e60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009930 tx=0x7f3e3402a420 comp rx=0 tx=0).stop 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/3002459489 shutdown_connections 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/3002459489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e44101420 0x7f3e4410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/3002459489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e44100a80 0x7f3e44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/3002459489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44106a80 0x7f3e44106e60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/3002459489 >> 172.21.15.137:0/3002459489 conn(0x7f3e440fc7a0 msgr2=0x7f3e440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/3002459489 shutdown_connections 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.193+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/3002459489 wait complete. 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 Processor -- start 2024-09-12T12:16:10.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 -- start start 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100a80 0x7f3e44199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e44101420 0x7f3e44199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e44106a80 0x7f3e441a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44110ea0 con 0x7f3e44100a80 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44110d20 con 0x7f3e44101420 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e4a7bb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44111020 con 0x7f3e44106a80 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100a80 0x7f3e44199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100a80 0x7f3e44199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45680/0 (socket says 172.21.15.137:45680) 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 -- 172.21.15.137:0/1161699605 learned_addr learned my addr 172.21.15.137:0/1161699605 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e3bfff640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e44101420 0x7f3e44199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e48d58640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e44106a80 0x7f3e441a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:10.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 -- 172.21.15.137:0/1161699605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e44106a80 msgr2=0x7f3e441a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e44106a80 0x7f3e441a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 -- 172.21.15.137:0/1161699605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e44101420 msgr2=0x7f3e44199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e44101420 0x7f3e44199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.194+0000 7f3e43fff640 1 -- 172.21.15.137:0/1161699605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a4810 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.195+0000 7f3e43fff640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100a80 0x7f3e44199250 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009a50 tx=0x7f3e34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.195+0000 7f3e41ffb640 1 -- 172.21.15.137:0/1161699605 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3403f030 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.195+0000 7f3e41ffb640 1 -- 172.21.15.137:0/1161699605 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e34003ad0 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.195+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a49e0 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.195+0000 7f3e41ffb640 1 -- 172.21.15.137:0/1161699605 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34035070 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.195+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a4e30 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.196+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e44100ee0 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.197+0000 7f3e41ffb640 1 -- 172.21.15.137:0/1161699605 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3e34042030 con 0x7f3e44100a80 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.198+0000 7f3e41ffb640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3e300a2cf0 0x7f3e300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:10.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.198+0000 7f3e3bfff640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3e300a2cf0 0x7f3e300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:10.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.198+0000 7f3e41ffb640 1 -- 172.21.15.137:0/1161699605 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f3e340efb30 con 0x7f3e44100a80 2024-09-12T12:16:10.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.198+0000 7f3e3bfff640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3e300a2cf0 0x7f3e300a51b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e4406b2e0 tx=0x7f3e2c00a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:10.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.202+0000 7f3e41ffb640 1 -- 172.21.15.137:0/1161699605 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e340bc7a0 con 0x7f3e44100a80 2024-09-12T12:16:10.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.402+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e441a5000 con 0x7f3e44100a80 2024-09-12T12:16:10.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.403+0000 7f3e41ffb640 1 -- 172.21.15.137:0/1161699605 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f3e340bc0a0 con 0x7f3e44100a80 2024-09-12T12:16:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:16:10.403 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":27,"btime":"2024-09-12T12:14:37:790728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:14:36.964904+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:16:10.404 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 27 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3e300a2cf0 msgr2=0x7f3e300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3e300a2cf0 0x7f3e300a51b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e4406b2e0 tx=0x7f3e2c00a2d0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100a80 msgr2=0x7f3e44199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100a80 0x7f3e44199250 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009a50 tx=0x7f3e34004270 comp rx=0 tx=0).stop 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 shutdown_connections 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3e300a2cf0 0x7f3e300a51b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e44106a80 0x7f3e441a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e44101420 0x7f3e44199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 --2- 172.21.15.137:0/1161699605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e44100a80 0x7f3e44199250 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:10.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 >> 172.21.15.137:0/1161699605 conn(0x7f3e440fc7a0 msgr2=0x7f3e440fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:10.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 shutdown_connections 2024-09-12T12:16:10.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:10.405+0000 7f3e4a7bb640 1 -- 172.21.15.137:0/1161699605 wait complete. 2024-09-12T12:16:10.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.35 MB/sec warmup 105 sec latency 23220.366 ms 2024-09-12T12:16:11.268 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 9a1af08d --release --await' 2024-09-12T12:16:11.268 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 9a1af08d --release --await 2024-09-12T12:16:11.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:11 smithi171 ceph-mon[35098]: pgmap v92: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.6 MiB/s wr, 119 op/s 2024-09-12T12:16:11.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:11 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1161699605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:16:11.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:11 smithi137 ceph-mon[30950]: pgmap v92: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.6 MiB/s wr, 119 op/s 2024-09-12T12:16:11.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:11 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1161699605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:16:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:16:11.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:11 smithi189 ceph-mon[35475]: pgmap v92: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.6 MiB/s wr, 119 op/s 2024-09-12T12:16:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:11 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1161699605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:16:11.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 178837 9.27 MB/sec warmup 106 sec latency 24220.422 ms 2024-09-12T12:16:12.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 -- 172.21.15.137:0/1596610323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 msgr2=0x7f0f341067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 --2- 172.21.15.137:0/1596610323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f341067e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c009900 tx=0x7f0f1c02a410 comp rx=0 tx=0).stop 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 -- 172.21.15.137:0/1596610323 shutdown_connections 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 --2- 172.21.15.137:0/1596610323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f34106f10 0x7f0f3410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 --2- 172.21.15.137:0/1596610323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f341067e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 --2- 172.21.15.137:0/1596610323 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f34101de0 0x7f0f341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 -- 172.21.15.137:0/1596610323 >> 172.21.15.137:0/1596610323 conn(0x7f0f340fd8c0 msgr2=0x7f0f340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 -- 172.21.15.137:0/1596610323 shutdown_connections 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.040+0000 7f0f399cc640 1 -- 172.21.15.137:0/1596610323 wait complete. 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 Processor -- start 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 -- start start 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f34101de0 0x7f0f34198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f34198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f34106f10 0x7f0f3419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34111f90 con 0x7f0f34102790 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34111e10 con 0x7f0f34101de0 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f399cc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112110 con 0x7f0f34106f10 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f34198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f34198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45698/0 (socket says 172.21.15.137:45698) 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.041+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4172403342 learned_addr learned my addr 172.21.15.137:0/4172403342 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f34101de0 0x7f0f34198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f337fe640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f34106f10 0x7f0f3419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4172403342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f34106f10 msgr2=0x7f0f3419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f34106f10 0x7f0f3419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4172403342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f34101de0 msgr2=0x7f0f34198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f34101de0 0x7f0f34198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4172403342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3419d3d0 con 0x7f0f34102790 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f34198960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005ab0 tx=0x7f0f1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f389ca640 1 -- 172.21.15.137:0/4172403342 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c03f030 con 0x7f0f34102790 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3419d660 con 0x7f0f34102790 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f389ca640 1 -- 172.21.15.137:0/4172403342 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f1c003ce0 con 0x7f0f34102790 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f389ca640 1 -- 172.21.15.137:0/4172403342 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c035070 con 0x7f0f34102790 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.042+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f341a4fa0 con 0x7f0f34102790 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.044+0000 7f0f389ca640 1 -- 172.21.15.137:0/4172403342 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0f1c042030 con 0x7f0f34102790 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 Processor -- start 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 -- start start 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9af440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bf9bc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5559bf914620 con 0x5559bf9af060 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5559bf8bbe50 con 0x5559bf9ba240 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5559bf9105e0 con 0x5559bf9af980 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f389ca640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0f200a2cf0 0x7f0f200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45714/0 (socket says 172.21.15.137:45714) 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 -- 172.21.15.137:0/2735805312 learned_addr learned my addr 172.21.15.137:0/2735805312 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:12.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0f200a2cf0 0x7f0f200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bf9bc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 -- 172.21.15.137:0/2735805312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 msgr2=0x5559bf9b9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 -- 172.21.15.137:0/2735805312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 msgr2=0x5559bf9bc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bf9bc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.045+0000 7f0f337fe640 1 -- 172.21.15.137:0/2735805312 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bf8acd40 con 0x5559bf9af060 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.046+0000 7f0f389ca640 1 -- 172.21.15.137:0/4172403342 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f0f1c0efda0 con 0x7f0f34102790 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.046+0000 7f0f337fe640 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9af440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00e920 tx=0x7f0f2c00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cb1dba64f0774d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.046+0000 7f0f29ffb640 1 -- 172.21.15.137:0/2735805312 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c01e020 con 0x5559bf9af060 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.046+0000 7f0f29ffb640 1 -- 172.21.15.137:0/2735805312 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c01acd0 con 0x5559bf9af060 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.046+0000 7f0f29ffb640 1 -- 172.21.15.137:0/2735805312 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0228e0 con 0x5559bf9af060 2024-09-12T12:16:12.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2735805312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 msgr2=0x5559bf9af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f337fe640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0f200a2cf0 0x7f0f200a51b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c01f000 tx=0x7f0f2c01b000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9af440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c00e920 tx=0x7f0f2c00ede0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2735805312 shutdown_connections 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bf9bc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2735805312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9af440 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2735805312 >> 172.21.15.137:0/2735805312 conn(0x5559bf9ab5f0 msgr2=0x5559bf9ab9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2735805312 shutdown_connections 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2735805312 wait complete. 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- start start 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 Processor -- start 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af980 0x5559bf9b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9ba240 0x5559bf9b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5559bf9bce10 con 0x5559bf9af980 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5559bf89cf10 con 0x5559bf9af060 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.047+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5559bf9bcf90 con 0x5559bf9ba240 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f3bda5740 10 client.0 _ll_register_callbacks cb 0x5559bf8c0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af980 0x5559bf9b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af980 0x5559bf9b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45722/0 (socket says 172.21.15.137:45722) 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4085724165 learned_addr learned my addr 172.21.15.137:0/4085724165 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f337fe640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9ba240 0x5559bf9b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4085724165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9ba240 msgr2=0x5559bf9b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9ba240 0x5559bf9b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 msgr2=0x5559bf9b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 -- 172.21.15.137:0/4085724165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bf623580 con 0x5559bf9af980 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af980 0x5559bf9b7650 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0b8770 tx=0x7f0f1c0ba820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c81a71095f85ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f12ffd640 1 -- 172.21.15.137:0/4085724165 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c049070 con 0x5559bf9af980 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f12ffd640 1 -- 172.21.15.137:0/4085724165 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f0f1c0b8980 con 0x5559bf9af980 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f2c01acd0 con 0x5559bf9af980 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f12ffd640 1 -- 172.21.15.137:0/4085724165 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f1c085030 con 0x5559bf9af980 2024-09-12T12:16:12.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f3bda5740 10 client.15741 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.048+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559bf6ed960 con 0x5559bf9af980 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f12ffd640 1 -- 172.21.15.137:0/4085724165 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c01acd0 con 0x5559bf9af980 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f12ffd640 1 -- 172.21.15.137:0/4085724165 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f1c03b460 con 0x5559bf9af980 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f3bda5740 10 client.15741 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f3bda5740 10 client.15741 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f3bda5740 20 client.15741 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bf9cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f3bda5740 4 client.15741 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559bf921b90 con 0x5559bf9c9c70 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bf9cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bf9cc130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22f6b1245822ed0 server_cookie=2fd51c816131364c in_seq=0 out_seq=0 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.049+0000 7f0f12ffd640 10 client.15741 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:12.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f12ffd640 1 -- 172.21.15.137:0/4085724165 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559bf921b90 con 0x5559bf9c9c70 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f12ffd640 10 client.15741 handle_command_reply: tid=0 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 client.15741 shutdown 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af980 msgr2=0x5559bf9b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af980 0x5559bf9b7650 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0b8770 tx=0x7f0f1c0ba820 comp rx=0 tx=0).stop 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 shutdown_connections 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9ba240 0x5559bf9b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bf9cc130 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f12ffd640 0 client.15741 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af980 0x5559bf9b7650 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/4085724165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 >> 172.21.15.137:0/4085724165 conn(0x5559bf9ab5f0 msgr2=0x5559bf9aa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 shutdown_connections 2024-09-12T12:16:12.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 1 -- 172.21.15.137:0/4085724165 wait complete. 2024-09-12T12:16:12.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.050+0000 7f0f3bda5740 20 client.15741 trim_cache size 0 max 16384 2024-09-12T12:16:12.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.067+0000 7f0f3bda5740 1 Processor -- start 2024-09-12T12:16:12.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f3bda5740 1 -- start start 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9ba240 0x5559bf9d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341aff80 con 0x5559bf9ba240 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ad7f0 con 0x5559bf9af060 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ab640 con 0x5559bf9af980 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9ba240 0x5559bf9d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9ba240 0x5559bf9d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45732/0 (socket says 172.21.15.137:45732) 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 -- 172.21.15.137:0/1681908239 learned_addr learned my addr 172.21.15.137:0/1681908239 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 -- 172.21.15.137:0/1681908239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 msgr2=0x5559bf9b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 -- 172.21.15.137:0/1681908239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 msgr2=0x5559bf9b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 -- 172.21.15.137:0/1681908239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bf9bc970 con 0x5559bf9ba240 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.068+0000 7f0f337fe640 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9ba240 0x5559bf9d1a40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0070b0 tx=0x7f0f2c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baede25dc3abba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f297fa640 1 -- 172.21.15.137:0/1681908239 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c022dc0 con 0x5559bf9ba240 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f297fa640 1 -- 172.21.15.137:0/1681908239 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c022890 con 0x5559bf9ba240 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f297fa640 1 -- 172.21.15.137:0/1681908239 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c01aae0 con 0x5559bf9ba240 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f3bda5740 1 -- 172.21.15.137:0/1681908239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9ba240 msgr2=0x5559bf9d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9ba240 0x5559bf9d1a40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0070b0 tx=0x7f0f2c00c000 comp rx=0 tx=0).stop 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f3bda5740 1 -- 172.21.15.137:0/1681908239 shutdown_connections 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9ba240 0x5559bf9d1a40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/1681908239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9af060 0x5559bf9b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.069+0000 7f0f3bda5740 1 -- 172.21.15.137:0/1681908239 >> 172.21.15.137:0/1681908239 conn(0x5559bf9ab5f0 msgr2=0x5559bf9aa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 -- 172.21.15.137:0/1681908239 shutdown_connections 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 -- 172.21.15.137:0/1681908239 wait complete. 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 -- start start 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 Processor -- start 2024-09-12T12:16:12.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bfa33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341abcb0 con 0x5559bf9af060 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ad120 con 0x5559bf9ba240 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ac380 con 0x5559bf9af980 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f3bda5740 10 client.0 _ll_register_callbacks cb 0x7f0f1c0044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.070+0000 7f0f32ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9be6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45742/0 (socket says 172.21.15.137:45742) 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 -- 172.21.15.137:0/2163078629 learned_addr learned my addr 172.21.15.137:0/2163078629 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f337fe640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bfa33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 -- 172.21.15.137:0/2163078629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 msgr2=0x5559bf9bebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9bebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 -- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 msgr2=0x5559bfa33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bfa33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 -- 172.21.15.137:0/2163078629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bfa33fd0 con 0x5559bf9af060 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f337fe640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bfa33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f32ffd640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9be6b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400ca90 tx=0x7f0f2400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e73d84e753223b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f12ffd640 1 -- 172.21.15.137:0/2163078629 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24005030 con 0x5559bf9af060 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f12ffd640 1 -- 172.21.15.137:0/2163078629 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f0f240051d0 con 0x5559bf9af060 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f2c022890 con 0x5559bf9af060 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f12ffd640 1 -- 172.21.15.137:0/2163078629 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0f2402b050 con 0x5559bf9af060 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f3bda5740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:12.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.071+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5559bfa35330 con 0x5559bf9af060 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f12ffd640 1 -- 172.21.15.137:0/2163078629 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f2c022890 con 0x5559bf9af060 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f12ffd640 1 -- 172.21.15.137:0/2163078629 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0f240115d0 con 0x5559bf9af060 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f3bda5740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f3bda5740 10 client.15747 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f3bda5740 20 client.15747 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bfa37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f3bda5740 4 client.15747 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9a1af08d", "release": true, "await": true}] 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9a1af08d", "release": true, "await": true}) -- 0x5559bf91c0a0 con 0x5559bf9c9c70 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bfa37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f2bfff640 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bfa37a70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be84ff45a1156cda server_cookie=703ad1438a17c360 in_seq=0 out_seq=0 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.072+0000 7f0f12ffd640 10 client.15747 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.079+0000 7f0f12ffd640 1 -- 172.21.15.137:0/2163078629 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5559bf91c0a0 con 0x5559bf9c9c70 2024-09-12T12:16:12.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.079+0000 7f0f12ffd640 10 client.15747 handle_command_reply: tid=0 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.079+0000 7f0f3bda5740 1 client.15747 shutdown 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 msgr2=0x5559bf9be6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9be6b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400ca90 tx=0x7f0f2400cf50 comp rx=0 tx=0).stop 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 shutdown_connections 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559bf9ba240 0x5559bfa33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559bf9af980 0x5559bf9bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559bf9c9c70 0x5559bfa37a70 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f12ffd640 0 client.15747 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 --2- 172.21.15.137:0/2163078629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559bf9af060 0x5559bf9be6b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 >> 172.21.15.137:0/2163078629 conn(0x5559bf9ab5f0 msgr2=0x5559bf9b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 shutdown_connections 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 1 -- 172.21.15.137:0/2163078629 wait complete. 2024-09-12T12:16:12.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f3bda5740 20 client.15747 trim_cache size 0 max 16384 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 27, 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 12, 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "9a1af08d": { 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 12, 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:16:12.089 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.7, 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:16:12.090 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:16:12.091 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:16:12.091 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:16:12.091 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:16:12.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0f200a2cf0 msgr2=0x7f0f200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f399cc640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0f200a2cf0 0x7f0f200a51b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c01f000 tx=0x7f0f2c01b000 comp rx=0 tx=0).stop 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 msgr2=0x7f0f34198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.080+0000 7f0f399cc640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f34198960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c005ab0 tx=0x7f0f1c0042f0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 shutdown_connections 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0f200a2cf0 0x7f0f200a51b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f34106f10 0x7f0f3419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f34102790 0x7f0f34198960 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 --2- 172.21.15.137:0/4172403342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f34101de0 0x7f0f34198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 >> 172.21.15.137:0/4172403342 conn(0x7f0f340fd8c0 msgr2=0x7f0f340fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 shutdown_connections 2024-09-12T12:16:12.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:12.081+0000 7f0f399cc640 1 -- 172.21.15.137:0/4172403342 wait complete. 2024-09-12T12:16:12.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 180872 9.29 MB/sec warmup 107 sec latency 24581.784 ms 2024-09-12T12:16:12.966 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9a1af08d', total seconds elapsed: 28.2 2024-09-12T12:16:13.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:13 smithi171 ceph-mon[35098]: pgmap v93: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-12T12:16:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:13 smithi137 ceph-mon[30950]: pgmap v93: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-12T12:16:13.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:13 smithi189 ceph-mon[35475]: pgmap v93: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-12T12:16:13.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 182322 9.27 MB/sec warmup 108 sec latency 49.167 ms 2024-09-12T12:16:14.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 184858 9.31 MB/sec warmup 109 sec latency 65.150 ms 2024-09-12T12:16:15.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:15 smithi171 ceph-mon[35098]: pgmap v94: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-12T12:16:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:15 smithi137 ceph-mon[30950]: pgmap v94: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-12T12:16:15.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:15 smithi189 ceph-mon[35475]: pgmap v94: 97 pgs: 97 active+clean; 63 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-12T12:16:15.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 187428 9.34 MB/sec warmup 110 sec latency 16.967 ms 2024-09-12T12:16:16.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 189237 9.34 MB/sec warmup 111 sec latency 15.526 ms 2024-09-12T12:16:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:17 smithi137 ceph-mon[30950]: pgmap v95: 97 pgs: 97 active+clean; 69 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 178 op/s 2024-09-12T12:16:17.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:17 smithi189 ceph-mon[35475]: pgmap v95: 97 pgs: 97 active+clean; 69 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 178 op/s 2024-09-12T12:16:17.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:17 smithi171 ceph-mon[35098]: pgmap v95: 97 pgs: 97 active+clean; 69 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 178 op/s 2024-09-12T12:16:17.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 191926 9.41 MB/sec warmup 112 sec latency 12.055 ms 2024-09-12T12:16:18.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 194307 9.42 MB/sec warmup 113 sec latency 59.208 ms 2024-09-12T12:16:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:19 smithi137 ceph-mon[30950]: pgmap v96: 97 pgs: 97 active+clean; 69 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 127 op/s 2024-09-12T12:16:19.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:19 smithi189 ceph-mon[35475]: pgmap v96: 97 pgs: 97 active+clean; 69 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 127 op/s 2024-09-12T12:16:19.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:19 smithi171 ceph-mon[35098]: pgmap v96: 97 pgs: 97 active+clean; 69 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 127 op/s 2024-09-12T12:16:19.721 INFO:tasks.workunit.client.0.smithi137.stdout: 1 196155 9.43 MB/sec warmup 114 sec latency 23.245 ms 2024-09-12T12:16:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.958+0000 7f8614caf640 1 -- 172.21.15.137:0/2752679375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86101035a0 msgr2=0x7f861010ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.958+0000 7f8614caf640 1 --2- 172.21.15.137:0/2752679375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86101035a0 0x7f861010ba80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8604009900 tx=0x7f860402a480 comp rx=0 tx=0).stop 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 -- 172.21.15.137:0/2752679375 shutdown_connections 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 --2- 172.21.15.137:0/2752679375 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f861010c0d0 0x7f861010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 --2- 172.21.15.137:0/2752679375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86101035a0 0x7f861010ba80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 --2- 172.21.15.137:0/2752679375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8610102c80 0x7f8610103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 -- 172.21.15.137:0/2752679375 >> 172.21.15.137:0/2752679375 conn(0x7f8610077990 msgr2=0x7f86100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 -- 172.21.15.137:0/2752679375 shutdown_connections 2024-09-12T12:16:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 -- 172.21.15.137:0/2752679375 wait complete. 2024-09-12T12:16:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 Processor -- start 2024-09-12T12:16:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.959+0000 7f8614caf640 1 -- start start 2024-09-12T12:16:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f8614caf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8610102c80 0x7f8610198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f8614caf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86101035a0 0x7f86101988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f8614caf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f861010c0d0 0x7f861019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f8614caf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8610111eb0 con 0x7f8610102c80 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f8614caf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8610111d30 con 0x7f861010c0d0 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f8614caf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8610112030 con 0x7f86101035a0 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8610102c80 0x7f8610198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860dd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86101035a0 0x7f86101988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8610102c80 0x7f8610198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51240/0 (socket says 172.21.15.137:51240) 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860dd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86101035a0 0x7f86101988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:60258/0 (socket says 172.21.15.137:60258) 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 -- 172.21.15.137:0/3248854481 learned_addr learned my addr 172.21.15.137:0/3248854481 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860ed9d640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f861010c0d0 0x7f861019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 -- 172.21.15.137:0/3248854481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86101035a0 msgr2=0x7f86101988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86101035a0 0x7f86101988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 -- 172.21.15.137:0/3248854481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f861010c0d0 msgr2=0x7f861019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f861010c0d0 0x7f861019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 -- 172.21.15.137:0/3248854481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861019d350 con 0x7f8610102c80 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860dd9b640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86101035a0 0x7f86101988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.960+0000 7f860e59c640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8610102c80 0x7f8610198360 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f860000e9f0 tx=0x7f860000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.961+0000 7f85ff7fe640 1 -- 172.21.15.137:0/3248854481 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860000cd80 con 0x7f8610102c80 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.961+0000 7f85ff7fe640 1 -- 172.21.15.137:0/3248854481 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8600005500 con 0x7f8610102c80 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.961+0000 7f85ff7fe640 1 -- 172.21.15.137:0/3248854481 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600010690 con 0x7f8610102c80 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.961+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861019d640 con 0x7f8610102c80 2024-09-12T12:16:19.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.961+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8610100800 con 0x7f8610102c80 2024-09-12T12:16:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.963+0000 7f85ff7fe640 1 -- 172.21.15.137:0/3248854481 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8600010830 con 0x7f8610102c80 2024-09-12T12:16:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.963+0000 7f8617a3e740 1 Processor -- start 2024-09-12T12:16:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.963+0000 7f8617a3e740 1 -- start start 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d74af0 0x562697d74ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d75410 0x562697d7f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d7fcb0 0x562697d820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f8617a3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562697cd92f0 con 0x562697d75410 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f8617a3e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562697c81270 con 0x562697d74af0 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f8617a3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562697cd52b0 con 0x562697d7fcb0 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d74af0 0x562697d74ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860e59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d75410 0x562697d7f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d74af0 0x562697d74ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:46428/0 (socket says 172.21.15.137:46428) 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 -- 172.21.15.137:0/1704430077 learned_addr learned my addr 172.21.15.137:0/1704430077 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860dd9b640 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d7fcb0 0x562697d820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 -- 172.21.15.137:0/1704430077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d7fcb0 msgr2=0x562697d820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d7fcb0 0x562697d820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 -- 172.21.15.137:0/1704430077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d75410 msgr2=0x562697d7f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d75410 0x562697d7f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 -- 172.21.15.137:0/1704430077 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562697c80340 con 0x562697d74af0 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f85ff7fe640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f85f80a71b0 0x7f85f80a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860e59c640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f85f80a71b0 0x7f85f80a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.964+0000 7f860ed9d640 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d74af0 0x562697d74ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8608009f50 tx=0x7f86080118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32aa87bae8c0a453 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.965+0000 7f85ff7fe640 1 -- 172.21.15.137:0/3248854481 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f86000cf850 con 0x7f8610102c80 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.965+0000 7f85fcff9640 1 -- 172.21.15.137:0/1704430077 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608004d20 con 0x562697d74af0 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.965+0000 7f85fcff9640 1 -- 172.21.15.137:0/1704430077 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608004050 con 0x562697d74af0 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.965+0000 7f85fcff9640 1 -- 172.21.15.137:0/1704430077 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608005280 con 0x562697d74af0 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.965+0000 7f8617a3e740 1 -- 172.21.15.137:0/1704430077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d74af0 msgr2=0x562697d74ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.965+0000 7f860e59c640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f85f80a71b0 0x7f85f80a9670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86000cd0a0 tx=0x7f86000d5550 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.965+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d74af0 0x562697d74ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8608009f50 tx=0x7f86080118d0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- 172.21.15.137:0/1704430077 shutdown_connections 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d7fcb0 0x562697d820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d75410 0x562697d7f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1704430077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d74af0 0x562697d74ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- 172.21.15.137:0/1704430077 >> 172.21.15.137:0/1704430077 conn(0x562697d71060 msgr2=0x562697d71440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- 172.21.15.137:0/1704430077 shutdown_connections 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- 172.21.15.137:0/1704430077 wait complete. 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- start start 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 Processor -- start 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d74af0 0x562697d7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d75410 0x562697d7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d7fcb0 0x562697d7d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562697d82880 con 0x562697d7fcb0 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562697c62290 con 0x562697d75410 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562697d82a00 con 0x562697d74af0 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f8617a3e740 10 client.0 _ll_register_callbacks cb 0x562697c93140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f860dd9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d75410 0x562697d7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f860ed9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d74af0 0x562697d7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f860dd9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d75410 0x562697d7d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:46438/0 (socket says 172.21.15.137:46438) 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f860dd9b640 1 -- 172.21.15.137:0/4202321333 learned_addr learned my addr 172.21.15.137:0/4202321333 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.966+0000 7f860ed9d640 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d7fcb0 0x562697d7d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f860dd9b640 1 -- 172.21.15.137:0/4202321333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d74af0 msgr2=0x562697d7cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f860dd9b640 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d74af0 0x562697d7cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f860dd9b640 1 -- 172.21.15.137:0/4202321333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d7fcb0 msgr2=0x562697d7d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f860dd9b640 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d7fcb0 0x562697d7d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f860dd9b640 1 -- 172.21.15.137:0/4202321333 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626979ec210 con 0x562697d75410 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f860ed9d640 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d7fcb0 0x562697d7d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f860dd9b640 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d75410 0x562697d7d0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8604009900 tx=0x7f86040317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480b55149fd165ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f85f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f85f5ffb640 1 -- 172.21.15.137:0/4202321333 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604035040 con 0x562697d75410 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f85f5ffb640 1 -- 172.21.15.137:0/4202321333 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f8604004250 con 0x562697d75410 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8608004050 con 0x562697d75410 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f85f5ffb640 1 -- 172.21.15.137:0/4202321333 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8604004470 con 0x562697d75410 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f8617a3e740 10 client.25232 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.967+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562697ab8620 con 0x562697d75410 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f85f5ffb640 1 -- 172.21.15.137:0/4202321333 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608004050 con 0x562697d75410 2024-09-12T12:16:19.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f85f5ffb640 1 -- 172.21.15.137:0/4202321333 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8604051790 con 0x562697d75410 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f8617a3e740 10 client.25232 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f8617a3e740 10 client.25232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f8617a3e740 20 client.25232 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f8617a3e740 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697d8fc10 0x562697d92000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f8617a3e740 4 client.25232 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562697ce6a20 con 0x562697d8fc10 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.968+0000 7f860ed9d640 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697d8fc10 0x562697d92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.969+0000 7f85f5ffb640 10 client.25232 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.969+0000 7f860ed9d640 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697d8fc10 0x562697d92000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6502722773563cb server_cookie=944d94eb11a3b098 in_seq=0 out_seq=0 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.969+0000 7f85f5ffb640 1 -- 172.21.15.137:0/4202321333 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562697ce6a20 con 0x562697d8fc10 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.969+0000 7f85f5ffb640 10 client.25232 handle_command_reply: tid=0 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.969+0000 7f8617a3e740 1 client.25232 shutdown 2024-09-12T12:16:19.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d75410 msgr2=0x562697d7d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d75410 0x562697d7d0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8604009900 tx=0x7f86040317e0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 shutdown_connections 2024-09-12T12:16:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697d8fc10 0x562697d92000 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d7fcb0 0x562697d7d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d75410 0x562697d7d0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f85f5ffb640 0 client.25232 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 --2- 172.21.15.137:0/4202321333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d74af0 0x562697d7cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 >> 172.21.15.137:0/4202321333 conn(0x562697d71060 msgr2=0x562697d76dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 shutdown_connections 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 1 -- 172.21.15.137:0/4202321333 wait complete. 2024-09-12T12:16:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.970+0000 7f8617a3e740 20 client.25232 trim_cache size 0 max 16384 2024-09-12T12:16:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.985+0000 7f8617a3e740 1 Processor -- start 2024-09-12T12:16:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.985+0000 7f8617a3e740 1 -- start start 2024-09-12T12:16:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.985+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.985+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d97920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d97e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f8617a3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86101addc0 con 0x562697d74af0 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f8617a3e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ab630 con 0x562697d7fcb0 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f8617a3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86101a94a0 con 0x562697d75410 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d7d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51272/0 (socket says 172.21.15.137:51272) 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 -- 172.21.15.137:0/1072215527 learned_addr learned my addr 172.21.15.137:0/1072215527 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860dd9b640 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d97920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d97e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 -- 172.21.15.137:0/1072215527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 msgr2=0x562697d97920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d97920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 -- 172.21.15.137:0/1072215527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 msgr2=0x562697d97e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d97e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 -- 172.21.15.137:0/1072215527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562697d92660 con 0x562697d74af0 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f860ed9d640 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d7d440 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8608009f20 tx=0x7f8608009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba1320d793be60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f85f7fff640 1 -- 172.21.15.137:0/1072215527 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608004050 con 0x562697d74af0 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f85f7fff640 1 -- 172.21.15.137:0/1072215527 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608009b30 con 0x562697d74af0 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.986+0000 7f85f7fff640 1 -- 172.21.15.137:0/1072215527 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080278e0 con 0x562697d74af0 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 -- 172.21.15.137:0/1072215527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 msgr2=0x562697d7d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d7d440 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8608009f20 tx=0x7f8608009500 comp rx=0 tx=0).stop 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 -- 172.21.15.137:0/1072215527 shutdown_connections 2024-09-12T12:16:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d97e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d97920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 --2- 172.21.15.137:0/1072215527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d7d440 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 -- 172.21.15.137:0/1072215527 >> 172.21.15.137:0/1072215527 conn(0x562697d71060 msgr2=0x562697d7ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 -- 172.21.15.137:0/1072215527 shutdown_connections 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.987+0000 7f8617a3e740 1 -- 172.21.15.137:0/1072215527 wait complete. 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 -- start start 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 Processor -- start 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d881d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d84c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86101a9af0 con 0x562697d74af0 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86101aaf60 con 0x562697d7fcb0 2024-09-12T12:16:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86101aa1c0 con 0x562697d75410 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f8617a3e740 10 client.0 _ll_register_callbacks cb 0x7f8600005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860ed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d87c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51288/0 (socket says 172.21.15.137:51288) 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860ed9d640 1 -- 172.21.15.137:0/494225556 learned_addr learned my addr 172.21.15.137:0/494225556 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860dd9b640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d881d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860ed9d640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d84c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860dd9b640 1 -- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 msgr2=0x562697d84c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860dd9b640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d84c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860dd9b640 1 -- 172.21.15.137:0/494225556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 msgr2=0x562697d87c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860dd9b640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d87c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860dd9b640 1 -- 172.21.15.137:0/494225556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562697d88890 con 0x562697d75410 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.988+0000 7f860ed9d640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d87c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f860dd9b640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d881d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8604031490 tx=0x7f86040314e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4434e2b0c74b2514 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f85f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f85f5ffb640 1 -- 172.21.15.137:0/494225556 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604035070 con 0x562697d75410 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f85f5ffb640 1 -- 172.21.15.137:0/494225556 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f8604051890 con 0x562697d75410 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8608009b30 con 0x562697d75410 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f85f5ffb640 1 -- 172.21.15.137:0/494225556 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8604005040 con 0x562697d75410 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f8617a3e740 10 client.25300 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562697dfaed0 con 0x562697d75410 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f85f5ffb640 1 -- 172.21.15.137:0/494225556 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608009b30 con 0x562697d75410 2024-09-12T12:16:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.989+0000 7f85f5ffb640 1 -- 172.21.15.137:0/494225556 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8604004880 con 0x562697d75410 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f8617a3e740 10 client.25300 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f8617a3e740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f8617a3e740 20 client.25300 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f8617a3e740 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697dfb660 0x562697dfda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f8617a3e740 4 client.25300 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562697ce0d70 con 0x562697dfb660 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f860ed9d640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697dfb660 0x562697dfda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f85f5ffb640 10 client.25300 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:19.990+0000 7f860ed9d640 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697dfb660 0x562697dfda80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d5aeab7628c11 server_cookie=ec5ffb05713624b0 in_seq=0 out_seq=0 2024-09-12T12:16:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f85f5ffb640 1 -- 172.21.15.137:0/494225556 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562697ce0d70 con 0x562697dfb660 2024-09-12T12:16:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f85f5ffb640 10 client.25300 handle_command_reply: tid=0 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 client.25300 shutdown 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 msgr2=0x562697d881d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d881d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8604031490 tx=0x7f86040314e0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 shutdown_connections 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562697d7fcb0 0x562697d84c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562697d75410 0x562697d881d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562697dfb660 0x562697dfda80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 --2- 172.21.15.137:0/494225556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562697d74af0 0x562697d87c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f85f5ffb640 0 client.25300 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 >> 172.21.15.137:0/494225556 conn(0x562697d71060 msgr2=0x562697d7e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 shutdown_connections 2024-09-12T12:16:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 1 -- 172.21.15.137:0/494225556 wait complete. 2024-09-12T12:16:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.037+0000 7f8617a3e740 20 client.25300 trim_cache size 0 max 16384 2024-09-12T12:16:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:16:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:16:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:16:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f85f80a71b0 msgr2=0x7f85f80a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f85f80a71b0 0x7f85f80a9670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86000cd0a0 tx=0x7f86000d5550 comp rx=0 tx=0).stop 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8610102c80 msgr2=0x7f8610198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8610102c80 0x7f8610198360 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f860000e9f0 tx=0x7f860000edf0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 shutdown_connections 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f85f80a71b0 0x7f85f80a9670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f861010c0d0 0x7f861019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86101035a0 0x7f86101988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 --2- 172.21.15.137:0/3248854481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8610102c80 0x7f8610198360 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 >> 172.21.15.137:0/3248854481 conn(0x7f8610077990 msgr2=0x7f86100fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 shutdown_connections 2024-09-12T12:16:20.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:20.038+0000 7f8614caf640 1 -- 172.21.15.137:0/3248854481 wait complete. 2024-09-12T12:16:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:16:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:16:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:16:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:16:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:16:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:16:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:16:20.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:16:20.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:16:20.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:16:20.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:16:20.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:16:20.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 198952 9.49 MB/sec warmup 115 sec latency 37.084 ms 2024-09-12T12:16:21.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:21 smithi137 ceph-mon[30950]: pgmap v97: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 261 op/s 2024-09-12T12:16:21.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:21 smithi189 ceph-mon[35475]: pgmap v97: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 261 op/s 2024-09-12T12:16:21.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:21 smithi171 ceph-mon[35098]: pgmap v97: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 261 op/s 2024-09-12T12:16:21.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 201197 9.48 MB/sec warmup 116 sec latency 11.614 ms 2024-09-12T12:16:22.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 203253 9.51 MB/sec warmup 117 sec latency 12.231 ms 2024-09-12T12:16:23.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:23 smithi137 ceph-mon[30950]: pgmap v98: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-12T12:16:23.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:23 smithi189 ceph-mon[35475]: pgmap v98: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-12T12:16:23.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:23 smithi171 ceph-mon[35098]: pgmap v98: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-12T12:16:23.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 206056 9.57 MB/sec warmup 118 sec latency 10.999 ms 2024-09-12T12:16:24.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 208336 9.56 MB/sec warmup 119 sec latency 14.918 ms 2024-09-12T12:16:25.501 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:25 smithi171 ceph-mon[35098]: pgmap v99: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-12T12:16:25.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:25 smithi137 ceph-mon[30950]: pgmap v99: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-12T12:16:25.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:25 smithi189 ceph-mon[35475]: pgmap v99: 97 pgs: 97 active+clean; 70 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-12T12:16:26.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 213183 17.13 MB/sec execute 1 sec latency 12.295 ms 2024-09-12T12:16:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:27 smithi137 ceph-mon[30950]: pgmap v100: 97 pgs: 97 active+clean; 62 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-09-12T12:16:27.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:27 smithi189 ceph-mon[35475]: pgmap v100: 97 pgs: 97 active+clean; 62 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-09-12T12:16:27.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:27 smithi171 ceph-mon[35098]: pgmap v100: 97 pgs: 97 active+clean; 62 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-09-12T12:16:27.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 215643 14.11 MB/sec execute 2 sec latency 20.071 ms 2024-09-12T12:16:28.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 217461 12.75 MB/sec execute 3 sec latency 9.980 ms 2024-09-12T12:16:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:29 smithi137 ceph-mon[30950]: pgmap v101: 97 pgs: 97 active+clean; 62 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-09-12T12:16:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:29 smithi189 ceph-mon[35475]: pgmap v101: 97 pgs: 97 active+clean; 62 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-09-12T12:16:29.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:29 smithi171 ceph-mon[35098]: pgmap v101: 97 pgs: 97 active+clean; 62 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 285 op/s 2024-09-12T12:16:29.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 220209 13.78 MB/sec execute 4 sec latency 52.221 ms 2024-09-12T12:16:30.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 222390 12.52 MB/sec execute 5 sec latency 12.932 ms 2024-09-12T12:16:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:31 smithi137 ceph-mon[30950]: pgmap v102: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.8 MiB/s wr, 401 op/s 2024-09-12T12:16:31.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:31 smithi189 ceph-mon[35475]: pgmap v102: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.8 MiB/s wr, 401 op/s 2024-09-12T12:16:31.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:31 smithi171 ceph-mon[35098]: pgmap v102: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.8 MiB/s wr, 401 op/s 2024-09-12T12:16:31.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 224381 12.53 MB/sec execute 6 sec latency 12.549 ms 2024-09-12T12:16:32.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 227351 13.34 MB/sec execute 7 sec latency 9.990 ms 2024-09-12T12:16:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 -- 172.21.15.137:0/3327485563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 msgr2=0x7f9af4103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 --2- 172.21.15.137:0/3327485563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 0x7f9af4103080 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009900 tx=0x7f9ae402a410 comp rx=0 tx=0).stop 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 -- 172.21.15.137:0/3327485563 shutdown_connections 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 --2- 172.21.15.137:0/3327485563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9af410c0d0 0x7f9af410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 --2- 172.21.15.137:0/3327485563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 0x7f9af410baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 --2- 172.21.15.137:0/3327485563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 0x7f9af4103080 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 -- 172.21.15.137:0/3327485563 >> 172.21.15.137:0/3327485563 conn(0x7f9af40fc6b0 msgr2=0x7f9af40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 -- 172.21.15.137:0/3327485563 shutdown_connections 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.129+0000 7f9af974b640 1 -- 172.21.15.137:0/3327485563 wait complete. 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 Processor -- start 2024-09-12T12:16:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 -- start start 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 0x7f9af4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 0x7f9af4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9af410c0d0 0x7f9af419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4111ed0 con 0x7f9af4102ca0 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4111d50 con 0x7f9af41035c0 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.130+0000 7f9af974b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4112050 con 0x7f9af410c0d0 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 0x7f9af4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9af410c0d0 0x7f9af419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 0x7f9af4198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:43382/0 (socket says 172.21.15.137:43382) 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2998541302 learned_addr learned my addr 172.21.15.137:0/2998541302 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9af410c0d0 0x7f9af419cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54516/0 (socket says 172.21.15.137:54516) 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 0x7f9af4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2998541302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9af410c0d0 msgr2=0x7f9af419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9af410c0d0 0x7f9af419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 msgr2=0x7f9af4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 0x7f9af4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2998541302 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af419d320 con 0x7f9af41035c0 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 0x7f9af4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 0x7f9af4198870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00e9c0 tx=0x7f9adc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af0ff9640 1 -- 172.21.15.137:0/2998541302 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00cd80 con 0x7f9af41035c0 2024-09-12T12:16:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af419d610 con 0x7f9af41035c0 2024-09-12T12:16:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af0ff9640 1 -- 172.21.15.137:0/2998541302 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc0035f0 con 0x7f9af41035c0 2024-09-12T12:16:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af0ff9640 1 -- 172.21.15.137:0/2998541302 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc010690 con 0x7f9af41035c0 2024-09-12T12:16:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.131+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a4ea0 con 0x7f9af41035c0 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.134+0000 7f9af0ff9640 1 -- 172.21.15.137:0/2998541302 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9adc0037a0 con 0x7f9af41035c0 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.134+0000 7f9afc5c0740 1 Processor -- start 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.134+0000 7f9afc5c0740 1 -- start start 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70c1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af0ff9640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ae00a2da0 0x7f9ae00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a3c701b2f0 con 0x55a3c70b6af0 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a3c6fc3270 con 0x55a3c70b7410 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a3c70172b0 con 0x55a3c70c1cb0 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70c1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70c1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:43388/0 (socket says 172.21.15.137:43388) 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53370/0 (socket says 172.21.15.137:53370) 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 -- 172.21.15.137:0/2446011842 learned_addr learned my addr 172.21.15.137:0/2446011842 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af37fe640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ae00a2da0 0x7f9ae00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af0ff9640 1 -- 172.21.15.137:0/2998541302 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f9adc0cf880 con 0x7f9af41035c0 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 -- 172.21.15.137:0/2446011842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 msgr2=0x55a3c70c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 -- 172.21.15.137:0/2446011842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 msgr2=0x55a3c70b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af2ffd640 1 -- 172.21.15.137:0/2446011842 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c6fc2340 con 0x55a3c70b7410 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.135+0000 7f9af37fe640 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.136+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70c1770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4033000 tx=0x7f9ae4005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38ea62b6e444eb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.136+0000 7f9af37fe640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ae00a2da0 0x7f9ae00a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9aec005fd0 tx=0x7f9aec009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.136+0000 7f9ae97fa640 1 -- 172.21.15.137:0/2446011842 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae403f070 con 0x55a3c70b7410 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.136+0000 7f9ae97fa640 1 -- 172.21.15.137:0/2446011842 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae4004030 con 0x55a3c70b7410 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.136+0000 7f9ae97fa640 1 -- 172.21.15.137:0/2446011842 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4035070 con 0x55a3c70b7410 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2446011842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 msgr2=0x55a3c70c1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70c1770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4033000 tx=0x7f9ae4005e60 comp rx=0 tx=0).stop 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2446011842 shutdown_connections 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70c1770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2446011842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2446011842 >> 172.21.15.137:0/2446011842 conn(0x55a3c70b3060 msgr2=0x55a3c70b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2446011842 shutdown_connections 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2446011842 wait complete. 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- start start 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 Processor -- start 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b6af0 0x55a3c70beb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b7410 0x55a3c70bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a3c70c4880 con 0x55a3c70b7410 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a3c6fa4290 con 0x55a3c70c1cb0 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a3c70c4a00 con 0x55a3c70b6af0 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9afc5c0740 10 client.0 _ll_register_callbacks cb 0x55a3c6fd5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.137+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b7410 0x55a3c70bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b7410 0x55a3c70bf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53384/0 (socket says 172.21.15.137:53384) 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 -- 172.21.15.137:0/3507263584 learned_addr learned my addr 172.21.15.137:0/3507263584 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b6af0 0x55a3c70beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9af37fe640 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 -- 172.21.15.137:0/3507263584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b6af0 msgr2=0x55a3c70beb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b6af0 0x55a3c70beb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 -- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70c1cb0 msgr2=0x55a3c70bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 -- 172.21.15.137:0/3507263584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c6d2e210 con 0x55a3c70b7410 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b7410 0x55a3c70bf050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9adc098250 tx=0x7f9adc01da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87009ff1d8092654 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9ada7fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9ada7fc640 1 -- 172.21.15.137:0/3507263584 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0af030 con 0x55a3c70b7410 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9ada7fc640 1 -- 172.21.15.137:0/3507263584 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f9adc0cd2a0 con 0x55a3c70b7410 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae4004030 con 0x55a3c70b7410 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9ada7fc640 1 -- 172.21.15.137:0/3507263584 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9adc0a9050 con 0x55a3c70b7410 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9afc5c0740 10 client.15777 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3c6dfa620 con 0x55a3c70b7410 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.138+0000 7f9ada7fc640 1 -- 172.21.15.137:0/3507263584 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae4004030 con 0x55a3c70b7410 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9ada7fc640 1 -- 172.21.15.137:0/3507263584 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9adc092070 con 0x55a3c70b7410 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9afc5c0740 10 client.15777 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9afc5c0740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9afc5c0740 20 client.15777 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c70d1be0 0x55a3c70d3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9afc5c0740 4 client.15777 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3c7028a20 con 0x55a3c70d1be0 2024-09-12T12:16:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c70d1be0 0x55a3c70d3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9ada7fc640 10 client.15777 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.139+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c70d1be0 0x55a3c70d3fd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b41bca8448b824a server_cookie=e17f5d005b6b9b70 in_seq=0 out_seq=0 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9ada7fc640 1 -- 172.21.15.137:0/3507263584 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3c7028a20 con 0x55a3c70d1be0 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9ada7fc640 10 client.15777 handle_command_reply: tid=0 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 client.15777 shutdown 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b7410 msgr2=0x55a3c70bf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b7410 0x55a3c70bf050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9adc098250 tx=0x7f9adc01da50 comp rx=0 tx=0).stop 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 shutdown_connections 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c70d1be0 0x55a3c70d3fd0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b7410 0x55a3c70bf050 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/3507263584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b6af0 0x55a3c70beb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9ada7fc640 0 client.15777 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 >> 172.21.15.137:0/3507263584 conn(0x55a3c70b3060 msgr2=0x55a3c70b8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 shutdown_connections 2024-09-12T12:16:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 1 -- 172.21.15.137:0/3507263584 wait complete. 2024-09-12T12:16:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.140+0000 7f9afc5c0740 20 client.15777 trim_cache size 0 max 16384 2024-09-12T12:16:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.155+0000 7f9afc5c0740 1 Processor -- start 2024-09-12T12:16:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.155+0000 7f9afc5c0740 1 -- start start 2024-09-12T12:16:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b6af0 0x55a3c70b95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b7410 0x55a3c70bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70c1cb0 0x55a3c70d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41aff20 con 0x55a3c70c1cb0 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ad790 con 0x55a3c70b6af0 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ab5e0 con 0x55a3c70b7410 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70c1cb0 0x55a3c70d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b7410 0x55a3c70bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b6af0 0x55a3c70b95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b7410 0x55a3c70bf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54536/0 (socket says 172.21.15.137:54536) 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b6af0 0x55a3c70b95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:43412/0 (socket says 172.21.15.137:43412) 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2982437524 learned_addr learned my addr 172.21.15.137:0/2982437524 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70c1cb0 0x55a3c70d98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53390/0 (socket says 172.21.15.137:53390) 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2982437524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b6af0 msgr2=0x55a3c70b95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b6af0 0x55a3c70b95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2982437524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70c1cb0 msgr2=0x55a3c70d98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70c1cb0 0x55a3c70d98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 -- 172.21.15.137:0/2982437524 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c70b2e90 con 0x55a3c70b7410 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9af37fe640 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70c1cb0 0x55a3c70d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.156+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b7410 0x55a3c70bf3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9adc012020 tx=0x7f9adc0abb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b80d4f320362446f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.157+0000 7f9ae8ff9640 1 -- 172.21.15.137:0/2982437524 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0af070 con 0x55a3c70b7410 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.157+0000 7f9ae8ff9640 1 -- 172.21.15.137:0/2982437524 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc096710 con 0x55a3c70b7410 2024-09-12T12:16:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.157+0000 7f9ae8ff9640 1 -- 172.21.15.137:0/2982437524 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc09b240 con 0x55a3c70b7410 2024-09-12T12:16:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.157+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2982437524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b7410 msgr2=0x55a3c70bf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.157+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b7410 0x55a3c70bf3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9adc012020 tx=0x7f9adc0abb20 comp rx=0 tx=0).stop 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2982437524 shutdown_connections 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70c1cb0 0x55a3c70d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70b7410 0x55a3c70bf3c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/2982437524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b6af0 0x55a3c70b95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2982437524 >> 172.21.15.137:0/2982437524 conn(0x55a3c70b3060 msgr2=0x55a3c70c0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2982437524 shutdown_connections 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- 172.21.15.137:0/2982437524 wait complete. 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- start start 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 Processor -- start 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41abc50 con 0x55a3c70b6af0 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ad0c0 con 0x55a3c70b7410 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ac320 con 0x55a3c70c1cb0 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9afc5c0740 10 client.0 _ll_register_callbacks cb 0x7f9adc09bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9af2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9aea7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70bb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:43418/0 (socket says 172.21.15.137:43418) 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9aea7fc640 1 -- 172.21.15.137:0/1535333162 learned_addr learned my addr 172.21.15.137:0/1535333162 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.158+0000 7f9af37fe640 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9af2ffd640 1 -- 172.21.15.137:0/1535333162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 msgr2=0x55a3c70bbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9af2ffd640 1 -- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 msgr2=0x55a3c70bb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70bb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9af2ffd640 1 -- 172.21.15.137:0/1535333162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c70bc620 con 0x55a3c70b6af0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9af2ffd640 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70ca3f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4006010 tx=0x7f9ae4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31081e4653d78979 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9ada7fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9ada7fc640 1 -- 172.21.15.137:0/1535333162 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4035050 con 0x55a3c70b6af0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9ada7fc640 1 -- 172.21.15.137:0/1535333162 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f9ae40315b0 con 0x55a3c70b6af0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9adc096710 con 0x55a3c70b6af0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9ada7fc640 1 -- 172.21.15.137:0/1535333162 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ae405c050 con 0x55a3c70b6af0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a3c713c920 con 0x55a3c70b6af0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9ada7fc640 1 -- 172.21.15.137:0/1535333162 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc096710 con 0x55a3c70b6af0 2024-09-12T12:16:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9ada7fc640 1 -- 172.21.15.137:0/1535333162 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9ae4003710 con 0x55a3c70b6af0 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 20 client.15783 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c713cd20 0x55a3c713f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 4 client.15783 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.159+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3c7022d70 con 0x55a3c713cd20 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.160+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c713cd20 0x55a3c713f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.160+0000 7f9ada7fc640 10 client.15783 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.160+0000 7f9aea7fc640 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c713cd20 0x55a3c713f110 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=113dc4fa97a7733f server_cookie=b0c726020206c519 in_seq=0 out_seq=0 2024-09-12T12:16:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9ada7fc640 1 -- 172.21.15.137:0/1535333162 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3c7022d70 con 0x55a3c713cd20 2024-09-12T12:16:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9ada7fc640 10 client.15783 handle_command_reply: tid=0 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 client.15783 shutdown 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 msgr2=0x55a3c70ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70ca3f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4006010 tx=0x7f9ae4031400 comp rx=0 tx=0).stop 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 shutdown_connections 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a3c70c1cb0 0x55a3c70bbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a3c70b7410 0x55a3c70bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a3c713cd20 0x55a3c713f110 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 --2- 172.21.15.137:0/1535333162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a3c70b6af0 0x55a3c70ca3f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9ada7fc640 0 client.15783 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 >> 172.21.15.137:0/1535333162 conn(0x55a3c70b3060 msgr2=0x55a3c70b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 shutdown_connections 2024-09-12T12:16:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 1 -- 172.21.15.137:0/1535333162 wait complete. 2024-09-12T12:16:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.219+0000 7f9afc5c0740 20 client.15783 trim_cache size 0 max 16384 2024-09-12T12:16:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:16:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:16:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:16:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ae00a2da0 msgr2=0x7f9ae00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ae00a2da0 0x7f9ae00a5260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9aec005fd0 tx=0x7f9aec009500 comp rx=0 tx=0).stop 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 msgr2=0x7f9af4198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 0x7f9af4198870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00e9c0 tx=0x7f9adc00ee80 comp rx=0 tx=0).stop 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 shutdown_connections 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ae00a2da0 0x7f9ae00a5260 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9af410c0d0 0x7f9af419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9af41035c0 0x7f9af4198870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 --2- 172.21.15.137:0/2998541302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9af4102ca0 0x7f9af4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 >> 172.21.15.137:0/2998541302 conn(0x7f9af40fc6b0 msgr2=0x7f9af40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 shutdown_connections 2024-09-12T12:16:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:33.220+0000 7f9af974b640 1 -- 172.21.15.137:0/2998541302 wait complete. 2024-09-12T12:16:33.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:33 smithi137 ceph-mon[30950]: pgmap v103: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-09-12T12:16:33.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:33 smithi189 ceph-mon[35475]: pgmap v103: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-09-12T12:16:33.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:33 smithi171 ceph-mon[35098]: pgmap v103: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-09-12T12:16:33.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 229504 12.61 MB/sec execute 8 sec latency 39.625 ms 2024-09-12T12:16:34.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 231488 12.61 MB/sec execute 9 sec latency 13.508 ms 2024-09-12T12:16:35.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:35 smithi137 ceph-mon[30950]: pgmap v104: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-09-12T12:16:35.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:35 smithi189 ceph-mon[35475]: pgmap v104: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-09-12T12:16:35.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:35 smithi171 ceph-mon[35098]: pgmap v104: 97 pgs: 97 active+clean; 70 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 267 op/s 2024-09-12T12:16:35.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 234371 13.16 MB/sec execute 10 sec latency 10.384 ms 2024-09-12T12:16:36.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 236555 12.64 MB/sec execute 11 sec latency 12.037 ms 2024-09-12T12:16:37.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:37 smithi137 ceph-mon[30950]: pgmap v105: 97 pgs: 97 active+clean; 69 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.3 MiB/s wr, 422 op/s 2024-09-12T12:16:37.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:37 smithi189 ceph-mon[35475]: pgmap v105: 97 pgs: 97 active+clean; 69 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.3 MiB/s wr, 422 op/s 2024-09-12T12:16:37.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:37 smithi171 ceph-mon[35098]: pgmap v105: 97 pgs: 97 active+clean; 69 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.3 MiB/s wr, 422 op/s 2024-09-12T12:16:37.722 INFO:tasks.workunit.client.0.smithi137.stdout: 1 238599 12.65 MB/sec execute 12 sec latency 12.625 ms 2024-09-12T12:16:38.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 241517 13.07 MB/sec execute 13 sec latency 9.395 ms 2024-09-12T12:16:39.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:39 smithi137 ceph-mon[30950]: pgmap v106: 97 pgs: 97 active+clean; 69 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 270 op/s 2024-09-12T12:16:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:39 smithi189 ceph-mon[35475]: pgmap v106: 97 pgs: 97 active+clean; 69 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 270 op/s 2024-09-12T12:16:39.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:39 smithi171 ceph-mon[35098]: pgmap v106: 97 pgs: 97 active+clean; 69 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 270 op/s 2024-09-12T12:16:39.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 243680 12.67 MB/sec execute 14 sec latency 12.425 ms 2024-09-12T12:16:40.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 245699 12.67 MB/sec execute 15 sec latency 12.490 ms 2024-09-12T12:16:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:41 smithi137 ceph-mon[30950]: pgmap v107: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-09-12T12:16:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:41 smithi189 ceph-mon[35475]: pgmap v107: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-09-12T12:16:41.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:41 smithi171 ceph-mon[35098]: pgmap v107: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 378 op/s 2024-09-12T12:16:41.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 248609 13.01 MB/sec execute 16 sec latency 12.065 ms 2024-09-12T12:16:42.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 250924 12.73 MB/sec execute 17 sec latency 18.471 ms 2024-09-12T12:16:43.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:43 smithi137 ceph-mon[30950]: pgmap v108: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-09-12T12:16:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:43 smithi189 ceph-mon[35475]: pgmap v108: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-09-12T12:16:43.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:43 smithi171 ceph-mon[35098]: pgmap v108: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-09-12T12:16:43.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 252782 12.68 MB/sec execute 18 sec latency 32.079 ms 2024-09-12T12:16:44.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 255664 12.97 MB/sec execute 19 sec latency 21.099 ms 2024-09-12T12:16:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:45 smithi137 ceph-mon[30950]: pgmap v109: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-09-12T12:16:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:45 smithi189 ceph-mon[35475]: pgmap v109: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-09-12T12:16:45.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:45 smithi171 ceph-mon[35098]: pgmap v109: 97 pgs: 97 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 262 op/s 2024-09-12T12:16:45.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 257823 12.68 MB/sec execute 20 sec latency 20.483 ms 2024-09-12T12:16:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.309+0000 7f326da37640 1 -- 172.21.15.137:0/2943910724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 msgr2=0x7f3268100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.309+0000 7f326da37640 1 --2- 172.21.15.137:0/2943910724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268100e10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f325c009930 tx=0x7f325c02a420 comp rx=0 tx=0).stop 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 -- 172.21.15.137:0/2943910724 shutdown_connections 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 --2- 172.21.15.137:0/2943910724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3268101350 0x7f326810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 --2- 172.21.15.137:0/2943910724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268100e10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 --2- 172.21.15.137:0/2943910724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32681069c0 0x7f3268106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 -- 172.21.15.137:0/2943910724 >> 172.21.15.137:0/2943910724 conn(0x7f3268077990 msgr2=0x7f32680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 -- 172.21.15.137:0/2943910724 shutdown_connections 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 -- 172.21.15.137:0/2943910724 wait complete. 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 Processor -- start 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.310+0000 7f326da37640 1 -- start start 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3268101350 0x7f32681996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32681069c0 0x7f32681a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3268110e50 con 0x7f32681009b0 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3268110cd0 con 0x7f3268101350 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3268110fd0 con 0x7f32681069c0 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36026/0 (socket says 172.21.15.137:36026) 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 -- 172.21.15.137:0/3886462216 learned_addr learned my addr 172.21.15.137:0/3886462216 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f32677fe640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32681069c0 0x7f32681a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f32667fc640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3268101350 0x7f32681996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 -- 172.21.15.137:0/3886462216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32681069c0 msgr2=0x7f32681a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32681069c0 0x7f32681a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 -- 172.21.15.137:0/3886462216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3268101350 msgr2=0x7f32681996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3268101350 0x7f32681996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 -- 172.21.15.137:0/3886462216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32681a4720 con 0x7f32681009b0 2024-09-12T12:16:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f3266ffd640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268199160 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f325800e9f0 tx=0x7f325800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326ca35640 1 -- 172.21.15.137:0/3886462216 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325800cd80 con 0x7f32681009b0 2024-09-12T12:16:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326ca35640 1 -- 172.21.15.137:0/3886462216 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3258005500 con 0x7f32681009b0 2024-09-12T12:16:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32681a48f0 con 0x7f32681009b0 2024-09-12T12:16:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326ca35640 1 -- 172.21.15.137:0/3886462216 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258010690 con 0x7f32681009b0 2024-09-12T12:16:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.311+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32681a4dc0 con 0x7f32681009b0 2024-09-12T12:16:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f326ca35640 1 -- 172.21.15.137:0/3886462216 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f32580046c0 con 0x7f32681009b0 2024-09-12T12:16:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 Processor -- start 2024-09-12T12:16:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 -- start start 2024-09-12T12:16:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623fa9f6af0 0x5623fa9f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa01770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623faa01cb0 0x5623faa040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5623fa95b2f0 con 0x5623fa9f6af0 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5623fa903270 con 0x5623fa9f7410 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32707ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623fa9572b0 con 0x5623faa01cb0 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623fa9f6af0 0x5623fa9f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f3266ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa01770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32667fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623faa01cb0 0x5623faa040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623fa9f6af0 0x5623fa9f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36032/0 (socket says 172.21.15.137:36032) 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f32677fe640 1 -- 172.21.15.137:0/2527967305 learned_addr learned my addr 172.21.15.137:0/2527967305 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.314+0000 7f326ca35640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f32500a2da0 0x7f32500a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32677fe640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f32500a2da0 0x7f32500a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32667fc640 1 -- 172.21.15.137:0/2527967305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 msgr2=0x5623faa01770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32667fc640 1 --2- 172.21.15.137:0/2527967305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa01770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32667fc640 1 -- 172.21.15.137:0/2527967305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623fa9f6af0 msgr2=0x5623fa9f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32667fc640 1 --2- 172.21.15.137:0/2527967305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623fa9f6af0 0x5623fa9f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32667fc640 1 -- 172.21.15.137:0/2527967305 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623fa902340 con 0x5623faa01cb0 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f326ca35640 1 -- 172.21.15.137:0/3886462216 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f32580cf7b0 con 0x7f32681009b0 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32667fc640 1 --2- 172.21.15.137:0/2527967305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623faa01cb0 0x5623faa040a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f325c02a940 tx=0x7f325c0040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3edcd38a2c15c951 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f32677fe640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f32500a2da0 0x7f32500a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3260007fd0 tx=0x7f3260004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f324dffb640 1 -- 172.21.15.137:0/2527967305 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c035040 con 0x5623faa01cb0 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f324dffb640 1 -- 172.21.15.137:0/2527967305 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f325c005520 con 0x5623faa01cb0 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.315+0000 7f324dffb640 1 -- 172.21.15.137:0/2527967305 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c0035f0 con 0x5623faa01cb0 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 -- 172.21.15.137:0/2527967305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623faa01cb0 msgr2=0x5623faa040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 --2- 172.21.15.137:0/2527967305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623faa01cb0 0x5623faa040a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f325c02a940 tx=0x7f325c0040f0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 -- 172.21.15.137:0/2527967305 shutdown_connections 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 --2- 172.21.15.137:0/2527967305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623faa01cb0 0x5623faa040a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 --2- 172.21.15.137:0/2527967305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa01770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 --2- 172.21.15.137:0/2527967305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623fa9f6af0 0x5623fa9f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 -- 172.21.15.137:0/2527967305 >> 172.21.15.137:0/2527967305 conn(0x5623fa9f3060 msgr2=0x5623fa9f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 -- 172.21.15.137:0/2527967305 shutdown_connections 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 -- 172.21.15.137:0/2527967305 wait complete. 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.316+0000 7f32707ce740 1 -- start start 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 1 Processor -- start 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623fa9feb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623fa9ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623fa9ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5623faa04880 con 0x5623faa01cb0 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5623fa8e4290 con 0x5623fa9f6af0 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5623faa04a00 con 0x5623fa9f7410 2024-09-12T12:16:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32707ce740 10 client.0 _ll_register_callbacks cb 0x5623fa915140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623fa9ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623fa9ff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f3266ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623fa9feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623fa9ff0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:38050/0 (socket says 172.21.15.137:38050) 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 -- 172.21.15.137:0/1415095093 learned_addr learned my addr 172.21.15.137:0/1415095093 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32677fe640 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623fa9ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36042/0 (socket says 172.21.15.137:36042) 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 -- 172.21.15.137:0/1415095093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 msgr2=0x5623fa9feb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623fa9feb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 -- 172.21.15.137:0/1415095093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 msgr2=0x5623fa9ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623fa9ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 -- 172.21.15.137:0/1415095093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623fa66e210 con 0x5623fa9f7410 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32677fe640 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623fa9ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f32667fc640 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623fa9ff0a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f325c031550 tx=0x7f325c02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e60025fe8f7ff520 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.317+0000 7f3246ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f3246ffd640 1 -- 172.21.15.137:0/1415095093 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c053060 con 0x5623fa9f7410 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f3246ffd640 1 -- 172.21.15.137:0/1415095093 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f325c004950 con 0x5623fa9f7410 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f325c005520 con 0x5623fa9f7410 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f3246ffd640 1 -- 172.21.15.137:0/1415095093 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f325c035050 con 0x5623fa9f7410 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 10 client.25330 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623fa73a620 con 0x5623fa9f7410 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f3246ffd640 1 -- 172.21.15.137:0/1415095093 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f325c005520 con 0x5623fa9f7410 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f3246ffd640 1 -- 172.21.15.137:0/1415095093 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f325c003b90 con 0x5623fa9f7410 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 10 client.25330 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 10 client.25330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 20 client.25330 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa11a50 0x5623faa13e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 4 client.25330 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623fa968a20 con 0x5623faa11a50 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.318+0000 7f3266ffd640 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa11a50 0x5623faa13e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.319+0000 7f3266ffd640 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa11a50 0x5623faa13e40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad36d8c0a2c7e802 server_cookie=1a384b0a62ac6d15 in_seq=0 out_seq=0 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.319+0000 7f3246ffd640 10 client.25330 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.319+0000 7f3246ffd640 1 -- 172.21.15.137:0/1415095093 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623fa968a20 con 0x5623faa11a50 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.319+0000 7f3246ffd640 10 client.25330 handle_command_reply: tid=0 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.319+0000 7f32707ce740 1 client.25330 shutdown 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.319+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 msgr2=0x5623fa9ff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.319+0000 7f32707ce740 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623fa9ff0a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f325c031550 tx=0x7f325c02a420 comp rx=0 tx=0).stop 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 shutdown_connections 2024-09-12T12:16:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa11a50 0x5623faa13e40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623fa9ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623fa9ff0a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f3246ffd640 0 client.25330 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 --2- 172.21.15.137:0/1415095093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623fa9feb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 >> 172.21.15.137:0/1415095093 conn(0x5623fa9f3060 msgr2=0x5623fa9f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 shutdown_connections 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 1 -- 172.21.15.137:0/1415095093 wait complete. 2024-09-12T12:16:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.320+0000 7f32707ce740 20 client.25330 trim_cache size 0 max 16384 2024-09-12T12:16:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 Processor -- start 2024-09-12T12:16:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 -- start start 2024-09-12T12:16:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f6af0 0x5623fa9ff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa00ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32681aff00 con 0x5623faa01cb0 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f32681ad770 con 0x5623fa9f7410 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32707ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32681ab5c0 con 0x5623fa9f6af0 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f6af0 0x5623fa9ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f6af0 0x5623fa9ff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:38062/0 (socket says 172.21.15.137:38062) 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 -- 172.21.15.137:0/1894998870 learned_addr learned my addr 172.21.15.137:0/1894998870 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32667fc640 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa00ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 -- 172.21.15.137:0/1894998870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 msgr2=0x5623faa00ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa00ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 -- 172.21.15.137:0/1894998870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 msgr2=0x5623faa19410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa19410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 -- 172.21.15.137:0/1894998870 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623fa9f7240 con 0x5623fa9f6af0 2024-09-12T12:16:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f32677fe640 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.335+0000 7f3266ffd640 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f6af0 0x5623fa9ff410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3258004c00 tx=0x7f32580d1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6746ba1b0f6257 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f324d7fa640 1 -- 172.21.15.137:0/1894998870 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32580b9070 con 0x5623fa9f6af0 2024-09-12T12:16:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f324d7fa640 1 -- 172.21.15.137:0/1894998870 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f325809c420 con 0x5623fa9f6af0 2024-09-12T12:16:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f324d7fa640 1 -- 172.21.15.137:0/1894998870 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258098390 con 0x5623fa9f6af0 2024-09-12T12:16:46.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 -- 172.21.15.137:0/1894998870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f6af0 msgr2=0x5623fa9ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f6af0 0x5623fa9ff410 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3258004c00 tx=0x7f32580d1040 comp rx=0 tx=0).stop 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 -- 172.21.15.137:0/1894998870 shutdown_connections 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f7410 0x5623faa00ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 --2- 172.21.15.137:0/1894998870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f6af0 0x5623fa9ff410 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 -- 172.21.15.137:0/1894998870 >> 172.21.15.137:0/1894998870 conn(0x5623fa9f3060 msgr2=0x5623faa003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 -- 172.21.15.137:0/1894998870 shutdown_connections 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.336+0000 7f32707ce740 1 -- 172.21.15.137:0/1894998870 wait complete. 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 -- start start 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 Processor -- start 2024-09-12T12:16:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623faa09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623faa0a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa05420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32681abc30 con 0x5623faa01cb0 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f32681ad0a0 con 0x5623fa9f6af0 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32681ac300 con 0x5623fa9f7410 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32707ce740 10 client.0 _ll_register_callbacks cb 0x7f32600079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa05420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32677fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa05420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36050/0 (socket says 172.21.15.137:36050) 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32677fe640 1 -- 172.21.15.137:0/364042867 learned_addr learned my addr 172.21.15.137:0/364042867 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f3266ffd640 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623faa09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.337+0000 7f32667fc640 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623faa0a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32677fe640 1 -- 172.21.15.137:0/364042867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 msgr2=0x5623faa0a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32677fe640 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623faa0a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32677fe640 1 -- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 msgr2=0x5623faa09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32677fe640 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623faa09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32677fe640 1 -- 172.21.15.137:0/364042867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623faa05c50 con 0x5623faa01cb0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32677fe640 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa05420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3260021970 tx=0x7f3260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f8fdad61c9da64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f3246ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f3246ffd640 1 -- 172.21.15.137:0/364042867 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260019070 con 0x5623faa01cb0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f3246ffd640 1 -- 172.21.15.137:0/364042867 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f3260013320 con 0x5623faa01cb0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f325809c420 con 0x5623faa01cb0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f3246ffd640 1 -- 172.21.15.137:0/364042867 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f326000f890 con 0x5623faa01cb0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623faa7c050 con 0x5623faa01cb0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f3246ffd640 1 -- 172.21.15.137:0/364042867 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f325809c420 con 0x5623faa01cb0 2024-09-12T12:16:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f3246ffd640 1 -- 172.21.15.137:0/364042867 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f326001c070 con 0x5623faa01cb0 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 20 client.15801 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa7c830 0x5623faa7ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 4 client.15801 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.338+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623fa962d70 con 0x5623faa7c830 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.339+0000 7f3266ffd640 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa7c830 0x5623faa7ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.339+0000 7f3266ffd640 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa7c830 0x5623faa7ec50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20be7d92f152e8bb server_cookie=a4cfd8797fdd75de in_seq=0 out_seq=0 2024-09-12T12:16:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.339+0000 7f3246ffd640 10 client.15801 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f3246ffd640 1 -- 172.21.15.137:0/364042867 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623fa962d70 con 0x5623faa7c830 2024-09-12T12:16:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f3246ffd640 10 client.15801 handle_command_reply: tid=0 2024-09-12T12:16:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 client.15801 shutdown 2024-09-12T12:16:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 msgr2=0x5623faa05420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa05420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3260021970 tx=0x7f3260004a40 comp rx=0 tx=0).stop 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 shutdown_connections 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5623faa7c830 0x5623faa7ec50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5623faa01cb0 0x5623faa05420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5623fa9f7410 0x5623faa0a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 --2- 172.21.15.137:0/364042867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5623fa9f6af0 0x5623faa09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 >> 172.21.15.137:0/364042867 conn(0x5623fa9f3060 msgr2=0x5623fa9f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f3246ffd640 0 client.15801 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 shutdown_connections 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 1 -- 172.21.15.137:0/364042867 wait complete. 2024-09-12T12:16:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.400+0000 7f32707ce740 20 client.15801 trim_cache size 0 max 16384 2024-09-12T12:16:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:16:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:16:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:16:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f32500a2da0 msgr2=0x7f32500a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f32500a2da0 0x7f32500a5260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3260007fd0 tx=0x7f3260004ab0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 msgr2=0x7f3268199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268199160 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f325800e9f0 tx=0x7f325800edf0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 shutdown_connections 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f32500a2da0 0x7f32500a5260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32681069c0 0x7f32681a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3268101350 0x7f32681996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 --2- 172.21.15.137:0/3886462216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32681009b0 0x7f3268199160 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 >> 172.21.15.137:0/3886462216 conn(0x7f3268077990 msgr2=0x7f3268103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:46.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 shutdown_connections 2024-09-12T12:16:46.405 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:46.401+0000 7f326da37640 1 -- 172.21.15.137:0/3886462216 wait complete. 2024-09-12T12:16:46.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 259724 12.67 MB/sec execute 21 sec latency 12.709 ms 2024-09-12T12:16:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:47 smithi137 ceph-mon[30950]: pgmap v110: 97 pgs: 97 active+clean; 83 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 368 op/s 2024-09-12T12:16:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:47 smithi189 ceph-mon[35475]: pgmap v110: 97 pgs: 97 active+clean; 83 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 368 op/s 2024-09-12T12:16:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:47 smithi171 ceph-mon[35098]: pgmap v110: 97 pgs: 97 active+clean; 83 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 368 op/s 2024-09-12T12:16:47.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 262740 12.94 MB/sec execute 22 sec latency 15.043 ms 2024-09-12T12:16:48.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 264517 12.63 MB/sec execute 23 sec latency 12.450 ms 2024-09-12T12:16:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:49 smithi137 ceph-mon[30950]: pgmap v111: 97 pgs: 97 active+clean; 83 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2024-09-12T12:16:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:49 smithi189 ceph-mon[35475]: pgmap v111: 97 pgs: 97 active+clean; 83 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2024-09-12T12:16:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:49 smithi171 ceph-mon[35098]: pgmap v111: 97 pgs: 97 active+clean; 83 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 213 op/s 2024-09-12T12:16:49.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 266635 12.67 MB/sec execute 24 sec latency 29.139 ms 2024-09-12T12:16:50.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 269721 12.87 MB/sec execute 25 sec latency 20.658 ms 2024-09-12T12:16:51.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:51 smithi137 ceph-mon[30950]: pgmap v112: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2024-09-12T12:16:51.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:51 smithi189 ceph-mon[35475]: pgmap v112: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2024-09-12T12:16:51.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:51 smithi171 ceph-mon[35098]: pgmap v112: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 383 op/s 2024-09-12T12:16:51.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 271428 12.63 MB/sec execute 26 sec latency 13.977 ms 2024-09-12T12:16:52.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 273745 12.68 MB/sec execute 27 sec latency 13.757 ms 2024-09-12T12:16:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:53 smithi137 ceph-mon[30950]: pgmap v113: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-09-12T12:16:53.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:53 smithi189 ceph-mon[35475]: pgmap v113: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-09-12T12:16:53.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:53 smithi171 ceph-mon[35098]: pgmap v113: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-09-12T12:16:53.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 276827 12.86 MB/sec execute 28 sec latency 23.393 ms 2024-09-12T12:16:54.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 278311 12.64 MB/sec execute 29 sec latency 25.222 ms 2024-09-12T12:16:55.501 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:55 smithi171 ceph-mon[35098]: pgmap v114: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-09-12T12:16:55.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:55 smithi137 ceph-mon[30950]: pgmap v114: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-09-12T12:16:55.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:55 smithi189 ceph-mon[35475]: pgmap v114: 97 pgs: 97 active+clean; 81 MiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 274 op/s 2024-09-12T12:16:55.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 280681 12.68 MB/sec execute 30 sec latency 21.438 ms 2024-09-12T12:16:56.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 283415 12.70 MB/sec execute 31 sec latency 13.949 ms 2024-09-12T12:16:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:57 smithi137 ceph-mon[30950]: pgmap v115: 97 pgs: 97 active+clean; 86 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 406 op/s 2024-09-12T12:16:57.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:57 smithi189 ceph-mon[35475]: pgmap v115: 97 pgs: 97 active+clean; 86 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 406 op/s 2024-09-12T12:16:57.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:57 smithi171 ceph-mon[35098]: pgmap v115: 97 pgs: 97 active+clean; 86 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 406 op/s 2024-09-12T12:16:57.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 285290 12.65 MB/sec execute 32 sec latency 12.350 ms 2024-09-12T12:16:58.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 287902 12.69 MB/sec execute 33 sec latency 10.383 ms 2024-09-12T12:16:58.838 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T12:16:58.838 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T12:16:58.839 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:16:59.105 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:16:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 -- 172.21.15.137:0/4010157458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f44101350 msgr2=0x7f7f4410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 --2- 172.21.15.137:0/4010157458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f44101350 0x7f7f4410d880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009cd0 tx=0x7f7f3c02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 -- 172.21.15.137:0/4010157458 shutdown_connections 2024-09-12T12:16:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 --2- 172.21.15.137:0/4010157458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f44101350 0x7f7f4410d880 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 --2- 172.21.15.137:0/4010157458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f441009b0 0x7f7f44100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 --2- 172.21.15.137:0/4010157458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f441069c0 0x7f7f44106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 -- 172.21.15.137:0/4010157458 >> 172.21.15.137:0/4010157458 conn(0x7f7f44077990 msgr2=0x7f7f440fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 -- 172.21.15.137:0/4010157458 shutdown_connections 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.495+0000 7f7f48a74640 1 -- 172.21.15.137:0/4010157458 wait complete. 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 Processor -- start 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 -- start start 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f441009b0 0x7f7f44197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f44101350 0x7f7f44197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f441069c0 0x7f7f4419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44110e00 con 0x7f7f441009b0 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44110c80 con 0x7f7f44101350 2024-09-12T12:16:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f48a74640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f44110f80 con 0x7f7f441069c0 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f4259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f441009b0 0x7f7f44197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f4259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f441009b0 0x7f7f44197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44052/0 (socket says 172.21.15.137:44052) 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f4259c640 1 -- 172.21.15.137:0/3802819007 learned_addr learned my addr 172.21.15.137:0/3802819007 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f441069c0 0x7f7f4419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.496+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f44101350 0x7f7f44197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f4259c640 1 -- 172.21.15.137:0/3802819007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f441069c0 msgr2=0x7f7f4419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f441069c0 0x7f7f4419bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f4259c640 1 -- 172.21.15.137:0/3802819007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f44101350 msgr2=0x7f7f44197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f44101350 0x7f7f44197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f4259c640 1 -- 172.21.15.137:0/3802819007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4419c200 con 0x7f7f441009b0 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f441069c0 0x7f7f4419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f441009b0 0x7f7f44197210 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400e9f0 tx=0x7f7f3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f337fe640 1 -- 172.21.15.137:0/3802819007 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3400cd80 con 0x7f7f441009b0 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f337fe640 1 -- 172.21.15.137:0/3802819007 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f34010910 con 0x7f7f441009b0 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4419c4f0 con 0x7f7f441009b0 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f337fe640 1 -- 172.21.15.137:0/3802819007 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34018990 con 0x7f7f441009b0 2024-09-12T12:16:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.497+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f441a3d80 con 0x7f7f441009b0 2024-09-12T12:16:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.499+0000 7f7f337fe640 1 -- 172.21.15.137:0/3802819007 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7f34010ac0 con 0x7f7f441009b0 2024-09-12T12:16:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 Processor -- start 2024-09-12T12:16:59.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 -- start start 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95d42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ed95292f0 con 0x563ed95c5310 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563ed94d1270 con 0x563ed95d1ec0 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4b817740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563ed95252b0 con 0x563ed95c49f0 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95d1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44058/0 (socket says 172.21.15.137:44058) 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f337fe640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7f2c0a2da0 0x7f7f2c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f4259c640 1 -- 172.21.15.137:0/143536278 learned_addr learned my addr 172.21.15.137:0/143536278 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:59.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f41d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95d42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.500+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f41d9b640 1 -- 172.21.15.137:0/143536278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 msgr2=0x563ed95c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7f2c0a2da0 0x7f7f2c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f41d9b640 1 -- 172.21.15.137:0/143536278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 msgr2=0x563ed95d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f337fe640 1 -- 172.21.15.137:0/3802819007 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f7f340cf690 con 0x7f7f441009b0 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f41d9b640 1 -- 172.21.15.137:0/143536278 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed94d0340 con 0x563ed95d1ec0 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95d42b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009bd0 tx=0x7f7f38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7565f05e9979977b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f30ff9640 1 -- 172.21.15.137:0/143536278 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004230 con 0x563ed95d1ec0 2024-09-12T12:16:59.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f30ff9640 1 -- 172.21.15.137:0/143536278 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f38009e40 con 0x563ed95d1ec0 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7f2c0a2da0 0x7f7f2c0a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009430 tx=0x7f7f3c0317e0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.501+0000 7f7f30ff9640 1 -- 172.21.15.137:0/143536278 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38004db0 con 0x563ed95d1ec0 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 -- 172.21.15.137:0/143536278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 msgr2=0x563ed95d42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95d42b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009bd0 tx=0x7f7f38011810 comp rx=0 tx=0).stop 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 -- 172.21.15.137:0/143536278 shutdown_connections 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95d42b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 --2- 172.21.15.137:0/143536278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 -- 172.21.15.137:0/143536278 >> 172.21.15.137:0/143536278 conn(0x563ed95c1060 msgr2=0x563ed95c1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 -- 172.21.15.137:0/143536278 shutdown_connections 2024-09-12T12:16:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 -- 172.21.15.137:0/143536278 wait complete. 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 -- start start 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.502+0000 7f7f4b817740 1 Processor -- start 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4b817740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ed95d4b20 con 0x563ed95d1ec0 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4b817740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563ed94b2290 con 0x563ed95c49f0 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4b817740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563ed95d4ca0 con 0x563ed95c5310 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4b817740 10 client.0 _ll_register_callbacks cb 0x563ed94e3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f42d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33368/0 (socket says 172.21.15.137:33368) 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 -- 172.21.15.137:0/3832008396 learned_addr learned my addr 172.21.15.137:0/3832008396 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 -- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 msgr2=0x563ed95ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 -- 172.21.15.137:0/3832008396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 msgr2=0x563ed95cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f41d9b640 1 -- 172.21.15.137:0/3832008396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed923c210 con 0x563ed95c5310 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.503+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cf250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005890 tx=0x7f7f380038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e50fbccb4431102 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3832008396 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38027020 con 0x563ed95c5310 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563ed9536a20 con 0x563ed95c5310 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3832008396 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f7f38003cc0 con 0x563ed95c5310 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3832008396 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f38032050 con 0x563ed95c5310 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 10 client.25348 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ed9308620 con 0x563ed95c5310 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3832008396 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563ed9536a20 con 0x563ed95c5310 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3832008396 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7f38018070 con 0x563ed95c5310 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 10 client.25348 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 10 client.25348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 20 client.25348 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:59.514 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95e1cd0 0x563ed95e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 4 client.25348 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.504+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed9530d70 con 0x563ed95e1cd0 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.505+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95e1cd0 0x563ed95e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.505+0000 7f7f29ffb640 10 client.25348 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.505+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95e1cd0 0x563ed95e40c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b52eb5cf468c8385 server_cookie=a7e1b671879d8ab4 in_seq=0 out_seq=0 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.506+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3832008396 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ed9530d70 con 0x563ed95e1cd0 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.506+0000 7f7f29ffb640 10 client.25348 handle_command_reply: tid=0 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.506+0000 7f7f4b817740 1 client.25348 shutdown 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.506+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 msgr2=0x563ed95cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.506+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cf250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f38005890 tx=0x7f7f380038a0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 shutdown_connections 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95e1cd0 0x563ed95e40c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cf250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3832008396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 >> 172.21.15.137:0/3832008396 conn(0x563ed95c1060 msgr2=0x563ed95c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f29ffb640 0 client.25348 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 shutdown_connections 2024-09-12T12:16:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 1 -- 172.21.15.137:0/3832008396 wait complete. 2024-09-12T12:16:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.507+0000 7f7f4b817740 20 client.25348 trim_cache size 0 max 16384 2024-09-12T12:16:59.518 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:16:59 smithi137 ceph-mon[30950]: pgmap v116: 97 pgs: 97 active+clean; 86 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 301 op/s 2024-09-12T12:16:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 Processor -- start 2024-09-12T12:16:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 -- start start 2024-09-12T12:16:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441ade30 con 0x563ed95c5310 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441ab6a0 con 0x563ed95d1ec0 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4b817740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441a94f0 con 0x563ed95c49f0 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f41d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f42d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f41d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95e9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44076/0 (socket says 172.21.15.137:44076) 2024-09-12T12:16:59.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f41d9b640 1 -- 172.21.15.137:0/1436258330 learned_addr learned my addr 172.21.15.137:0/1436258330 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.525+0000 7f7f4259c640 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f41d9b640 1 -- 172.21.15.137:0/1436258330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 msgr2=0x563ed95e99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95e99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f41d9b640 1 -- 172.21.15.137:0/1436258330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 msgr2=0x563ed95c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f41d9b640 1 -- 172.21.15.137:0/1436258330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed95c0e90 con 0x563ed95c5310 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f4259c640 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95e9ef0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009bd0 tx=0x7f7f38011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c715a961a56a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f2bfff640 1 -- 172.21.15.137:0/1436258330 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38027040 con 0x563ed95c5310 2024-09-12T12:16:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f2bfff640 1 -- 172.21.15.137:0/1436258330 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f38003df0 con 0x563ed95c5310 2024-09-12T12:16:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.526+0000 7f7f2bfff640 1 -- 172.21.15.137:0/1436258330 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3801e040 con 0x563ed95c5310 2024-09-12T12:16:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 -- 172.21.15.137:0/1436258330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 msgr2=0x563ed95e9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95e9ef0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009bd0 tx=0x7f7f38011d10 comp rx=0 tx=0).stop 2024-09-12T12:16:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 -- 172.21.15.137:0/1436258330 shutdown_connections 2024-09-12T12:16:59.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95d1ec0 0x563ed95c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95c5310 0x563ed95e9ef0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 --2- 172.21.15.137:0/1436258330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c49f0 0x563ed95e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 -- 172.21.15.137:0/1436258330 >> 172.21.15.137:0/1436258330 conn(0x563ed95c1060 msgr2=0x563ed95c1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 -- 172.21.15.137:0/1436258330 shutdown_connections 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 -- 172.21.15.137:0/1436258330 wait complete. 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 -- start start 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 Processor -- start 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.527+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4b817740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4b817740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441a9b60 con 0x563ed95d1ec0 2024-09-12T12:16:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4b817740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441aafd0 con 0x563ed95c49f0 2024-09-12T12:16:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4b817740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7f441aa230 con 0x563ed95c5310 2024-09-12T12:16:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4b817740 10 client.0 _ll_register_callbacks cb 0x7f7f3409bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:16:59.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44088/0 (socket says 172.21.15.137:44088) 2024-09-12T12:16:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39200/0 (socket says 172.21.15.137:39200) 2024-09-12T12:16:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 -- 172.21.15.137:0/3519504801 learned_addr learned my addr 172.21.15.137:0/3519504801 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f41d9b640 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 -- 172.21.15.137:0/3519504801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 msgr2=0x563ed95cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 -- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 msgr2=0x563ed95cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:16:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 -- 172.21.15.137:0/3519504801 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed95d8f50 con 0x563ed95d1ec0 2024-09-12T12:16:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:16:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f42d9d640 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cba80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c02ae00 tx=0x7f7f3c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f0cf3cafa8dc57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.528+0000 7f7f29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:16:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3519504801 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c058060 con 0x563ed95d1ec0 2024-09-12T12:16:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3519504801 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f7f3c030bf0 con 0x563ed95d1ec0 2024-09-12T12:16:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f38003df0 con 0x563ed95d1ec0 2024-09-12T12:16:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3519504801 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f3c030df0 con 0x563ed95d1ec0 2024-09-12T12:16:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-09-12T12:16:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ed95da220 con 0x563ed95d1ec0 2024-09-12T12:16:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3519504801 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f38003df0 con 0x563ed95d1ec0 2024-09-12T12:16:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3519504801 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7f3c030070 con 0x563ed95d1ec0 2024-09-12T12:16:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:16:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:16:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 20 client.15819 populate_metadata read hostname 'smithi137' 2024-09-12T12:16:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95da960 0x563ed9650d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 4 client.15819 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:16:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ed94ad900 con 0x563ed95da960 2024-09-12T12:16:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.529+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95da960 0x563ed9650d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.530+0000 7f7f29ffb640 10 client.15819 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.530+0000 7f7f4259c640 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95da960 0x563ed9650d90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b83b6c8dab1163 server_cookie=b1c34bd3d2442ad0 in_seq=0 out_seq=0 2024-09-12T12:16:59.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:16:59 smithi189 ceph-mon[35475]: pgmap v116: 97 pgs: 97 active+clean; 86 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 301 op/s 2024-09-12T12:16:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f29ffb640 1 -- 172.21.15.137:0/3519504801 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ed94ad900 con 0x563ed95da960 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f29ffb640 10 client.15819 handle_command_reply: tid=0 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 client.15819 shutdown 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 msgr2=0x563ed95cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cba80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c02ae00 tx=0x7f7f3c02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 shutdown_connections 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ed95da960 0x563ed9650d90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ed95d1ec0 0x563ed95cba80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ed95c5310 0x563ed95cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 --2- 172.21.15.137:0/3519504801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ed95c49f0 0x563ed95cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f29ffb640 0 client.15819 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.600+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 >> 172.21.15.137:0/3519504801 conn(0x563ed95c1060 msgr2=0x563ed95bf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 shutdown_connections 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f4b817740 1 -- 172.21.15.137:0/3519504801 wait complete. 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f4b817740 20 client.15819 trim_cache size 0 max 16384 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:16:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:16:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7f2c0a2da0 msgr2=0x7f7f2c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7f2c0a2da0 0x7f7f2c0a5260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009430 tx=0x7f7f3c0317e0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f441009b0 msgr2=0x7f7f44197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f441009b0 0x7f7f44197210 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400e9f0 tx=0x7f7f3400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 shutdown_connections 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7f2c0a2da0 0x7f7f2c0a5260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7f441069c0 0x7f7f4419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f44101350 0x7f7f44197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 --2- 172.21.15.137:0/3802819007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f441009b0 0x7f7f44197210 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 >> 172.21.15.137:0/3802819007 conn(0x7f7f44077990 msgr2=0x7f7f44103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 shutdown_connections 2024-09-12T12:16:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:16:59.601+0000 7f7f48a74640 1 -- 172.21.15.137:0/3802819007 wait complete. 2024-09-12T12:16:59.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:16:59 smithi171 ceph-mon[35098]: pgmap v116: 97 pgs: 97 active+clean; 86 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 301 op/s 2024-09-12T12:16:59.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 290540 12.71 MB/sec execute 34 sec latency 46.190 ms 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 -- 172.21.15.137:0/1400398959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 msgr2=0x7f55f81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/1400398959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81067f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009900 tx=0x7f55e002a410 comp rx=0 tx=0).stop 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 -- 172.21.15.137:0/1400398959 shutdown_connections 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/1400398959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f8106f20 0x7f55f810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/1400398959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81067f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/1400398959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55f8101df0 0x7f55f81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 -- 172.21.15.137:0/1400398959 >> 172.21.15.137:0/1400398959 conn(0x7f55f80fd8f0 msgr2=0x7f55f80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 -- 172.21.15.137:0/1400398959 shutdown_connections 2024-09-12T12:16:59.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.886+0000 7f55fd9ce640 1 -- 172.21.15.137:0/1400398959 wait complete. 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 Processor -- start 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 -- start start 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f8101df0 0x7f55f8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55f8106f20 0x7f55f819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111f80 con 0x7f55f81027a0 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111e00 con 0x7f55f8106f20 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55fd9ce640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8112100 con 0x7f55f8101df0 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55effff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f8101df0 0x7f55f8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55effff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44112/0 (socket says 172.21.15.137:44112) 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55effff640 1 -- 172.21.15.137:0/130418638 learned_addr learned my addr 172.21.15.137:0/130418638 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:16:59.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.887+0000 7f55f77fe640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55f8106f20 0x7f55f819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55effff640 1 -- 172.21.15.137:0/130418638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f8101df0 msgr2=0x7f55f8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55effff640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f8101df0 0x7f55f8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55effff640 1 -- 172.21.15.137:0/130418638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55f8106f20 msgr2=0x7f55f819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55effff640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55f8106f20 0x7f55f819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55effff640 1 -- 172.21.15.137:0/130418638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819d410 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55effff640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81989a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f55e0006010 tx=0x7f55e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55fc9cc640 1 -- 172.21.15.137:0/130418638 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e003f030 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55fc9cc640 1 -- 172.21.15.137:0/130418638 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e0003ce0 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55fc9cc640 1 -- 172.21.15.137:0/130418638 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e0035070 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819d6a0 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.888+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f819db30 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.889+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f81021d0 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.891+0000 7f55fc9cc640 1 -- 172.21.15.137:0/130418638 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f55e0042030 con 0x7f55f81027a0 2024-09-12T12:16:59.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.892+0000 7f55fc9cc640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55e40a2cf0 0x7f55e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:16:59.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.892+0000 7f55f6ffd640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55e40a2cf0 0x7f55e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:16:59.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.892+0000 7f55fc9cc640 1 -- 172.21.15.137:0/130418638 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f55e00efb10 con 0x7f55f81027a0 2024-09-12T12:16:59.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.892+0000 7f55f6ffd640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55e40a2cf0 0x7f55e40a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55e800bbe0 tx=0x7f55e800b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:16:59.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:16:59.895+0000 7f55fc9cc640 1 -- 172.21.15.137:0/130418638 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55e00bc1f0 con 0x7f55f81027a0 2024-09-12T12:17:00.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.127+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55f802d7c0 con 0x7f55f81027a0 2024-09-12T12:17:00.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.128+0000 7f55fc9cc640 1 -- 172.21.15.137:0/130418638 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f55e00c04f0 con 0x7f55f81027a0 2024-09-12T12:17:00.129 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:17:00.129 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":27,"btime":"2024-09-12T12:14:37:790728+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:14:36.964904+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:17:00.129 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 27 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55e40a2cf0 msgr2=0x7f55e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55e40a2cf0 0x7f55e40a51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f55e800bbe0 tx=0x7f55e800b390 comp rx=0 tx=0).stop 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 msgr2=0x7f55f81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81989a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f55e0006010 tx=0x7f55e00042f0 comp rx=0 tx=0).stop 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 shutdown_connections 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55e40a2cf0 0x7f55e40a51b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55f8106f20 0x7f55f819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55f81027a0 0x7f55f81989a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 --2- 172.21.15.137:0/130418638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f8101df0 0x7f55f8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 >> 172.21.15.137:0/130418638 conn(0x7f55f80fd8f0 msgr2=0x7f55f80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:00.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 shutdown_connections 2024-09-12T12:17:00.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:00.131+0000 7f55fd9ce640 1 -- 172.21.15.137:0/130418638 wait complete. 2024-09-12T12:17:00.434 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:00 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/130418638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:00.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:00 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/130418638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:00.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:00 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/130418638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:00.723 INFO:tasks.workunit.client.0.smithi137.stdout: 1 291891 12.54 MB/sec execute 35 sec latency 43.168 ms 2024-09-12T12:17:00.992 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T12:17:00.992 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T12:17:01.266 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:17:01.514 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:01 smithi137 ceph-mon[30950]: pgmap v117: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-09-12T12:17:01.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:01 smithi189 ceph-mon[35475]: pgmap v117: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-09-12T12:17:01.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:01 smithi171 ceph-mon[35098]: pgmap v117: 97 pgs: 97 active+clean; 88 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 381 op/s 2024-09-12T12:17:01.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294492 12.65 MB/sec execute 36 sec latency 56.965 ms 2024-09-12T12:17:01.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 -- 172.21.15.137:0/1092765981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c102d50 msgr2=0x7f299c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 --2- 172.21.15.137:0/1092765981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c102d50 0x7f299c103130 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f298c009ca0 tx=0x7f298c02a950 comp rx=0 tx=0).stop 2024-09-12T12:17:01.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 -- 172.21.15.137:0/1092765981 shutdown_connections 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 --2- 172.21.15.137:0/1092765981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c10c180 0x7f299c10e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 --2- 172.21.15.137:0/1092765981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c103670 0x7f299c10bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 --2- 172.21.15.137:0/1092765981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c102d50 0x7f299c103130 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 -- 172.21.15.137:0/1092765981 >> 172.21.15.137:0/1092765981 conn(0x7f299c0fc7d0 msgr2=0x7f299c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 -- 172.21.15.137:0/1092765981 shutdown_connections 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.810+0000 7f29a2216640 1 -- 172.21.15.137:0/1092765981 wait complete. 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 Processor -- start 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 -- start start 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c102d50 0x7f299c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c103670 0x7f299c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c10c180 0x7f299c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f299c111f80 con 0x7f299c103670 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c111e00 con 0x7f299c102d50 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.811+0000 7f29a2216640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f299c112100 con 0x7f299c10c180 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f2993fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c103670 0x7f299c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c102d50 0x7f299c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c102d50 0x7f299c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39218/0 (socket says 172.21.15.137:39218) 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f2993fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c103670 0x7f299c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44118/0 (socket says 172.21.15.137:44118) 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 -- 172.21.15.137:0/808801786 learned_addr learned my addr 172.21.15.137:0/808801786 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299bfff640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c10c180 0x7f299c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 -- 172.21.15.137:0/808801786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c10c180 msgr2=0x7f299c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c10c180 0x7f299c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 -- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c103670 msgr2=0x7f299c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c103670 0x7f299c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 -- 172.21.15.137:0/808801786 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c19d3d0 con 0x7f299c102d50 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f299b7fe640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c102d50 0x7f299c198420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f298c005cc0 tx=0x7f298c02f420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f29997fa640 1 -- 172.21.15.137:0/808801786 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c034040 con 0x7f299c102d50 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c19d660 con 0x7f299c102d50 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f29997fa640 1 -- 172.21.15.137:0/808801786 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c030920 con 0x7f299c102d50 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f29997fa640 1 -- 172.21.15.137:0/808801786 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0046e0 con 0x7f299c102d50 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.812+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1a4fa0 con 0x7f299c102d50 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29997fa640 1 -- 172.21.15.137:0/808801786 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f298c037020 con 0x7f299c102d50 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 Processor -- start 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 -- start start 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7bd7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fae7b3c160 con 0x55fae7be2a30 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fae7ae3720 con 0x55fae7bd8170 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29a45ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fae7b38120 con 0x55fae7bd7850 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f299bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7bd7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f299bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7bd7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33448/0 (socket says 172.21.15.137:33448) 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f299b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f2993fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f299bfff640 1 -- 172.21.15.137:0/3362564940 learned_addr learned my addr 172.21.15.137:0/3362564940 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f29997fa640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29880a2cf0 0x7f29880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.815+0000 7f299bfff640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29880a2cf0 0x7f29880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2993fff640 1 -- 172.21.15.137:0/3362564940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 msgr2=0x55fae7bd7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2993fff640 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7bd7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2993fff640 1 -- 172.21.15.137:0/3362564940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 msgr2=0x55fae7be24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2993fff640 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2993fff640 1 -- 172.21.15.137:0/3362564940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae78adba0 con 0x55fae7be2a30 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f299bfff640 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7bd7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f29997fa640 1 -- 172.21.15.137:0/808801786 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f298c0f0070 con 0x7f299c102d50 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2993fff640 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be4e20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2984009be0 tx=0x7f2984011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dff803c6a7a6769 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2991ffb640 1 -- 172.21.15.137:0/3362564940 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984004050 con 0x55fae7be2a30 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2991ffb640 1 -- 172.21.15.137:0/3362564940 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2984005500 con 0x55fae7be2a30 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.816+0000 7f2991ffb640 1 -- 172.21.15.137:0/3362564940 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984004dd0 con 0x55fae7be2a30 2024-09-12T12:17:01.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 -- 172.21.15.137:0/3362564940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 msgr2=0x55fae7be4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be4e20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2984009be0 tx=0x7f2984011810 comp rx=0 tx=0).stop 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f299bfff640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29880a2cf0 0x7f29880a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29940058c0 tx=0x7f2994004730 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 -- 172.21.15.137:0/3362564940 shutdown_connections 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be4e20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3362564940 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7bd7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 -- 172.21.15.137:0/3362564940 >> 172.21.15.137:0/3362564940 conn(0x55fae7bd3de0 msgr2=0x55fae7bd41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 -- 172.21.15.137:0/3362564940 shutdown_connections 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 -- 172.21.15.137:0/3362564940 wait complete. 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 -- start start 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.817+0000 7f29a45ef740 1 Processor -- start 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bdf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bdfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7be0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f29a45ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fae7be5600 con 0x55fae7bd8170 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f29a45ef740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fae7ac4900 con 0x55fae7bd7850 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f29a45ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fae7be5780 con 0x55fae7be2a30 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f29a45ef740 10 client.0 _ll_register_callbacks cb 0x55fae7ac2780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bdf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f2993fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bdfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f2993fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bdfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44134/0 (socket says 172.21.15.137:44134) 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f2993fff640 1 -- 172.21.15.137:0/3486668637 learned_addr learned my addr 172.21.15.137:0/3486668637 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299bfff640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7be0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299b7fe640 1 -- 172.21.15.137:0/3486668637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 msgr2=0x55fae7be0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299b7fe640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7be0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299b7fe640 1 -- 172.21.15.137:0/3486668637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 msgr2=0x55fae7bdfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299b7fe640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bdfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299b7fe640 1 -- 172.21.15.137:0/3486668637 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae7b04c50 con 0x55fae7bd7850 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299bfff640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7be0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f2993fff640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bdfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f299b7fe640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bdf8c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f298c0ed9c0 tx=0x7f298c004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49fa419971e9d55a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f297affd640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f297affd640 1 -- 172.21.15.137:0/3486668637 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0bc0f0 con 0x55fae7bd7850 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.818+0000 7f297affd640 1 -- 172.21.15.137:0/3486668637 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f298c003ce0 con 0x55fae7bd7850 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2984005500 con 0x55fae7bd7850 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f297affd640 1 -- 172.21.15.137:0/3486668637 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f298c0bce90 con 0x55fae7bd7850 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 10 client.25307 fetch_fsmap learned FSMap version 27 2024-09-12T12:17:01.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fae7be5600 con 0x55fae7bd7850 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f297affd640 1 -- 172.21.15.137:0/3486668637 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2984005500 con 0x55fae7bd7850 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f297affd640 1 -- 172.21.15.137:0/3486668637 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f298c04d030 con 0x55fae7bd7850 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 10 client.25307 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 10 client.25307 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 20 client.25307 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7bf25c0 0x55fae7bf4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 4 client.25307 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fae7b49890 con 0x55fae7bf25c0 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.819+0000 7f2993fff640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7bf25c0 0x55fae7bf4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.820+0000 7f297affd640 10 client.25307 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:01.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.820+0000 7f2993fff640 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7bf25c0 0x55fae7bf4a30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6cfa0ee17621e19 server_cookie=6339f83a58c91cca in_seq=0 out_seq=0 2024-09-12T12:17:01.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.820+0000 7f297affd640 1 -- 172.21.15.137:0/3486668637 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fae7b49890 con 0x55fae7bf25c0 2024-09-12T12:17:01.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.820+0000 7f297affd640 10 client.25307 handle_command_reply: tid=0 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.820+0000 7f29a45ef740 1 client.25307 shutdown 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 msgr2=0x55fae7bdf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bdf8c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f298c0ed9c0 tx=0x7f298c004140 comp rx=0 tx=0).stop 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 shutdown_connections 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7be0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7bf25c0 0x55fae7bf4a30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bdfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3486668637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bdf8c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f297affd640 0 client.25307 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 >> 172.21.15.137:0/3486668637 conn(0x55fae7bd3de0 msgr2=0x55fae7bd9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 shutdown_connections 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 1 -- 172.21.15.137:0/3486668637 wait complete. 2024-09-12T12:17:01.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.821+0000 7f29a45ef740 20 client.25307 trim_cache size 0 max 16384 2024-09-12T12:17:01.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.838+0000 7f29a45ef740 1 Processor -- start 2024-09-12T12:17:01.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f29a45ef740 1 -- start start 2024-09-12T12:17:01.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bd9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bde190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7bfa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f29a45ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1ade50 con 0x55fae7bd8170 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f29a45ef740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1ab6c0 con 0x55fae7bd7850 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f29a45ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a9510 con 0x55fae7be2a30 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f2993fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bde190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f2993fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bde190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44150/0 (socket says 172.21.15.137:44150) 2024-09-12T12:17:01.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bd9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f2993fff640 1 -- 172.21.15.137:0/3394927659 learned_addr learned my addr 172.21.15.137:0/3394927659 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299bfff640 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7bfa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299b7fe640 1 -- 172.21.15.137:0/3394927659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 msgr2=0x55fae7bfa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299b7fe640 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7bfa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299b7fe640 1 -- 172.21.15.137:0/3394927659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 msgr2=0x55fae7bde190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299b7fe640 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bde190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299b7fe640 1 -- 172.21.15.137:0/3394927659 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fae7bd3c10 con 0x55fae7bd7850 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f2993fff640 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bde190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.839+0000 7f299b7fe640 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bd9910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f298c0b9160 tx=0x7f298c0baca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ed9356c11583f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29917fa640 1 -- 172.21.15.137:0/3394927659 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0bc0f0 con 0x55fae7bd7850 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29917fa640 1 -- 172.21.15.137:0/3394927659 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c0c0b40 con 0x55fae7bd7850 2024-09-12T12:17:01.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29917fa640 1 -- 172.21.15.137:0/3394927659 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c02c040 con 0x55fae7bd7850 2024-09-12T12:17:01.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 -- 172.21.15.137:0/3394927659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 msgr2=0x55fae7bd9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bd9910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f298c0b9160 tx=0x7f298c0baca0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 -- 172.21.15.137:0/3394927659 shutdown_connections 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7be2a30 0x55fae7bfa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7bd8170 0x55fae7bde190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 --2- 172.21.15.137:0/3394927659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd7850 0x55fae7bd9910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 -- 172.21.15.137:0/3394927659 >> 172.21.15.137:0/3394927659 conn(0x55fae7bd3de0 msgr2=0x55fae7be00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 -- 172.21.15.137:0/3394927659 shutdown_connections 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.840+0000 7f29a45ef740 1 -- 172.21.15.137:0/3394927659 wait complete. 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 -- start start 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 Processor -- start 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7beb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1a9b80 con 0x55fae7be2a30 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1aaff0 con 0x55fae7bd8170 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1aa250 con 0x55fae7bd7850 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f29a45ef740 10 client.0 _ll_register_callbacks cb 0x7f298c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f299b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7beb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f2993fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f2993fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39252/0 (socket says 172.21.15.137:39252) 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f2993fff640 1 -- 172.21.15.137:0/206833292 learned_addr learned my addr 172.21.15.137:0/206833292 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.841+0000 7f299bfff640 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f299b7fe640 1 -- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 msgr2=0x55fae7be73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f299b7fe640 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f299b7fe640 1 -- 172.21.15.137:0/206833292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 msgr2=0x55fae7be7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f299b7fe640 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f299b7fe640 1 -- 172.21.15.137:0/206833292 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fae7bda6c0 con 0x55fae7bd7850 2024-09-12T12:17:01.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f2993fff640 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f299b7fe640 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7beb080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f298c0bc490 tx=0x7f298c003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89eaa11aee45bbd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f297affd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f297affd640 1 -- 172.21.15.137:0/206833292 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0bc0f0 con 0x55fae7bd7850 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298c0c0b40 con 0x55fae7bd7850 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f297affd640 1 -- 172.21.15.137:0/206833292 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f298c087300 con 0x55fae7bd7850 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f297affd640 1 -- 172.21.15.137:0/206833292 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f298c0c5400 con 0x55fae7bd7850 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f29a45ef740 10 client.25378 fetch_fsmap learned FSMap version 27 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.842+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fae7bcf3f0 con 0x55fae7bd7850 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f297affd640 1 -- 172.21.15.137:0/206833292 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f298c0c0b40 con 0x55fae7bd7850 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f297affd640 1 -- 172.21.15.137:0/206833292 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f298c04d070 con 0x55fae7bd7850 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f29a45ef740 10 client.25378 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f29a45ef740 10 client.25378 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f29a45ef740 20 client.25378 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:01.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f29a45ef740 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7c5e210 0x55fae7c60600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:01.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f29a45ef740 4 client.25378 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T12:17:01.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x55fae7b43be0 con 0x55fae7c5e210 2024-09-12T12:17:01.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.843+0000 7f2993fff640 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7c5e210 0x55fae7c60600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:01.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.844+0000 7f297affd640 10 client.25378 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:01.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.844+0000 7f2993fff640 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7c5e210 0x55fae7c60600 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2aebda711920bbf server_cookie=bd256e4fd447a255 in_seq=0 out_seq=0 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.919+0000 7f297affd640 1 -- 172.21.15.137:0/206833292 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fae7b43be0 con 0x55fae7c5e210 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.919+0000 7f297affd640 10 client.25378 handle_command_reply: tid=0 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.919+0000 7f29a45ef740 1 client.25378 shutdown 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 msgr2=0x55fae7beb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7beb080 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f298c0bc490 tx=0x7f298c003f10 comp rx=0 tx=0).stop 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 shutdown_connections 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fae7c5e210 0x55fae7c60600 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fae7be2a30 0x55fae7be7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fae7bd8170 0x55fae7be73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f297affd640 0 client.25378 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 --2- 172.21.15.137:0/206833292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fae7bd7850 0x55fae7beb080 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 >> 172.21.15.137:0/206833292 conn(0x55fae7bd3de0 msgr2=0x55fae7bd26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 shutdown_connections 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 1 -- 172.21.15.137:0/206833292 wait complete. 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.920+0000 7f29a45ef740 20 client.25378 trim_cache size 0 max 16384 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 27, 2024-09-12T12:17:01.922 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 14, 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "d885e5b2": { 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 14, 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 140.2, 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:17:01.923 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:17:01.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29880a2cf0 msgr2=0x7f29880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29880a2cf0 0x7f29880a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f29940058c0 tx=0x7f2994004730 comp rx=0 tx=0).stop 2024-09-12T12:17:01.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c102d50 msgr2=0x7f299c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:01.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c102d50 0x7f299c198420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f298c005cc0 tx=0x7f298c02f420 comp rx=0 tx=0).stop 2024-09-12T12:17:01.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 shutdown_connections 2024-09-12T12:17:01.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29880a2cf0 0x7f29880a51b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f299c10c180 0x7f299c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f299c103670 0x7f299c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 --2- 172.21.15.137:0/808801786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f299c102d50 0x7f299c198420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:01.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 >> 172.21.15.137:0/808801786 conn(0x7f299c0fc7d0 msgr2=0x7f299c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:01.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 shutdown_connections 2024-09-12T12:17:01.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:01.921+0000 7f29a2216640 1 -- 172.21.15.137:0/808801786 wait complete. 2024-09-12T12:17:02.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 12.34 MB/sec execute 37 sec latency 871.017 ms 2024-09-12T12:17:02.835 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd885e5b2', quiesce took 4.0 seconds. Will release after: 16.15 2024-09-12T12:17:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:03 smithi137 ceph-mon[30950]: pgmap v118: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-09-12T12:17:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:03 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:17:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:03 smithi189 ceph-mon[35475]: pgmap v118: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-09-12T12:17:03.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:03 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:17:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:03 smithi171 ceph-mon[35098]: pgmap v118: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-09-12T12:17:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:03 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:17:03.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 12.02 MB/sec execute 38 sec latency 1871.146 ms 2024-09-12T12:17:04.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 11.71 MB/sec execute 39 sec latency 2871.264 ms 2024-09-12T12:17:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:05 smithi137 ceph-mon[30950]: pgmap v119: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-09-12T12:17:05.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:05 smithi189 ceph-mon[35475]: pgmap v119: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-09-12T12:17:05.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:05 smithi171 ceph-mon[35098]: pgmap v119: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-09-12T12:17:05.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 11.42 MB/sec execute 40 sec latency 3871.383 ms 2024-09-12T12:17:06.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 11.14 MB/sec execute 41 sec latency 4871.468 ms 2024-09-12T12:17:07.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:07 smithi137 ceph-mon[30950]: pgmap v120: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-09-12T12:17:07.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:07 smithi189 ceph-mon[35475]: pgmap v120: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-09-12T12:17:07.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:07 smithi171 ceph-mon[35098]: pgmap v120: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 432 op/s 2024-09-12T12:17:07.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 10.87 MB/sec execute 42 sec latency 5871.529 ms 2024-09-12T12:17:08.552 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:08 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:17:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:08 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:17:08.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:08 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:17:08.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 10.62 MB/sec execute 43 sec latency 6871.637 ms 2024-09-12T12:17:09.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:09 smithi137 ceph-mon[30950]: pgmap v121: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-09-12T12:17:09.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:09 smithi189 ceph-mon[35475]: pgmap v121: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-09-12T12:17:09.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:09 smithi171 ceph-mon[35098]: pgmap v121: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-09-12T12:17:09.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 10.38 MB/sec execute 44 sec latency 7871.746 ms 2024-09-12T12:17:10.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 10.15 MB/sec execute 45 sec latency 8871.852 ms 2024-09-12T12:17:11.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:11 smithi137 ceph-mon[30950]: pgmap v122: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-09-12T12:17:11.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:11 smithi189 ceph-mon[35475]: pgmap v122: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-09-12T12:17:11.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:11 smithi171 ceph-mon[35098]: pgmap v122: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-09-12T12:17:11.724 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 9.93 MB/sec execute 46 sec latency 9871.928 ms 2024-09-12T12:17:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 -- 172.21.15.137:0/1872619092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101ad0 msgr2=0x7f5238101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1872619092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101ad0 0x7f5238101f30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f522c009930 tx=0x7f522c02a420 comp rx=0 tx=0).stop 2024-09-12T12:17:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 -- 172.21.15.137:0/1872619092 shutdown_connections 2024-09-12T12:17:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1872619092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238102470 0x7f523810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1872619092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238101ad0 0x7f5238101f30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1872619092 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238107b70 0x7f5238107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 -- 172.21.15.137:0/1872619092 >> 172.21.15.137:0/1872619092 conn(0x7f52380fd7d0 msgr2=0x7f52380ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 -- 172.21.15.137:0/1872619092 shutdown_connections 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.697+0000 7f523ebbd640 1 -- 172.21.15.137:0/1872619092 wait complete. 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 Processor -- start 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 -- start start 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238101ad0 0x7f5238198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238102470 0x7f52381988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238107b70 0x7f523819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111f20 con 0x7f5238107b70 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5238111da0 con 0x7f5238102470 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523ebbd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52381120a0 con 0x7f5238101ad0 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238107b70 0x7f523819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238107b70 0x7f523819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34592/0 (socket says 172.21.15.137:34592) 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 -- 172.21.15.137:0/1676660877 learned_addr learned my addr 172.21.15.137:0/1676660877 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f5237fff640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238102470 0x7f52381988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523c959640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238101ad0 0x7f5238198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 -- 172.21.15.137:0/1676660877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238101ad0 msgr2=0x7f5238198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238101ad0 0x7f5238198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 -- 172.21.15.137:0/1676660877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238102470 msgr2=0x7f52381988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238102470 0x7f52381988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 -- 172.21.15.137:0/1676660877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523819d360 con 0x7f5238107b70 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.698+0000 7f523d15a640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238107b70 0x7f523819cc30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f523000eaa0 tx=0x7f523000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.699+0000 7f5235ffb640 1 -- 172.21.15.137:0/1676660877 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523000cdb0 con 0x7f5238107b70 2024-09-12T12:17:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.699+0000 7f5235ffb640 1 -- 172.21.15.137:0/1676660877 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5230005020 con 0x7f5238107b70 2024-09-12T12:17:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.699+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523819d650 con 0x7f5238107b70 2024-09-12T12:17:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.699+0000 7f5235ffb640 1 -- 172.21.15.137:0/1676660877 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52300116b0 con 0x7f5238107b70 2024-09-12T12:17:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.699+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52381a4ee0 con 0x7f5238107b70 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.701+0000 7f5235ffb640 1 -- 172.21.15.137:0/1676660877 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5230005560 con 0x7f5238107b70 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 Processor -- start 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 -- start start 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5235ffb640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52200a2da0 0x7f52200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684445af0 0x564684445ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 0x564684450770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x5646844530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5646843aa2f0 con 0x564684450cb0 2024-09-12T12:17:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5235ffb640 1 -- 172.21.15.137:0/1676660877 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f52300cf760 con 0x7f5238107b70 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564684352270 con 0x564684445af0 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5241a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5646843a62b0 con 0x564684446410 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f5237fff640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52200a2da0 0x7f52200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x5646844530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x5646844530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34604/0 (socket says 172.21.15.137:34604) 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523c959640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684445af0 0x564684445ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 -- 172.21.15.137:0/1242970296 learned_addr learned my addr 172.21.15.137:0/1242970296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 -- 172.21.15.137:0/1242970296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 msgr2=0x564684450770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 --2- 172.21.15.137:0/1242970296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 0x564684450770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 -- 172.21.15.137:0/1242970296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684445af0 msgr2=0x564684445ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 --2- 172.21.15.137:0/1242970296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684445af0 0x564684445ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.702+0000 7f523d15a640 1 -- 172.21.15.137:0/1242970296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564684351340 con 0x564684450cb0 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.703+0000 7f5237fff640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52200a2da0 0x7f52200a5260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c032d10 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.703+0000 7f523d15a640 1 --2- 172.21.15.137:0/1242970296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x5646844530a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f52300a43b0 tx=0x7f52300a0020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9533d5f7a77be4a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.703+0000 7f521effd640 1 -- 172.21.15.137:0/1242970296 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523001f070 con 0x564684450cb0 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.703+0000 7f521effd640 1 -- 172.21.15.137:0/1242970296 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52300051d0 con 0x564684450cb0 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.703+0000 7f521effd640 1 -- 172.21.15.137:0/1242970296 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523009b3b0 con 0x564684450cb0 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.703+0000 7f5241a31740 1 -- 172.21.15.137:0/1242970296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 msgr2=0x5646844530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.703+0000 7f5241a31740 1 --2- 172.21.15.137:0/1242970296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x5646844530a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f52300a43b0 tx=0x7f52300a0020 comp rx=0 tx=0).stop 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- 172.21.15.137:0/1242970296 shutdown_connections 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 --2- 172.21.15.137:0/1242970296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x5646844530a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 --2- 172.21.15.137:0/1242970296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 0x564684450770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 --2- 172.21.15.137:0/1242970296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684445af0 0x564684445ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- 172.21.15.137:0/1242970296 >> 172.21.15.137:0/1242970296 conn(0x564684442060 msgr2=0x564684442440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- 172.21.15.137:0/1242970296 shutdown_connections 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- 172.21.15.137:0/1242970296 wait complete. 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- start start 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 Processor -- start 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x56468444db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684446410 0x56468444e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x56468444e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564684453880 con 0x564684450cb0 2024-09-12T12:17:12.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564684333290 con 0x564684446410 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.704+0000 7f5241a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564684453a00 con 0x564684445af0 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5241a31740 10 client.0 _ll_register_callbacks cb 0x564684364140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x56468444e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x56468444e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34618/0 (socket says 172.21.15.137:34618) 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 -- 172.21.15.137:0/2267604422 learned_addr learned my addr 172.21.15.137:0/2267604422 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523c959640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x56468444db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5237fff640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684446410 0x56468444e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 -- 172.21.15.137:0/2267604422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 msgr2=0x56468444db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x56468444db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 -- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684446410 msgr2=0x56468444e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684446410 0x56468444e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 -- 172.21.15.137:0/2267604422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646840bd210 con 0x564684450cb0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5237fff640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684446410 0x56468444e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523c959640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x56468444db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f523d15a640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x56468444e590 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f523000efd0 tx=0x7f523009c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d443e5a41a26346 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5217fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5217fff640 1 -- 172.21.15.137:0/2267604422 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230062030 con 0x564684450cb0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5217fff640 1 -- 172.21.15.137:0/2267604422 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f52300cd120 con 0x564684450cb0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52300051d0 con 0x564684450cb0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5217fff640 1 -- 172.21.15.137:0/2267604422 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52300b9070 con 0x564684450cb0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5241a31740 10 client.15855 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.705+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564684189620 con 0x564684450cb0 2024-09-12T12:17:12.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5217fff640 1 -- 172.21.15.137:0/2267604422 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52300051d0 con 0x564684450cb0 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5217fff640 1 -- 172.21.15.137:0/2267604422 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f523000f040 con 0x564684450cb0 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5241a31740 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5241a31740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5241a31740 20 client.15855 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5241a31740 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x564684462df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5241a31740 4 client.15855 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646843b7a20 con 0x564684460a00 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f523c959640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x564684462df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f5217fff640 10 client.15855 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.706+0000 7f523c959640 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x564684462df0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e810fdfdc7a21e server_cookie=a7d272ddb39f45dc in_seq=0 out_seq=0 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5217fff640 1 -- 172.21.15.137:0/2267604422 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646843b7a20 con 0x564684460a00 2024-09-12T12:17:12.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5217fff640 10 client.15855 handle_command_reply: tid=0 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 client.15855 shutdown 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 msgr2=0x56468444e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x56468444e590 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f523000efd0 tx=0x7f523009c370 comp rx=0 tx=0).stop 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 shutdown_connections 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x564684462df0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684450cb0 0x56468444e590 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684446410 0x56468444e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5217fff640 0 client.15855 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 --2- 172.21.15.137:0/2267604422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x56468444db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 >> 172.21.15.137:0/2267604422 conn(0x564684442060 msgr2=0x564684447d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 shutdown_connections 2024-09-12T12:17:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.707+0000 7f5241a31740 1 -- 172.21.15.137:0/2267604422 wait complete. 2024-09-12T12:17:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.708+0000 7f5241a31740 20 client.15855 trim_cache size 0 max 16384 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 Processor -- start 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 -- start start 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684445af0 0x56468444e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 0x56468444c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684468390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52381b0030 con 0x564684445af0 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52381ad900 con 0x564684450cb0 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5241a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52381ab750 con 0x564684446410 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684445af0 0x56468444e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684445af0 0x56468444e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34632/0 (socket says 172.21.15.137:34632) 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 -- 172.21.15.137:0/2134206706 learned_addr learned my addr 172.21.15.137:0/2134206706 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:12.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 -- 172.21.15.137:0/2134206706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 msgr2=0x56468444c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f5237fff640 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 0x56468444c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 0x56468444c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 -- 172.21.15.137:0/2134206706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 msgr2=0x564684468390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523d15a640 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684468390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684468390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.722+0000 7f523c959640 1 -- 172.21.15.137:0/2134206706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564684441e90 con 0x564684445af0 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f523c959640 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684445af0 0x56468444e3c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f522800b990 tx=0x7f522800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26e4ac30427302d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f521e7fc640 1 -- 172.21.15.137:0/2134206706 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522801c470 con 0x564684445af0 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f521e7fc640 1 -- 172.21.15.137:0/2134206706 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522801caf0 con 0x564684445af0 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f521e7fc640 1 -- 172.21.15.137:0/2134206706 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522801bda0 con 0x564684445af0 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 -- 172.21.15.137:0/2134206706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684445af0 msgr2=0x56468444e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684445af0 0x56468444e3c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f522800b990 tx=0x7f522800f7f0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 -- 172.21.15.137:0/2134206706 shutdown_connections 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684468390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684446410 0x56468444c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 --2- 172.21.15.137:0/2134206706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684445af0 0x56468444e3c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 -- 172.21.15.137:0/2134206706 >> 172.21.15.137:0/2134206706 conn(0x564684442060 msgr2=0x56468444c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 -- 172.21.15.137:0/2134206706 shutdown_connections 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.723+0000 7f5241a31740 1 -- 172.21.15.137:0/2134206706 wait complete. 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 -- start start 2024-09-12T12:17:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 Processor -- start 2024-09-12T12:17:12.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 9.72 MB/sec execute 47 sec latency 10871.976 ms 2024-09-12T12:17:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x564684451890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.730 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684446410 0x564684451dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684457c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52381abdc0 con 0x564684446410 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52381ad230 con 0x564684450cb0 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52381ac490 con 0x564684445af0 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5241a31740 10 client.0 _ll_register_callbacks cb 0x7f5228005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5237fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684446410 0x564684451dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5237fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684446410 0x564684451dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34644/0 (socket says 172.21.15.137:34644) 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f5237fff640 1 -- 172.21.15.137:0/872462510 learned_addr learned my addr 172.21.15.137:0/872462510 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f523c959640 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x564684451890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.724+0000 7f523d15a640 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684457c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5237fff640 1 -- 172.21.15.137:0/872462510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 msgr2=0x564684451890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.731 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5237fff640 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x564684451890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5237fff640 1 -- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 msgr2=0x564684457c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5237fff640 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684457c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5237fff640 1 -- 172.21.15.137:0/872462510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564684458250 con 0x564684446410 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5237fff640 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684446410 0x564684451dd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f522c009930 tx=0x7f522c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5058de18b39667 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5217fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5217fff640 1 -- 172.21.15.137:0/872462510 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c04e020 con 0x564684446410 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5217fff640 1 -- 172.21.15.137:0/872462510 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f522c0046c0 con 0x564684446410 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522801caf0 con 0x564684446410 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5217fff640 1 -- 172.21.15.137:0/872462510 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f522c059050 con 0x564684446410 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 10 client.15861 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564684459590 con 0x564684446410 2024-09-12T12:17:12.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5217fff640 1 -- 172.21.15.137:0/872462510 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522801caf0 con 0x564684446410 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5217fff640 1 -- 172.21.15.137:0/872462510 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f522c003a10 con 0x564684446410 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 10 client.15861 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 20 client.15861 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x5646844cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 4 client.15861 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.725+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646843b1d70 con 0x564684460a00 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.726+0000 7f523c959640 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x5646844cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.726+0000 7f5217fff640 10 client.15861 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:12.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.726+0000 7f523c959640 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x5646844cd650 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a145890733eee9 server_cookie=94940aca345321fc in_seq=0 out_seq=0 2024-09-12T12:17:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5217fff640 1 -- 172.21.15.137:0/872462510 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646843b1d70 con 0x564684460a00 2024-09-12T12:17:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5217fff640 10 client.15861 handle_command_reply: tid=0 2024-09-12T12:17:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 client.15861 shutdown 2024-09-12T12:17:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684446410 msgr2=0x564684451dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684446410 0x564684451dd0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f522c009930 tx=0x7f522c009260 comp rx=0 tx=0).stop 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 shutdown_connections 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564684450cb0 0x564684457c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564684460a00 0x5646844cd650 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564684446410 0x564684451dd0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5217fff640 0 client.15861 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 --2- 172.21.15.137:0/872462510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564684445af0 0x564684451890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 >> 172.21.15.137:0/872462510 conn(0x564684442060 msgr2=0x56468444cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 shutdown_connections 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 1 -- 172.21.15.137:0/872462510 wait complete. 2024-09-12T12:17:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.758+0000 7f5241a31740 20 client.15861 trim_cache size 0 max 16384 2024-09-12T12:17:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:17:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:17:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:17:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:17:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52200a2da0 msgr2=0x7f52200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52200a2da0 0x7f52200a5260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c032d10 comp rx=0 tx=0).stop 2024-09-12T12:17:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238107b70 msgr2=0x7f523819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238107b70 0x7f523819cc30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f523000eaa0 tx=0x7f523000ef60 comp rx=0 tx=0).stop 2024-09-12T12:17:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 shutdown_connections 2024-09-12T12:17:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f52200a2da0 0x7f52200a5260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5238107b70 0x7f523819cc30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5238102470 0x7f52381988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 --2- 172.21.15.137:0/1676660877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5238101ad0 0x7f5238198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 >> 172.21.15.137:0/1676660877 conn(0x7f52380fd7d0 msgr2=0x7f52380fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 shutdown_connections 2024-09-12T12:17:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:12.759+0000 7f523ebbd640 1 -- 172.21.15.137:0/1676660877 wait complete. 2024-09-12T12:17:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:13 smithi137 ceph-mon[30950]: pgmap v123: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.6 MiB/s wr, 253 op/s 2024-09-12T12:17:13.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:13 smithi189 ceph-mon[35475]: pgmap v123: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.6 MiB/s wr, 253 op/s 2024-09-12T12:17:13.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:13 smithi171 ceph-mon[35098]: pgmap v123: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 2.6 MiB/s wr, 253 op/s 2024-09-12T12:17:13.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 9.51 MB/sec execute 48 sec latency 11872.058 ms 2024-09-12T12:17:14.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 9.32 MB/sec execute 49 sec latency 12872.162 ms 2024-09-12T12:17:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:15 smithi137 ceph-mon[30950]: pgmap v124: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-12T12:17:15.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:15 smithi189 ceph-mon[35475]: pgmap v124: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-12T12:17:15.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:15 smithi171 ceph-mon[35098]: pgmap v124: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-12T12:17:15.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 9.13 MB/sec execute 50 sec latency 13872.225 ms 2024-09-12T12:17:16.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 8.95 MB/sec execute 51 sec latency 14872.321 ms 2024-09-12T12:17:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:17 smithi137 ceph-mon[30950]: pgmap v125: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-09-12T12:17:17.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:17 smithi189 ceph-mon[35475]: pgmap v125: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-09-12T12:17:17.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:17 smithi171 ceph-mon[35098]: pgmap v125: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 3.3 MiB/s wr, 250 op/s 2024-09-12T12:17:17.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 8.78 MB/sec execute 52 sec latency 15872.468 ms 2024-09-12T12:17:18.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 8.62 MB/sec execute 53 sec latency 16872.572 ms 2024-09-12T12:17:18.986 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:17:19.250 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:17:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:19 smithi137 ceph-mon[30950]: pgmap v126: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-12T12:17:19.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:19 smithi189 ceph-mon[35475]: pgmap v126: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-12T12:17:19.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:19 smithi171 ceph-mon[35098]: pgmap v126: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.9 MiB/s wr, 86 op/s 2024-09-12T12:17:19.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 8.46 MB/sec execute 54 sec latency 17872.625 ms 2024-09-12T12:17:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 -- 172.21.15.137:0/2173101722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 msgr2=0x7f92b410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2173101722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b410da40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009ca0 tx=0x7f92ac02a950 comp rx=0 tx=0).stop 2024-09-12T12:17:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 -- 172.21.15.137:0/2173101722 shutdown_connections 2024-09-12T12:17:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2173101722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b410da40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2173101722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92b4102570 0x7f92b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2173101722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b4100e20 0x7f92b4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 -- 172.21.15.137:0/2173101722 >> 172.21.15.137:0/2173101722 conn(0x7f92b40fc7d0 msgr2=0x7f92b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 -- 172.21.15.137:0/2173101722 shutdown_connections 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.749+0000 7f92b9a97640 1 -- 172.21.15.137:0/2173101722 wait complete. 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 Processor -- start 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 -- start start 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b4100e20 0x7f92b4197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92b4102570 0x7f92b41978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4110f80 con 0x7f92b4102ef0 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4110e00 con 0x7f92b4100e20 2024-09-12T12:17:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b9a97640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92b4111100 con 0x7f92b4102570 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44614/0 (socket says 172.21.15.137:44614) 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b3fff640 1 -- 172.21.15.137:0/2750410288 learned_addr learned my addr 172.21.15.137:0/2750410288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b37fe640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b4100e20 0x7f92b4197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.750+0000 7f92b2ffd640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92b4102570 0x7f92b41978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b3fff640 1 -- 172.21.15.137:0/2750410288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92b4102570 msgr2=0x7f92b41978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b3fff640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92b4102570 0x7f92b41978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b3fff640 1 -- 172.21.15.137:0/2750410288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b4100e20 msgr2=0x7f92b4197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b3fff640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b4100e20 0x7f92b4197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b3fff640 1 -- 172.21.15.137:0/2750410288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b419c330 con 0x7f92b4102ef0 2024-09-12T12:17:19.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b3fff640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b419bc50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f92ac006010 tx=0x7f92ac02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b0ff9640 1 -- 172.21.15.137:0/2750410288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac034040 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b419c5c0 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b0ff9640 1 -- 172.21.15.137:0/2750410288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92ac003e00 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b0ff9640 1 -- 172.21.15.137:0/2750410288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac045480 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.751+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b4069f70 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.752+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b41021b0 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.753+0000 7f92b0ff9640 1 -- 172.21.15.137:0/2750410288 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f92ac037020 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.754+0000 7f92b0ff9640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f929c0a4f00 0x7f929c0a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.754+0000 7f92b37fe640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f929c0a4f00 0x7f929c0a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.754+0000 7f92b0ff9640 1 -- 172.21.15.137:0/2750410288 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f92ac0f03d0 con 0x7f92b4102ef0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.754+0000 7f92b37fe640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f929c0a4f00 0x7f929c0a73c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92a40058c0 tx=0x7f92a400a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:19.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.757+0000 7f92b0ff9640 1 -- 172.21.15.137:0/2750410288 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92ac0bd1e0 con 0x7f92b4102ef0 2024-09-12T12:17:19.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.959+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92b41986e0 con 0x7f92b4102ef0 2024-09-12T12:17:19.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.960+0000 7f92b0ff9640 1 -- 172.21.15.137:0/2750410288 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f92ac0bcae0 con 0x7f92b4102ef0 2024-09-12T12:17:19.961 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:17:19.961 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":29,"btime":"2024-09-12T12:17:07:265882+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:17:06.970365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:17:19.962 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 29 2024-09-12T12:17:19.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f929c0a4f00 msgr2=0x7f929c0a73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:19.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f929c0a4f00 0x7f929c0a73c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92a40058c0 tx=0x7f92a400a2d0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 msgr2=0x7f92b419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:19.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b419bc50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f92ac006010 tx=0x7f92ac02f6f0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 shutdown_connections 2024-09-12T12:17:19.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f929c0a4f00 0x7f929c0a73c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f92b4102ef0 0x7f92b419bc50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92b4102570 0x7f92b41978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 --2- 172.21.15.137:0/2750410288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92b4100e20 0x7f92b4197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:19.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.962+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 >> 172.21.15.137:0/2750410288 conn(0x7f92b40fc7d0 msgr2=0x7f92b4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:19.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.963+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 shutdown_connections 2024-09-12T12:17:19.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:19.963+0000 7f92b9a97640 1 -- 172.21.15.137:0/2750410288 wait complete. 2024-09-12T12:17:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:20 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2750410288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:20.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:20 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:17:20.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:20 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2750410288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:20.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:20 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:17:20.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294644 8.30 MB/sec execute 55 sec latency 18872.729 ms 2024-09-12T12:17:20.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:20 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2750410288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:20.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:20 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:17:20.791 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id d885e5b2 --release --await' 2024-09-12T12:17:20.792 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id d885e5b2 --release --await 2024-09-12T12:17:21.051 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:17:21.473 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:21 smithi137 ceph-mon[30950]: pgmap v127: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2024-09-12T12:17:21.473 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:21 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:17:21.473 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:21 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:17:21.474 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:21 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:17:21.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 -- 172.21.15.137:0/2503068041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c101410 msgr2=0x7ff10c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 --2- 172.21.15.137:0/2503068041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c101410 0x7ff10c10d940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff104009cd0 tx=0x7ff10402a8e0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 -- 172.21.15.137:0/2503068041 shutdown_connections 2024-09-12T12:17:21.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 --2- 172.21.15.137:0/2503068041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c101410 0x7ff10c10d940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 --2- 172.21.15.137:0/2503068041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff10c100a70 0x7ff10c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 --2- 172.21.15.137:0/2503068041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c106a70 0x7ff10c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 -- 172.21.15.137:0/2503068041 >> 172.21.15.137:0/2503068041 conn(0x7ff10c0fc790 msgr2=0x7ff10c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 -- 172.21.15.137:0/2503068041 shutdown_connections 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.561+0000 7ff113579640 1 -- 172.21.15.137:0/2503068041 wait complete. 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 Processor -- start 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 -- start start 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c100a70 0x7ff10c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff10c101410 0x7ff10c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c106a70 0x7ff10c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c110e70 con 0x7ff10c100a70 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c110cf0 con 0x7ff10c106a70 2024-09-12T12:17:21.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff113579640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c110ff0 con 0x7ff10c101410 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff111315640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c100a70 0x7ff10c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff111315640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c100a70 0x7ff10c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44642/0 (socket says 172.21.15.137:44642) 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff111315640 1 -- 172.21.15.137:0/1417250325 learned_addr learned my addr 172.21.15.137:0/1417250325 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.562+0000 7ff110b14640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff10c101410 0x7ff10c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff111b16640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c106a70 0x7ff10c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff111315640 1 -- 172.21.15.137:0/1417250325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff10c101410 msgr2=0x7ff10c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff111315640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff10c101410 0x7ff10c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff111315640 1 -- 172.21.15.137:0/1417250325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c106a70 msgr2=0x7ff10c19bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff111315640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c106a70 0x7ff10c19bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff111315640 1 -- 172.21.15.137:0/1417250325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c19c2e0 con 0x7ff10c100a70 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff111315640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c100a70 0x7ff10c197330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d950 tx=0x7ff10800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:21.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff1027fc640 1 -- 172.21.15.137:0/1417250325 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108017070 con 0x7ff10c100a70 2024-09-12T12:17:21.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff1027fc640 1 -- 172.21.15.137:0/1417250325 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff108009c30 con 0x7ff10c100a70 2024-09-12T12:17:21.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10c19c5d0 con 0x7ff10c100a70 2024-09-12T12:17:21.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff1027fc640 1 -- 172.21.15.137:0/1417250325 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10800f6b0 con 0x7ff10c100a70 2024-09-12T12:17:21.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.563+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10c1a3eb0 con 0x7ff10c100a70 2024-09-12T12:17:21.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.565+0000 7ff1027fc640 1 -- 172.21.15.137:0/1417250325 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff1080109d0 con 0x7ff10c100a70 2024-09-12T12:17:21.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 Processor -- start 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 -- start start 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a2748440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2752d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2755630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3a26ad620 con 0x55d3a2748980 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d3a2654e50 con 0x55d3a2748060 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff115952740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d3a26a95e0 con 0x55d3a2753240 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff111315640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2752d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff1027fc640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff0f80a2cf0 0x7ff0f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff111315640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2752d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44656/0 (socket says 172.21.15.137:44656) 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff111315640 1 -- 172.21.15.137:0/1567758201 learned_addr learned my addr 172.21.15.137:0/1567758201 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:21.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff111b16640 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a2748440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff110b14640 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2755630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.566+0000 7ff111b16640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff0f80a2cf0 0x7ff0f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff1027fc640 1 -- 172.21.15.137:0/1417250325 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7ff1080cefe0 con 0x7ff10c100a70 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff111315640 1 -- 172.21.15.137:0/1567758201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 msgr2=0x55d3a2755630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff111315640 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2755630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff111315640 1 -- 172.21.15.137:0/1567758201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 msgr2=0x55d3a2748440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff111315640 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a2748440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff111315640 1 -- 172.21.15.137:0/1567758201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a2645d40 con 0x55d3a2748980 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff110b14640 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2755630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff111b16640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff0f80a2cf0 0x7ff0f80a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff104004af0 tx=0x7ff1040095c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff111315640 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2752d00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff108075e20 tx=0x7ff1080971b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c870f3a6bd57ffb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:21.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff0f77fe640 1 -- 172.21.15.137:0/1567758201 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108020070 con 0x55d3a2748980 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff0f77fe640 1 -- 172.21.15.137:0/1567758201 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff108009de0 con 0x55d3a2748980 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.567+0000 7ff0f77fe640 1 -- 172.21.15.137:0/1567758201 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1080d1080 con 0x55d3a2748980 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 -- 172.21.15.137:0/1567758201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 msgr2=0x55d3a2752d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2752d00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff108075e20 tx=0x7ff1080971b0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 -- 172.21.15.137:0/1567758201 shutdown_connections 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2755630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2752d00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 --2- 172.21.15.137:0/1567758201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a2748440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 -- 172.21.15.137:0/1567758201 >> 172.21.15.137:0/1567758201 conn(0x55d3a27445f0 msgr2=0x55d3a27449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 -- 172.21.15.137:0/1567758201 shutdown_connections 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 -- 172.21.15.137:0/1567758201 wait complete. 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 -- start start 2024-09-12T12:17:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.568+0000 7ff115952740 1 Processor -- start 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff115952740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a27501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff115952740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2750710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff115952740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2750c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff115952740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3a2755e10 con 0x55d3a2748980 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff115952740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d3a2635f10 con 0x55d3a2748060 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff115952740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d3a2755f90 con 0x55d3a2753240 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff115952740 10 client.0 _ll_register_callbacks cb 0x55d3a2659db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2750710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2750710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44670/0 (socket says 172.21.15.137:44670) 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 -- 172.21.15.137:0/3940429226 learned_addr learned my addr 172.21.15.137:0/3940429226 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff111315640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a27501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff111b16640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2750c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 -- 172.21.15.137:0/3940429226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 msgr2=0x55d3a2750c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2750c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 -- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 msgr2=0x55d3a27501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a27501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 -- 172.21.15.137:0/3940429226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a23bc580 con 0x55d3a2748980 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff111b16640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2750c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff111315640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a27501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff110b14640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2750710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b940 tx=0x7ff0fc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1c474ee4ac630c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff0f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.569+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/3940429226 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc003ea0 con 0x55d3a2748980 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/3940429226 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7ff0fc005220 con 0x55d3a2748980 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff108009de0 con 0x55d3a2748980 2024-09-12T12:17:21.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/3940429226 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0fc005420 con 0x55d3a2748980 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 10 client.15879 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3a2486960 con 0x55d3a2748980 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/3940429226 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff108009de0 con 0x55d3a2748980 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/3940429226 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff0fc024860 con 0x55d3a2748980 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 10 client.15879 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 10 client.15879 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 20 client.15879 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a2762e10 0x55d3a2765230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 4 client.15879 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3a26bab90 con 0x55d3a2762e10 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.570+0000 7ff111315640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a2762e10 0x55d3a2765230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.571+0000 7ff0f4ff9640 10 client.15879 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.571+0000 7ff111315640 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a2762e10 0x55d3a2765230 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62829d46503df586 server_cookie=9245ccff0016204b in_seq=0 out_seq=0 2024-09-12T12:17:21.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/3940429226 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3a26bab90 con 0x55d3a2762e10 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff0f4ff9640 10 client.15879 handle_command_reply: tid=0 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 client.15879 shutdown 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 msgr2=0x55d3a2750710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2750710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b940 tx=0x7ff0fc00be00 comp rx=0 tx=0).stop 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 shutdown_connections 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a2750c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a2762e10 0x55d3a2765230 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a2750710 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 --2- 172.21.15.137:0/3940429226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748060 0x55d3a27501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff0f4ff9640 0 client.15879 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 >> 172.21.15.137:0/3940429226 conn(0x55d3a27445f0 msgr2=0x55d3a274a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 shutdown_connections 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 1 -- 172.21.15.137:0/3940429226 wait complete. 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.572+0000 7ff115952740 20 client.15879 trim_cache size 0 max 16384 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 Processor -- start 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 -- start start 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2748060 0x55d3a275b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a275b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2753240 0x55d3a276ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1aefc0 con 0x55d3a2748980 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1ac890 con 0x55d3a2753240 2024-09-12T12:17:21.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff115952740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1aa6e0 con 0x55d3a2748060 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a275b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a275b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44684/0 (socket says 172.21.15.137:44684) 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 -- 172.21.15.137:0/1784970237 learned_addr learned my addr 172.21.15.137:0/1784970237 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff111315640 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2748060 0x55d3a275b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 -- 172.21.15.137:0/1784970237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2748060 msgr2=0x55d3a275b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2748060 0x55d3a275b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff111b16640 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2753240 0x55d3a276ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 -- 172.21.15.137:0/1784970237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2753240 msgr2=0x55d3a276ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2753240 0x55d3a276ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.589+0000 7ff110b14640 1 -- 172.21.15.137:0/1784970237 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3a2747e90 con 0x55d3a2748980 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff110b14640 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a275b910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b910 tx=0x7ff0fc005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681f4ba39cc23acf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:21.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff0f6ffd640 1 -- 172.21.15.137:0/1784970237 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc024480 con 0x55d3a2748980 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff0f6ffd640 1 -- 172.21.15.137:0/1784970237 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc016070 con 0x55d3a2748980 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff0f6ffd640 1 -- 172.21.15.137:0/1784970237 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc01a420 con 0x55d3a2748980 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff115952740 1 -- 172.21.15.137:0/1784970237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 msgr2=0x55d3a275b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff115952740 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a275b910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b910 tx=0x7ff0fc005c30 comp rx=0 tx=0).stop 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff115952740 1 -- 172.21.15.137:0/1784970237 shutdown_connections 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff115952740 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2753240 0x55d3a276ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff115952740 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748980 0x55d3a275b910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff115952740 1 --2- 172.21.15.137:0/1784970237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2748060 0x55d3a275b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.590+0000 7ff115952740 1 -- 172.21.15.137:0/1784970237 >> 172.21.15.137:0/1784970237 conn(0x55d3a27445f0 msgr2=0x55d3a274eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 -- 172.21.15.137:0/1784970237 shutdown_connections 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 -- 172.21.15.137:0/1784970237 wait complete. 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 -- start start 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 Processor -- start 2024-09-12T12:17:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748060 0x55d3a274db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748980 0x55d3a274c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a274c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1aad50 con 0x55d3a2748060 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1ac1c0 con 0x55d3a2748980 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1ab420 con 0x55d3a2753240 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff115952740 10 client.0 _ll_register_callbacks cb 0x7ff0fc004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.591+0000 7ff111315640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748060 0x55d3a274db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748060 0x55d3a274db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44700/0 (socket says 172.21.15.137:44700) 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 -- 172.21.15.137:0/1032674645 learned_addr learned my addr 172.21.15.137:0/1032674645 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff110b14640 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748980 0x55d3a274c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111b16640 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a274c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 -- 172.21.15.137:0/1032674645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 msgr2=0x55d3a274c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a274c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 -- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748980 msgr2=0x55d3a274c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748980 0x55d3a274c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 -- 172.21.15.137:0/1032674645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3a2761a10 con 0x55d3a2748060 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff111315640 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748060 0x55d3a274db10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff108097ee0 tx=0x7ff1080ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4da2175462f411b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff0f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/1032674645 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108020040 con 0x55d3a2748060 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/1032674645 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7ff108097b80 con 0x55d3a2748060 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0fc016070 con 0x55d3a2748060 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/1032674645 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff108009de0 con 0x55d3a2748060 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff115952740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3a27d0810 con 0x55d3a2748060 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.592+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/1032674645 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc016070 con 0x55d3a2748060 2024-09-12T12:17:21.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/1032674645 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7ff1080710b0 con 0x55d3a2748060 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff115952740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff115952740 10 client.15885 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff115952740 20 client.15885 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff115952740 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a27d0b00 0x55d3a27d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff115952740 4 client.15885 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d885e5b2", "release": true, "await": true}] 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d885e5b2", "release": true, "await": true}) -- 0x55d3a26b50a0 con 0x55d3a27d0b00 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff110b14640 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a27d0b00 0x55d3a27d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff0f4ff9640 10 client.15885 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:21.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.593+0000 7ff110b14640 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a27d0b00 0x55d3a27d2ef0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ee1aba4c84360 server_cookie=8e38342d5ef4bb14 in_seq=0 out_seq=0 2024-09-12T12:17:21.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:21 smithi189 ceph-mon[35475]: pgmap v127: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2024-09-12T12:17:21.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:21 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:17:21.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:21 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:17:21.610 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:21 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:17:21.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.603+0000 7ff0f4ff9640 1 -- 172.21.15.137:0/1032674645 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d3a26b50a0 con 0x55d3a27d0b00 2024-09-12T12:17:21.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.603+0000 7ff0f4ff9640 10 client.15885 handle_command_reply: tid=0 2024-09-12T12:17:21.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.603+0000 7ff115952740 1 client.15885 shutdown 2024-09-12T12:17:21.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748060 msgr2=0x55d3a274db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748060 0x55d3a274db10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff108097ee0 tx=0x7ff1080ce040 comp rx=0 tx=0).stop 2024-09-12T12:17:21.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 shutdown_connections 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3a2753240 0x55d3a274c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d3a2748980 0x55d3a274c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d3a27d0b00 0x55d3a27d2ef0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 --2- 172.21.15.137:0/1032674645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3a2748060 0x55d3a274db10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff0f4ff9640 0 client.15885 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 >> 172.21.15.137:0/1032674645 conn(0x55d3a27445f0 msgr2=0x55d3a2742f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 shutdown_connections 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 1 -- 172.21.15.137:0/1032674645 wait complete. 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.604+0000 7ff115952740 20 client.15885 trim_cache size 0 max 16384 2024-09-12T12:17:21.612 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 29, 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 16, 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "d885e5b2": { 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 16, 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:17:21.613 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 140.2, 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:21.614 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:17:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff0f80a2cf0 msgr2=0x7ff0f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff0f80a2cf0 0x7ff0f80a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff104004af0 tx=0x7ff1040095c0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c100a70 msgr2=0x7ff10c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c100a70 0x7ff10c197330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff10800d950 tx=0x7ff10800de10 comp rx=0 tx=0).stop 2024-09-12T12:17:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 shutdown_connections 2024-09-12T12:17:21.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff0f80a2cf0 0x7ff0f80a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff10c106a70 0x7ff10c19bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff10c101410 0x7ff10c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 --2- 172.21.15.137:0/1417250325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff10c100a70 0x7ff10c197330 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 >> 172.21.15.137:0/1417250325 conn(0x7ff10c0fc790 msgr2=0x7ff10c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 shutdown_connections 2024-09-12T12:17:21.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:21.605+0000 7ff113579640 1 -- 172.21.15.137:0/1417250325 wait complete. 2024-09-12T12:17:21.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:21 smithi171 ceph-mon[35098]: pgmap v127: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.6 MiB/s wr, 112 op/s 2024-09-12T12:17:21.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:21 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:17:21.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:21 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:17:21.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:21 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:17:21.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 294711 8.15 MB/sec execute 56 sec latency 19753.011 ms 2024-09-12T12:17:22.650 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd885e5b2', total seconds elapsed: 23.8 2024-09-12T12:17:22.650 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T12:17:22.650 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:17:22.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 296923 8.15 MB/sec execute 57 sec latency 21.353 ms 2024-09-12T12:17:22.914 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:17:23.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 -- 172.21.15.137:0/631723353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c105de0 msgr2=0x7f703c10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:23.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 --2- 172.21.15.137:0/631723353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c105de0 0x7f703c10d960 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7034009ca0 tx=0x7f703402a8c0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 -- 172.21.15.137:0/631723353 shutdown_connections 2024-09-12T12:17:23.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 --2- 172.21.15.137:0/631723353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c105de0 0x7f703c10d960 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 --2- 172.21.15.137:0/631723353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f703c101e20 0x7f703c1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 --2- 172.21.15.137:0/631723353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f703c101470 0x7f703c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 -- 172.21.15.137:0/631723353 >> 172.21.15.137:0/631723353 conn(0x7f703c077230 msgr2=0x7f703c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:23.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 -- 172.21.15.137:0/631723353 shutdown_connections 2024-09-12T12:17:23.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.422+0000 7f70418fe640 1 -- 172.21.15.137:0/631723353 wait complete. 2024-09-12T12:17:23.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 Processor -- start 2024-09-12T12:17:23.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 -- start start 2024-09-12T12:17:23.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c101470 0x7f703c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:23.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f703c101e20 0x7f703c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f703c105de0 0x7f703c1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f703c110e40 con 0x7f703c101470 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f703c110cc0 con 0x7f703c101e20 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70418fe640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f703c110fc0 con 0x7f703c105de0 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c101470 0x7f703c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c101470 0x7f703c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44724/0 (socket says 172.21.15.137:44724) 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 -- 172.21.15.137:0/1892456682 learned_addr learned my addr 172.21.15.137:0/1892456682 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:23.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f7033fff640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f703c101e20 0x7f703c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:23.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703b7fe640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f703c105de0 0x7f703c1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:23.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 -- 172.21.15.137:0/1892456682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f703c105de0 msgr2=0x7f703c1a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:23.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f703c105de0 0x7f703c1a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 -- 172.21.15.137:0/1892456682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f703c101e20 msgr2=0x7f703c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:23.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f703c101e20 0x7f703c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 -- 172.21.15.137:0/1892456682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c1a47d0 con 0x7f703c101470 2024-09-12T12:17:23.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f703affd640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c101470 0x7f703c199210 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f702c00e9f0 tx=0x7f702c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:23.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.423+0000 7f70408fc640 1 -- 172.21.15.137:0/1892456682 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c00cd80 con 0x7f703c101470 2024-09-12T12:17:23.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.424+0000 7f70408fc640 1 -- 172.21.15.137:0/1892456682 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702c005500 con 0x7f703c101470 2024-09-12T12:17:23.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.424+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c1a49a0 con 0x7f703c101470 2024-09-12T12:17:23.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.424+0000 7f70408fc640 1 -- 172.21.15.137:0/1892456682 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c010690 con 0x7f703c101470 2024-09-12T12:17:23.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.424+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c1a4eb0 con 0x7f703c101470 2024-09-12T12:17:23.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.425+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703c101850 con 0x7f703c101470 2024-09-12T12:17:23.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.426+0000 7f70408fc640 1 -- 172.21.15.137:0/1892456682 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f702c0046c0 con 0x7f703c101470 2024-09-12T12:17:23.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.427+0000 7f70408fc640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f70280a2d70 0x7f70280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:23.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.427+0000 7f7033fff640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f70280a2d70 0x7f70280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:23.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.427+0000 7f70408fc640 1 -- 172.21.15.137:0/1892456682 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f702c0cf960 con 0x7f703c101470 2024-09-12T12:17:23.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.428+0000 7f7033fff640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f70280a2d70 0x7f70280a5230 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f703c100c50 tx=0x7f702400b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:23.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.431+0000 7f70408fc640 1 -- 172.21.15.137:0/1892456682 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f702c0541b0 con 0x7f703c101470 2024-09-12T12:17:23.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:23 smithi189 ceph-mon[35475]: pgmap v128: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-12T12:17:23.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.653+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f703c109020 con 0x7f703c101470 2024-09-12T12:17:23.653 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:23 smithi137 ceph-mon[30950]: pgmap v128: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-12T12:17:23.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.654+0000 7f70408fc640 1 -- 172.21.15.137:0/1892456682 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f702c09c5d0 con 0x7f703c101470 2024-09-12T12:17:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:17:23.655 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":29,"btime":"2024-09-12T12:17:07:265882+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:17:06.970365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:17:23.655 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 29 2024-09-12T12:17:23.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f70280a2d70 msgr2=0x7f70280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:23.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f70280a2d70 0x7f70280a5230 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f703c100c50 tx=0x7f702400b390 comp rx=0 tx=0).stop 2024-09-12T12:17:23.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c101470 msgr2=0x7f703c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:23.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c101470 0x7f703c199210 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f702c00e9f0 tx=0x7f702c00edf0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 shutdown_connections 2024-09-12T12:17:23.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f70280a2d70 0x7f70280a5230 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f703c105de0 0x7f703c1a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f703c101e20 0x7f703c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.657+0000 7f70418fe640 1 --2- 172.21.15.137:0/1892456682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c101470 0x7f703c199210 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:23.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.658+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 >> 172.21.15.137:0/1892456682 conn(0x7f703c077230 msgr2=0x7f703c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:23.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.658+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 shutdown_connections 2024-09-12T12:17:23.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:23.658+0000 7f70418fe640 1 -- 172.21.15.137:0/1892456682 wait complete. 2024-09-12T12:17:23.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:23 smithi171 ceph-mon[35098]: pgmap v128: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-12T12:17:23.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 298942 8.22 MB/sec execute 58 sec latency 12.913 ms 2024-09-12T12:17:24.467 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:24 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1892456682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:24.495 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T12:17:24.495 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T12:17:24.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:24 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1892456682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:24.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:24 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1892456682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:24.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 301771 8.39 MB/sec execute 59 sec latency 13.984 ms 2024-09-12T12:17:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 -- 172.21.15.137:0/3426598687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 msgr2=0x7f468410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 --2- 172.21.15.137:0/3426598687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 0x7f468410da40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f467c009ca0 tx=0x7f467c02a950 comp rx=0 tx=0).stop 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 -- 172.21.15.137:0/3426598687 shutdown_connections 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 --2- 172.21.15.137:0/3426598687 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 0x7f468410da40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 --2- 172.21.15.137:0/3426598687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684102570 0x7f46841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 --2- 172.21.15.137:0/3426598687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4684100e20 0x7f4684101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 -- 172.21.15.137:0/3426598687 >> 172.21.15.137:0/3426598687 conn(0x7f46840fc7d0 msgr2=0x7f46840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 -- 172.21.15.137:0/3426598687 shutdown_connections 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.300+0000 7f468c312640 1 -- 172.21.15.137:0/3426598687 wait complete. 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 Processor -- start 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 -- start start 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684100e20 0x7f4684198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4684102570 0x7f46841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 0x7f468419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4684110f80 con 0x7f4684102ef0 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4684110e00 con 0x7f4684100e20 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468c312640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4684111100 con 0x7f4684102570 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468a0ae640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684100e20 0x7f4684198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468a8af640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 0x7f468419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468a0ae640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684100e20 0x7f4684198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39940/0 (socket says 172.21.15.137:39940) 2024-09-12T12:17:25.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f468a0ae640 1 -- 172.21.15.137:0/1266227194 learned_addr learned my addr 172.21.15.137:0/1266227194 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.301+0000 7f46898ad640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4684102570 0x7f46841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468a0ae640 1 -- 172.21.15.137:0/1266227194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4684102570 msgr2=0x7f46841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4684102570 0x7f46841989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468a0ae640 1 -- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 msgr2=0x7f468419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 0x7f468419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468a0ae640 1 -- 172.21.15.137:0/1266227194 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468419d440 con 0x7f4684100e20 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468a8af640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 0x7f468419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684100e20 0x7f4684198490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f468000e9f0 tx=0x7f468000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f467b7fe640 1 -- 172.21.15.137:0/1266227194 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468000cd80 con 0x7f4684100e20 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468419d730 con 0x7f4684100e20 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f467b7fe640 1 -- 172.21.15.137:0/1266227194 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680005500 con 0x7f4684100e20 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f467b7fe640 1 -- 172.21.15.137:0/1266227194 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680010690 con 0x7f4684100e20 2024-09-12T12:17:25.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.302+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46841a5010 con 0x7f4684100e20 2024-09-12T12:17:25.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.304+0000 7f467b7fe640 1 -- 172.21.15.137:0/1266227194 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f46800046c0 con 0x7f4684100e20 2024-09-12T12:17:25.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 Processor -- start 2024-09-12T12:17:25.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 -- start start 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f467b7fe640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f46700a2cf0 0x7f46700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f4fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f5a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468a8af640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f46700a2cf0 0x7f46700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a20eb4160 con 0x560a20f50170 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560a20e5b720 con 0x560a20f4f850 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f468e6eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a20eb0120 con 0x560a20f5aa30 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.305+0000 7f467b7fe640 1 -- 172.21.15.137:0/1266227194 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f46800cf7d0 con 0x7f4684100e20 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f46898ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f5a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a0ae640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f4fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f46898ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f5a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44750/0 (socket says 172.21.15.137:44750) 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f46898ad640 1 -- 172.21.15.137:0/2445019636 learned_addr learned my addr 172.21.15.137:0/2445019636 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a8af640 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a0ae640 1 -- 172.21.15.137:0/2445019636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 msgr2=0x560a20f5ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a0ae640 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a0ae640 1 -- 172.21.15.137:0/2445019636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 msgr2=0x560a20f5a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a0ae640 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f5a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a0ae640 1 -- 172.21.15.137:0/2445019636 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a20c25ba0 con 0x560a20f4f850 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a8af640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f46700a2cf0 0x7f46700a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f467c02fd80 tx=0x7f467c02f3a0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f468a0ae640 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f4fc30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46800ccf10 tx=0x7f468009b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdec868d8f71c66 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f4678dea640 1 -- 172.21.15.137:0/2445019636 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800a0090 con 0x560a20f4f850 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f4678dea640 1 -- 172.21.15.137:0/2445019636 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46800a4950 con 0x560a20f4f850 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.306+0000 7f4678dea640 1 -- 172.21.15.137:0/2445019636 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800928c0 con 0x560a20f4f850 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 -- 172.21.15.137:0/2445019636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 msgr2=0x560a20f4fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f4fc30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46800ccf10 tx=0x7f468009b640 comp rx=0 tx=0).stop 2024-09-12T12:17:25.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 -- 172.21.15.137:0/2445019636 shutdown_connections 2024-09-12T12:17:25.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f5a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 --2- 172.21.15.137:0/2445019636 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f4fc30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 -- 172.21.15.137:0/2445019636 >> 172.21.15.137:0/2445019636 conn(0x560a20f4bde0 msgr2=0x560a20f4c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 -- 172.21.15.137:0/2445019636 shutdown_connections 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.307+0000 7f468e6eb740 1 -- 172.21.15.137:0/2445019636 wait complete. 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 -- start start 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 Processor -- start 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f4f850 0x560a20f57990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f57ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f58410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560a20f5d600 con 0x560a20f50170 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560a20e3c900 con 0x560a20f5aa30 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.308+0000 7f468e6eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560a20f5d780 con 0x560a20f4f850 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f468e6eb740 10 client.0 _ll_register_callbacks cb 0x560a20e3a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f57ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f57ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44756/0 (socket says 172.21.15.137:44756) 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 -- 172.21.15.137:0/1929984376 learned_addr learned my addr 172.21.15.137:0/1929984376 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f468a8af640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f58410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 -- 172.21.15.137:0/1929984376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f4f850 msgr2=0x560a20f57990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f4f850 0x560a20f57990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f4f850 0x560a20f57990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 -- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 msgr2=0x560a20f58410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f58410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 -- 172.21.15.137:0/1929984376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a20e7cc50 con 0x560a20f50170 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f468a8af640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f58410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f46898ad640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f57ed0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4674012680 tx=0x7f4674012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6b71d8e652ad3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f466dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f466dffb640 1 -- 172.21.15.137:0/1929984376 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467400f920 con 0x560a20f50170 2024-09-12T12:17:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f466dffb640 1 -- 172.21.15.137:0/1929984376 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f467400fac0 con 0x560a20f50170 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.309+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46800a4950 con 0x560a20f50170 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f466dffb640 1 -- 172.21.15.137:0/1929984376 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f467400fcc0 con 0x560a20f50170 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 10 client.15903 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a20f69390 con 0x560a20f50170 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f466dffb640 1 -- 172.21.15.137:0/1929984376 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46800a4950 con 0x560a20f50170 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f466dffb640 1 -- 172.21.15.137:0/1929984376 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f467401c590 con 0x560a20f50170 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 10 client.15903 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 20 client.15903 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20f6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 4 client.15903 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a20ec1890 con 0x560a20f6a7f0 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.310+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20f6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20f6cc30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d507d44f95eb787c server_cookie=ab326510c64b6510 in_seq=0 out_seq=0 2024-09-12T12:17:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f466dffb640 10 client.15903 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f466dffb640 1 -- 172.21.15.137:0/1929984376 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a20ec1890 con 0x560a20f6a7f0 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f466dffb640 10 client.15903 handle_command_reply: tid=0 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 client.15903 shutdown 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 msgr2=0x560a20f57ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f57ed0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4674012680 tx=0x7f4674012b40 comp rx=0 tx=0).stop 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 shutdown_connections 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f58410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20f6cc30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f466dffb640 0 client.15903 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f57ed0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1929984376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f4f850 0x560a20f57990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.311+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 >> 172.21.15.137:0/1929984376 conn(0x560a20f4bde0 msgr2=0x560a20f51b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.312+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 shutdown_connections 2024-09-12T12:17:25.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.312+0000 7f468e6eb740 1 -- 172.21.15.137:0/1929984376 wait complete. 2024-09-12T12:17:25.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.312+0000 7f468e6eb740 20 client.15903 trim_cache size 0 max 16384 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 Processor -- start 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 -- start start 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f4f850 0x560a20f568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f50170 0x560a20f522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f72500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46841b0050 con 0x560a20f4f850 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46841ad920 con 0x560a20f5aa30 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468e6eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46841ab770 con 0x560a20f50170 2024-09-12T12:17:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468a0ae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f4f850 0x560a20f568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468a0ae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f4f850 0x560a20f568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44762/0 (socket says 172.21.15.137:44762) 2024-09-12T12:17:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.331+0000 7f468a0ae640 1 -- 172.21.15.137:0/396500212 learned_addr learned my addr 172.21.15.137:0/396500212 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f46898ad640 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f50170 0x560a20f522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f468a0ae640 1 -- 172.21.15.137:0/396500212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f50170 msgr2=0x560a20f522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f468a0ae640 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f50170 0x560a20f522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f468a0ae640 1 -- 172.21.15.137:0/396500212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 msgr2=0x560a20f72500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f468a0ae640 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f72500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f468a0ae640 1 -- 172.21.15.137:0/396500212 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a20f5d190 con 0x560a20f4f850 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f468a0ae640 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f4f850 0x560a20f568f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f468009b9e0 tx=0x7f46800980b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f21b84b413751b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f466ffff640 1 -- 172.21.15.137:0/396500212 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800a4950 con 0x560a20f4f850 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f466ffff640 1 -- 172.21.15.137:0/396500212 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680099d80 con 0x560a20f4f850 2024-09-12T12:17:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.332+0000 7f466ffff640 1 -- 172.21.15.137:0/396500212 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800a44c0 con 0x560a20f4f850 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 -- 172.21.15.137:0/396500212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f4f850 msgr2=0x560a20f568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f4f850 0x560a20f568f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f468009b9e0 tx=0x7f46800980b0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 -- 172.21.15.137:0/396500212 shutdown_connections 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f5aa30 0x560a20f72500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f50170 0x560a20f522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 --2- 172.21.15.137:0/396500212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f4f850 0x560a20f568f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 -- 172.21.15.137:0/396500212 >> 172.21.15.137:0/396500212 conn(0x560a20f4bde0 msgr2=0x560a20f59a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 -- 172.21.15.137:0/396500212 shutdown_connections 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 -- 172.21.15.137:0/396500212 wait complete. 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 -- start start 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.333+0000 7f468e6eb740 1 Processor -- start 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f62f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f46841abde0 con 0x560a20f50170 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f46841ad250 con 0x560a20f4f850 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f46841ac4b0 con 0x560a20f5aa30 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 10 client.0 _ll_register_callbacks cb 0x7f4674005af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f62f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f62f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44764/0 (socket says 172.21.15.137:44764) 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 -- 172.21.15.137:0/1322398509 learned_addr learned my addr 172.21.15.137:0/1322398509 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468a8af640 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 -- 172.21.15.137:0/1322398509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 msgr2=0x560a20f5f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 -- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 msgr2=0x560a20f629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f629d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 -- 172.21.15.137:0/1322398509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a20f5ff60 con 0x560a20f50170 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f46898ad640 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f62f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x560a20f57220 tx=0x7f46740123b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6eaf49a1b1951ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f466dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f466dffb640 1 -- 172.21.15.137:0/1322398509 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467402b030 con 0x560a20f50170 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4680099d80 con 0x560a20f50170 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.334+0000 7f466dffb640 1 -- 172.21.15.137:0/1322398509 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f4674018070 con 0x560a20f50170 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f466dffb640 1 -- 172.21.15.137:0/1322398509 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4674035050 con 0x560a20f50170 2024-09-12T12:17:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a20fd5d60 con 0x560a20f50170 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f466dffb640 1 -- 172.21.15.137:0/1322398509 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680099d80 con 0x560a20f50170 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f466dffb640 1 -- 172.21.15.137:0/1322398509 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4674003680 con 0x560a20f50170 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 10 client.15909 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 20 client.15909 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20fd8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 4 client.15909 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x560a20ebbbe0 con 0x560a20f6a7f0 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20fd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f468a0ae640 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20fd8580 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e74061c9caaa2016 server_cookie=eca46964097867c7 in_seq=0 out_seq=0 2024-09-12T12:17:25.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.335+0000 7f466dffb640 10 client.15909 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.450+0000 7f466dffb640 1 -- 172.21.15.137:0/1322398509 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560a20ebbbe0 con 0x560a20f6a7f0 2024-09-12T12:17:25.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f466dffb640 10 client.15909 handle_command_reply: tid=0 2024-09-12T12:17:25.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 client.15909 shutdown 2024-09-12T12:17:25.451 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:25 smithi137 ceph-mon[30950]: pgmap v129: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-12T12:17:25.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 msgr2=0x560a20f62f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f62f10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x560a20f57220 tx=0x7f46740123b0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 shutdown_connections 2024-09-12T12:17:25.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560a20f5aa30 0x560a20f5f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560a20f6a7f0 0x560a20fd8580 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560a20f50170 0x560a20f62f10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 --2- 172.21.15.137:0/1322398509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a20f4f850 0x560a20f629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f466dffb640 0 client.15909 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 >> 172.21.15.137:0/1322398509 conn(0x560a20f4bde0 msgr2=0x560a20f4a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 shutdown_connections 2024-09-12T12:17:25.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 1 -- 172.21.15.137:0/1322398509 wait complete. 2024-09-12T12:17:25.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.451+0000 7f468e6eb740 20 client.15909 trim_cache size 0 max 16384 2024-09-12T12:17:25.454 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:17:25.455 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 29, 2024-09-12T12:17:25.455 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:17:25.455 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 18, 2024-09-12T12:17:25.455 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:17:25.455 INFO:teuthology.orchestra.run.smithi137.stdout: "3a364b90": { 2024-09-12T12:17:25.455 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 18, 2024-09-12T12:17:25.456 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:17:25.456 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:25.456 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:17:25.456 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:25.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:17:25.456 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:17:25.457 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 140.2, 2024-09-12T12:17:25.457 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:17:25.457 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:17:25.457 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:17:25.457 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:25.457 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:17:25.458 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:25.458 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:25.458 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:25.458 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:25.459 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:25.459 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:25.459 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:17:25.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f46700a2cf0 msgr2=0x7f46700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f46700a2cf0 0x7f46700a51b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f467c02fd80 tx=0x7f467c02f3a0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684100e20 msgr2=0x7f4684198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684100e20 0x7f4684198490 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f468000e9f0 tx=0x7f468000edf0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 shutdown_connections 2024-09-12T12:17:25.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f46700a2cf0 0x7f46700a51b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4684102ef0 0x7f468419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4684102570 0x7f46841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 --2- 172.21.15.137:0/1266227194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4684100e20 0x7f4684198490 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 >> 172.21.15.137:0/1266227194 conn(0x7f46840fc7d0 msgr2=0x7f468406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 shutdown_connections 2024-09-12T12:17:25.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:25.452+0000 7f468c312640 1 -- 172.21.15.137:0/1266227194 wait complete. 2024-09-12T12:17:25.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:25 smithi189 ceph-mon[35475]: pgmap v129: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-12T12:17:25.601 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:25 smithi171 ceph-mon[35098]: pgmap v129: 97 pgs: 97 active+clean; 81 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 2.0 MiB/s wr, 87 op/s 2024-09-12T12:17:25.725 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 8.27 MB/sec execute 60 sec latency 396.760 ms 2024-09-12T12:17:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.863+0000 7f181959f640 1 -- 172.21.15.137:0/3515813934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 msgr2=0x7f1814106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.863+0000 7f181959f640 1 --2- 172.21.15.137:0/3515813934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f1814106bc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1804009cd0 tx=0x7f180402a8e0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.863+0000 7f181959f640 1 -- 172.21.15.137:0/3515813934 shutdown_connections 2024-09-12T12:17:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.863+0000 7f181959f640 1 --2- 172.21.15.137:0/3515813934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1814101170 0x7f181410d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.863+0000 7f181959f640 1 --2- 172.21.15.137:0/3515813934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18141007d0 0x7f1814100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.863+0000 7f181959f640 1 --2- 172.21.15.137:0/3515813934 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f1814106bc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.863+0000 7f181959f640 1 -- 172.21.15.137:0/3515813934 >> 172.21.15.137:0/3515813934 conn(0x7f18140777c0 msgr2=0x7f18140feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.864+0000 7f181959f640 1 -- 172.21.15.137:0/3515813934 shutdown_connections 2024-09-12T12:17:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.864+0000 7f181959f640 1 -- 172.21.15.137:0/3515813934 wait complete. 2024-09-12T12:17:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 Processor -- start 2024-09-12T12:17:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 -- start start 2024-09-12T12:17:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18141007d0 0x7f181419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1814101170 0x7f181419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f181419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1814110be0 con 0x7f18141067e0 2024-09-12T12:17:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1814110a60 con 0x7f1814101170 2024-09-12T12:17:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.865+0000 7f181959f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1814110d60 con 0x7f18141007d0 2024-09-12T12:17:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f1818d9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f181419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f1818d9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f181419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44782/0 (socket says 172.21.15.137:44782) 2024-09-12T12:17:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f1818d9e640 1 -- 172.21.15.137:0/142160728 learned_addr learned my addr 172.21.15.137:0/142160728 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f18137fe640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1814101170 0x7f181419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f1813fff640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18141007d0 0x7f181419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f18137fe640 1 -- 172.21.15.137:0/142160728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18141007d0 msgr2=0x7f181419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f18137fe640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18141007d0 0x7f181419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f18137fe640 1 -- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 msgr2=0x7f181419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f18137fe640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f181419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.866+0000 7f18137fe640 1 -- 172.21.15.137:0/142160728 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181419f3f0 con 0x7f1814101170 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f1818d9e640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f181419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f1813fff640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18141007d0 0x7f181419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f18137fe640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1814101170 0x7f181419a940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f180800e9f0 tx=0x7f180800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f18117fa640 1 -- 172.21.15.137:0/142160728 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180800ccc0 con 0x7f1814101170 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f18117fa640 1 -- 172.21.15.137:0/142160728 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1808010910 con 0x7f1814101170 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f18117fa640 1 -- 172.21.15.137:0/142160728 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808018960 con 0x7f1814101170 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181419f6e0 con 0x7f1814101170 2024-09-12T12:17:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.867+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1814107630 con 0x7f1814101170 2024-09-12T12:17:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.870+0000 7f181d675740 1 Processor -- start 2024-09-12T12:17:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.870+0000 7f18117fa640 1 -- 172.21.15.137:0/142160728 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1808010ac0 con 0x7f1814101170 2024-09-12T12:17:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f181d675740 1 -- start start 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f181d675740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c392cc0 0x55927c3930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f181d675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c39d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f181d675740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f181d675740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55927c2f72f0 con 0x55927c39de80 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f181d675740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55927c29f270 con 0x55927c392cc0 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f181d675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55927c2f32b0 con 0x55927c3935e0 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c392cc0 0x55927c3930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c392cc0 0x55927c3930a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:40006/0 (socket says 172.21.15.137:40006) 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 -- 172.21.15.137:0/2207162955 learned_addr learned my addr 172.21.15.137:0/2207162955 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1813fff640 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c39d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f18117fa640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17fc0a2d70 0x7f17fc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 -- 172.21.15.137:0/2207162955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 msgr2=0x55927c39d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c39d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 -- 172.21.15.137:0/2207162955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 msgr2=0x55927c3a0270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3a0270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 -- 172.21.15.137:0/2207162955 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927c29e340 con 0x55927c392cc0 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f18117fa640 1 -- 172.21.15.137:0/142160728 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f18080cf8a0 con 0x7f1814101170 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.871+0000 7f1818d9e640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17fc0a2d70 0x7f17fc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.872+0000 7f1818d9e640 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c392cc0 0x55927c3930a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f180c0116f0 tx=0x7f180c011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad0d14a1b75fc32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.872+0000 7f17fa7fc640 1 -- 172.21.15.137:0/2207162955 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c017050 con 0x55927c392cc0 2024-09-12T12:17:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.872+0000 7f17fa7fc640 1 -- 172.21.15.137:0/2207162955 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c01bdb0 con 0x55927c392cc0 2024-09-12T12:17:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.872+0000 7f17fa7fc640 1 -- 172.21.15.137:0/2207162955 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c026c10 con 0x55927c392cc0 2024-09-12T12:17:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.872+0000 7f1818d9e640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17fc0a2d70 0x7f17fc0a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f180c023000 tx=0x7f180c024000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.872+0000 7f181d675740 1 -- 172.21.15.137:0/2207162955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c392cc0 msgr2=0x55927c3930a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.872+0000 7f181d675740 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c392cc0 0x55927c3930a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f180c0116f0 tx=0x7f180c011bb0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- 172.21.15.137:0/2207162955 shutdown_connections 2024-09-12T12:17:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.878 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c39d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 --2- 172.21.15.137:0/2207162955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c392cc0 0x55927c3930a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- 172.21.15.137:0/2207162955 >> 172.21.15.137:0/2207162955 conn(0x55927c38f230 msgr2=0x55927c38f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- 172.21.15.137:0/2207162955 shutdown_connections 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- 172.21.15.137:0/2207162955 wait complete. 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- start start 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 Processor -- start 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c39b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55927c3a0a50 con 0x55927c39de80 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55927c280290 con 0x55927c3935e0 2024-09-12T12:17:25.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55927c3a0bd0 con 0x55927c392cc0 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.873+0000 7f181d675740 10 client.0 _ll_register_callbacks cb 0x55927c2b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f1813fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f1818d9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c39b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f18137fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f1813fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:60664/0 (socket says 172.21.15.137:60664) 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f1813fff640 1 -- 172.21.15.137:0/512187704 learned_addr learned my addr 172.21.15.137:0/512187704 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f18137fe640 1 -- 172.21.15.137:0/512187704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 msgr2=0x55927c39ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f18137fe640 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f18137fe640 1 -- 172.21.15.137:0/512187704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 msgr2=0x55927c39b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f18137fe640 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c39b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f18137fe640 1 -- 172.21.15.137:0/512187704 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927c00a210 con 0x55927c3935e0 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f1818d9e640 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c39b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f18137fe640 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39b340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18080ccfe0 tx=0x7f180809b7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=659c8d9b7bb931ec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f17f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f17f37fe640 1 -- 172.21.15.137:0/512187704 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18080cd470 con 0x55927c3935e0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180c01bdb0 con 0x55927c3935e0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f17f37fe640 1 -- 172.21.15.137:0/512187704 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f180809b400 con 0x55927c3935e0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f17f37fe640 1 -- 172.21.15.137:0/512187704 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18080ba070 con 0x55927c3935e0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f181d675740 10 client.25370 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55927c0d6620 con 0x55927c3935e0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.874+0000 7f17f37fe640 1 -- 172.21.15.137:0/512187704 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c01bdb0 con 0x55927c3935e0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f17f37fe640 1 -- 172.21.15.137:0/512187704 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1808086040 con 0x55927c3935e0 2024-09-12T12:17:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f181d675740 10 client.25370 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f181d675740 10 client.25370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f181d675740 20 client.25370 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f181d675740 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c3ade70 0x55927c3b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f181d675740 4 client.25370 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55927c304a20 con 0x55927c3ade70 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f1813fff640 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c3ade70 0x55927c3b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f17f37fe640 10 client.25370 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.875+0000 7f1813fff640 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c3ade70 0x55927c3b0260 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2945eab6b68c0e server_cookie=ab699e15986f6fbb in_seq=0 out_seq=0 2024-09-12T12:17:25.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f17f37fe640 1 -- 172.21.15.137:0/512187704 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55927c304a20 con 0x55927c3ade70 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f17f37fe640 10 client.25370 handle_command_reply: tid=0 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 client.25370 shutdown 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 msgr2=0x55927c39b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39b340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18080ccfe0 tx=0x7f180809b7d0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 shutdown_connections 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c3ade70 0x55927c3b0260 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c39b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f17f37fe640 0 client.25370 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39b340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 --2- 172.21.15.137:0/512187704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 >> 172.21.15.137:0/512187704 conn(0x55927c38f230 msgr2=0x55927c394f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 shutdown_connections 2024-09-12T12:17:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 1 -- 172.21.15.137:0/512187704 wait complete. 2024-09-12T12:17:25.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.876+0000 7f181d675740 20 client.25370 trim_cache size 0 max 16384 2024-09-12T12:17:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.890+0000 7f181d675740 1 Processor -- start 2024-09-12T12:17:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.890+0000 7f181d675740 1 -- start start 2024-09-12T12:17:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.890+0000 7f181d675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.890+0000 7f181d675740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.890+0000 7f181d675740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.890+0000 7f181d675740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18141b6480 con 0x55927c39de80 2024-09-12T12:17:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f181d675740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18141b3d50 con 0x55927c3935e0 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f181d675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18141b1be0 con 0x55927c392cc0 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44804/0 (socket says 172.21.15.137:44804) 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 -- 172.21.15.137:0/2753901078 learned_addr learned my addr 172.21.15.137:0/2753901078 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1813fff640 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f18137fe640 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 -- 172.21.15.137:0/2753901078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 msgr2=0x55927c39b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 -- 172.21.15.137:0/2753901078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 msgr2=0x55927c39cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 -- 172.21.15.137:0/2753901078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55927c3aa830 con 0x55927c39de80 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f18137fe640 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f1818d9e640 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3b5b50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f180c009f80 tx=0x7f180c026930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857127ec7a25e449 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f17f9ffb640 1 -- 172.21.15.137:0/2753901078 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c00fd80 con 0x55927c39de80 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f17f9ffb640 1 -- 172.21.15.137:0/2753901078 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c01bd30 con 0x55927c39de80 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.891+0000 7f17f9ffb640 1 -- 172.21.15.137:0/2753901078 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c02f930 con 0x55927c39de80 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 -- 172.21.15.137:0/2753901078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 msgr2=0x55927c3b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3b5b50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f180c009f80 tx=0x7f180c026930 comp rx=0 tx=0).stop 2024-09-12T12:17:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 -- 172.21.15.137:0/2753901078 shutdown_connections 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c39de80 0x55927c3b5b50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c3935e0 0x55927c39cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 --2- 172.21.15.137:0/2753901078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c392cc0 0x55927c39b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 -- 172.21.15.137:0/2753901078 >> 172.21.15.137:0/2753901078 conn(0x55927c38f230 msgr2=0x55927c39c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 -- 172.21.15.137:0/2753901078 shutdown_connections 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 -- 172.21.15.137:0/2753901078 wait complete. 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.892+0000 7f181d675740 1 -- start start 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 1 Processor -- start 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c392cc0 0x55927c3a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c3a6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c39de80 0x55927c3a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f18141b2230 con 0x55927c392cc0 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f18141b3680 con 0x55927c39de80 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18141b28e0 con 0x55927c3935e0 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f181d675740 10 client.0 _ll_register_callbacks cb 0x7f180800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c392cc0 0x55927c3a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c392cc0 0x55927c3a6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44818/0 (socket says 172.21.15.137:44818) 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 -- 172.21.15.137:0/147367553 learned_addr learned my addr 172.21.15.137:0/147367553 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1818d9e640 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c39de80 0x55927c3a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f18137fe640 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c3a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 -- 172.21.15.137:0/147367553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 msgr2=0x55927c3a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c3a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 -- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c39de80 msgr2=0x55927c3a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c39de80 0x55927c3a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 -- 172.21.15.137:0/147367553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55927c3a1c50 con 0x55927c392cc0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f1813fff640 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c392cc0 0x55927c3a6040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1804000f80 tx=0x7f1804031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced2897005336f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.893+0000 7f17f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f17f37fe640 1 -- 172.21.15.137:0/147367553 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040318c0 con 0x55927c392cc0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f17f37fe640 1 -- 172.21.15.137:0/147367553 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f1804031a60 con 0x55927c392cc0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180c01bd30 con 0x55927c392cc0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f17f37fe640 1 -- 172.21.15.137:0/147367553 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1804059050 con 0x55927c392cc0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 10 client.15924 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55927c418630 con 0x55927c392cc0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f17f37fe640 1 -- 172.21.15.137:0/147367553 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c01bd30 con 0x55927c392cc0 2024-09-12T12:17:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f17f37fe640 1 -- 172.21.15.137:0/147367553 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1804030070 con 0x55927c392cc0 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 10 client.15924 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 20 client.15924 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c418d50 0x55927c41b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 4 client.15924 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55927c2fed70 con 0x55927c418d50 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.894+0000 7f18137fe640 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c418d50 0x55927c41b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.895+0000 7f17f37fe640 10 client.15924 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.895+0000 7f18137fe640 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c418d50 0x55927c41b190 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22dadf6d939a4534 server_cookie=987fd4a57ec6904f in_seq=0 out_seq=0 2024-09-12T12:17:25.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f17f37fe640 1 -- 172.21.15.137:0/147367553 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55927c2fed70 con 0x55927c418d50 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f17f37fe640 10 client.15924 handle_command_reply: tid=0 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 client.15924 shutdown 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c392cc0 msgr2=0x55927c3a6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c392cc0 0x55927c3a6040 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1804000f80 tx=0x7f1804031610 comp rx=0 tx=0).stop 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 shutdown_connections 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55927c39de80 0x55927c3a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55927c3935e0 0x55927c3a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55927c418d50 0x55927c41b190 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 --2- 172.21.15.137:0/147367553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55927c392cc0 0x55927c3a6040 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f17f37fe640 0 client.15924 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.940+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 >> 172.21.15.137:0/147367553 conn(0x55927c38f230 msgr2=0x55927c394720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 shutdown_connections 2024-09-12T12:17:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181d675740 1 -- 172.21.15.137:0/147367553 wait complete. 2024-09-12T12:17:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181d675740 20 client.15924 trim_cache size 0 max 16384 2024-09-12T12:17:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:17:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:17:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:17:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:17:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17fc0a2d70 msgr2=0x7f17fc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17fc0a2d70 0x7f17fc0a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f180c023000 tx=0x7f180c024000 comp rx=0 tx=0).stop 2024-09-12T12:17:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1814101170 msgr2=0x7f181419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1814101170 0x7f181419a940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f180800e9f0 tx=0x7f180800eeb0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 shutdown_connections 2024-09-12T12:17:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17fc0a2d70 0x7f17fc0a5230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18141067e0 0x7f181419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1814101170 0x7f181419a940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 --2- 172.21.15.137:0/142160728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18141007d0 0x7f181419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 >> 172.21.15.137:0/142160728 conn(0x7f18140777c0 msgr2=0x7f1814069de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 shutdown_connections 2024-09-12T12:17:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:25.941+0000 7f181959f640 1 -- 172.21.15.137:0/142160728 wait complete. 2024-09-12T12:17:26.248 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3a364b90', quiesce took 3.6 seconds. Will release after: 16.549999999999997 2024-09-12T12:17:26.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 8.13 MB/sec execute 61 sec latency 1396.856 ms 2024-09-12T12:17:27.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:27 smithi189 ceph-mon[35475]: pgmap v130: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 180 op/s 2024-09-12T12:17:27.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:27 smithi171 ceph-mon[35098]: pgmap v130: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 180 op/s 2024-09-12T12:17:27.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 8.00 MB/sec execute 62 sec latency 2396.959 ms 2024-09-12T12:17:27.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:27 smithi137 ceph-mon[30950]: pgmap v130: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 180 op/s 2024-09-12T12:17:28.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.87 MB/sec execute 63 sec latency 3397.059 ms 2024-09-12T12:17:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:29 smithi189 ceph-mon[35475]: pgmap v131: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-12T12:17:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:29 smithi171 ceph-mon[35098]: pgmap v131: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-12T12:17:29.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.75 MB/sec execute 64 sec latency 4397.178 ms 2024-09-12T12:17:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:29 smithi137 ceph-mon[30950]: pgmap v131: 97 pgs: 97 active+clean; 84 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-12T12:17:30.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.63 MB/sec execute 65 sec latency 5397.258 ms 2024-09-12T12:17:31.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:31 smithi189 ceph-mon[35475]: pgmap v132: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-12T12:17:31.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:31 smithi171 ceph-mon[35098]: pgmap v132: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-12T12:17:31.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.51 MB/sec execute 66 sec latency 6397.383 ms 2024-09-12T12:17:31.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:31 smithi137 ceph-mon[30950]: pgmap v132: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-12T12:17:32.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.40 MB/sec execute 67 sec latency 7397.473 ms 2024-09-12T12:17:33.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:33 smithi189 ceph-mon[35475]: pgmap v133: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-12T12:17:33.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:33 smithi171 ceph-mon[35098]: pgmap v133: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-12T12:17:33.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.29 MB/sec execute 68 sec latency 8397.540 ms 2024-09-12T12:17:33.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:33 smithi137 ceph-mon[30950]: pgmap v133: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-12T12:17:34.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.19 MB/sec execute 69 sec latency 9397.617 ms 2024-09-12T12:17:35.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:35 smithi189 ceph-mon[35475]: pgmap v134: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-09-12T12:17:35.601 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:35 smithi137 ceph-mon[30950]: pgmap v134: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-09-12T12:17:35.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:35 smithi171 ceph-mon[35098]: pgmap v134: 97 pgs: 97 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 168 op/s 2024-09-12T12:17:35.726 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 7.08 MB/sec execute 70 sec latency 10397.674 ms 2024-09-12T12:17:36.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.98 MB/sec execute 71 sec latency 11397.724 ms 2024-09-12T12:17:37.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:37 smithi189 ceph-mon[35475]: pgmap v135: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-09-12T12:17:37.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:37 smithi171 ceph-mon[35098]: pgmap v135: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-09-12T12:17:37.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.89 MB/sec execute 72 sec latency 12398.067 ms 2024-09-12T12:17:37.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:37 smithi137 ceph-mon[30950]: pgmap v135: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 197 op/s 2024-09-12T12:17:38.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.79 MB/sec execute 73 sec latency 13398.198 ms 2024-09-12T12:17:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 -- 172.21.15.137:0/292403734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8103600 msgr2=0x7f34e810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/292403734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8103600 0x7f34e810bae0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009900 tx=0x7f34d002a480 comp rx=0 tx=0).stop 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 -- 172.21.15.137:0/292403734 shutdown_connections 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/292403734 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34e810c0b0 0x7f34e810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/292403734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8103600 0x7f34e810bae0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/292403734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34e8102ce0 0x7f34e81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 -- 172.21.15.137:0/292403734 >> 172.21.15.137:0/292403734 conn(0x7f34e8077990 msgr2=0x7f34e80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 -- 172.21.15.137:0/292403734 shutdown_connections 2024-09-12T12:17:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.036+0000 7f34ee1e4640 1 -- 172.21.15.137:0/292403734 wait complete. 2024-09-12T12:17:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 Processor -- start 2024-09-12T12:17:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 -- start start 2024-09-12T12:17:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8102ce0 0x7f34e8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34e8103600 0x7f34e81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34e810c0b0 0x7f34e819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8111eb0 con 0x7f34e8102ce0 2024-09-12T12:17:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8111d30 con 0x7f34e8103600 2024-09-12T12:17:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34ee1e4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8112030 con 0x7f34e810c0b0 2024-09-12T12:17:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8102ce0 0x7f34e8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8102ce0 0x7f34e8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41564/0 (socket says 172.21.15.137:41564) 2024-09-12T12:17:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.037+0000 7f34e77fe640 1 -- 172.21.15.137:0/3235502606 learned_addr learned my addr 172.21.15.137:0/3235502606 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34dffff640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34e8103600 0x7f34e81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e7fff640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34e810c0b0 0x7f34e819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e77fe640 1 -- 172.21.15.137:0/3235502606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34e810c0b0 msgr2=0x7f34e819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e77fe640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34e810c0b0 0x7f34e819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e77fe640 1 -- 172.21.15.137:0/3235502606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34e8103600 msgr2=0x7f34e81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e77fe640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34e8103600 0x7f34e81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e77fe640 1 -- 172.21.15.137:0/3235502606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e819d350 con 0x7f34e8102ce0 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e7fff640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34e810c0b0 0x7f34e819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34dffff640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34e8103600 0x7f34e81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e77fe640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8102ce0 0x7f34e8198360 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d770 tx=0x7f34d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e57fa640 1 -- 172.21.15.137:0/3235502606 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8017070 con 0x7f34e8102ce0 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e57fa640 1 -- 172.21.15.137:0/3235502606 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d8005500 con 0x7f34e8102ce0 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e819d640 con 0x7f34e8102ce0 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34e57fa640 1 -- 172.21.15.137:0/3235502606 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8004e00 con 0x7f34e8102ce0 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.038+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e81a4ed0 con 0x7f34e8102ce0 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.040+0000 7f34e57fa640 1 -- 172.21.15.137:0/3235502606 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f34d8004fa0 con 0x7f34e8102ce0 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 Processor -- start 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 -- start start 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650104062f0 con 0x5650104a1af0 2024-09-12T12:17:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5650103ae270 con 0x5650104accb0 2024-09-12T12:17:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34f105c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5650104022b0 con 0x5650104a2410 2024-09-12T12:17:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41576/0 (socket says 172.21.15.137:41576) 2024-09-12T12:17:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34e7fff640 1 -- 172.21.15.137:0/1853306801 learned_addr learned my addr 172.21.15.137:0/1853306801 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34e57fa640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34d40a2da0 0x7f34d40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34dffff640 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34e7fff640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34d40a2da0 0x7f34d40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.041+0000 7f34e77fe640 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e7fff640 1 -- 172.21.15.137:0/1853306801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 msgr2=0x5650104ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e57fa640 1 -- 172.21.15.137:0/3235502606 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f34d80cef30 con 0x7f34e8102ce0 2024-09-12T12:17:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e7fff640 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e7fff640 1 -- 172.21.15.137:0/1853306801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 msgr2=0x5650104af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e7fff640 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e7fff640 1 -- 172.21.15.137:0/1853306801 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650103ad340 con 0x5650104a1af0 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34dffff640 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e77fe640 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34e7fff640 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a1ed0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f34e00139e0 tx=0x7f34e0013ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a838f795c86f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34ddffb640 1 -- 172.21.15.137:0/1853306801 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e001fb80 con 0x5650104a1af0 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34ddffb640 1 -- 172.21.15.137:0/1853306801 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e001fd20 con 0x5650104a1af0 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.042+0000 7f34ddffb640 1 -- 172.21.15.137:0/1853306801 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00273c0 con 0x5650104a1af0 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 -- 172.21.15.137:0/1853306801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 msgr2=0x5650104a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a1ed0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f34e00139e0 tx=0x7f34e0013ea0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34e7fff640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34d40a2da0 0x7f34d40a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34e0022000 tx=0x7f34e0020000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 -- 172.21.15.137:0/1853306801 shutdown_connections 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 --2- 172.21.15.137:0/1853306801 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a1ed0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 -- 172.21.15.137:0/1853306801 >> 172.21.15.137:0/1853306801 conn(0x56501049e060 msgr2=0x56501049e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 -- 172.21.15.137:0/1853306801 shutdown_connections 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 -- 172.21.15.137:0/1853306801 wait complete. 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 -- start start 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.043+0000 7f34f105c740 1 Processor -- start 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34f105c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34f105c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34f105c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34f105c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5650104af880 con 0x5650104a1af0 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34f105c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56501038f290 con 0x5650104accb0 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34f105c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5650104afa00 con 0x5650104a2410 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34f105c740 10 client.0 _ll_register_callbacks cb 0x5650103c0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41584/0 (socket says 172.21.15.137:41584) 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 -- 172.21.15.137:0/2018201072 learned_addr learned my addr 172.21.15.137:0/2018201072 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 -- 172.21.15.137:0/2018201072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 msgr2=0x5650104aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34dffff640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e7fff640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 -- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 msgr2=0x5650104aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 -- 172.21.15.137:0/2018201072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565010119210 con 0x5650104a1af0 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34dffff640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e7fff640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34e77fe640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a9b90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d740 tx=0x7f34d80209d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c4b87f374a0c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.044+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2018201072 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8020040 con 0x5650104a1af0 2024-09-12T12:17:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2018201072 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f34d8018d60 con 0x5650104a1af0 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e001fd20 con 0x5650104a1af0 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2018201072 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34d809bba0 con 0x5650104a1af0 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 10 client.15936 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650101e5620 con 0x5650104a1af0 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2018201072 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e001fd20 con 0x5650104a1af0 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2018201072 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f34d80cc950 con 0x5650104a1af0 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 10 client.15936 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 20 client.15936 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5650104bcbc0 0x5650104befb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 4 client.15936 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565010413a20 con 0x5650104bcbc0 2024-09-12T12:17:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.045+0000 7f34dffff640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5650104bcbc0 0x5650104befb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.046+0000 7f34dffff640 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5650104bcbc0 0x5650104befb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e1c5567141a8f2 server_cookie=ff4683695fa4d0fb in_seq=0 out_seq=0 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.046+0000 7f34c6ffd640 10 client.15936 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2018201072 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565010413a20 con 0x5650104bcbc0 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34c6ffd640 10 client.15936 handle_command_reply: tid=0 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 client.15936 shutdown 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 msgr2=0x5650104a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a9b90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d740 tx=0x7f34d80209d0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 shutdown_connections 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104accb0 0x5650104aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5650104bcbc0 0x5650104befb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 --2- 172.21.15.137:0/2018201072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a1af0 0x5650104a9b90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 >> 172.21.15.137:0/2018201072 conn(0x56501049e060 msgr2=0x56501049d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34c6ffd640 0 client.15936 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 shutdown_connections 2024-09-12T12:17:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 1 -- 172.21.15.137:0/2018201072 wait complete. 2024-09-12T12:17:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.047+0000 7f34f105c740 20 client.15936 trim_cache size 0 max 16384 2024-09-12T12:17:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 Processor -- start 2024-09-12T12:17:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 -- start start 2024-09-12T12:17:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104accb0 0x5650104c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81affb0 con 0x5650104accb0 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81ad880 con 0x5650104a1af0 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.061+0000 7f34f105c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81ab6d0 con 0x5650104a2410 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104accb0 0x5650104c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104accb0 0x5650104c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41598/0 (socket says 172.21.15.137:41598) 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 -- 172.21.15.137:0/2090970897 learned_addr learned my addr 172.21.15.137:0/2090970897 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34dffff640 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e77fe640 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 -- 172.21.15.137:0/2090970897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 msgr2=0x5650104abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 -- 172.21.15.137:0/2090970897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 msgr2=0x5650104aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 -- 172.21.15.137:0/2090970897 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56501049eb20 con 0x5650104accb0 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34e7fff640 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104accb0 0x5650104c47d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f34e0014010 tx=0x7f34e00117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6bd5f391a938b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34dd7fa640 1 -- 172.21.15.137:0/2090970897 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0037070 con 0x5650104accb0 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34dd7fa640 1 -- 172.21.15.137:0/2090970897 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e0003970 con 0x5650104accb0 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.062+0000 7f34dd7fa640 1 -- 172.21.15.137:0/2090970897 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00278f0 con 0x5650104accb0 2024-09-12T12:17:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 -- 172.21.15.137:0/2090970897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104accb0 msgr2=0x5650104c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104accb0 0x5650104c47d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f34e0014010 tx=0x7f34e00117d0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 -- 172.21.15.137:0/2090970897 shutdown_connections 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104accb0 0x5650104c47d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104a2410 0x5650104abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 --2- 172.21.15.137:0/2090970897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 -- 172.21.15.137:0/2090970897 >> 172.21.15.137:0/2090970897 conn(0x56501049e060 msgr2=0x5650104ab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 -- 172.21.15.137:0/2090970897 shutdown_connections 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 -- 172.21.15.137:0/2090970897 wait complete. 2024-09-12T12:17:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 -- start start 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.063+0000 7f34f105c740 1 Processor -- start 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a2410 0x5650104b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104accb0 0x5650104b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81abd40 con 0x5650104a2410 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81ad1b0 con 0x5650104a1af0 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34e81ac410 con 0x5650104accb0 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 10 client.0 _ll_register_callbacks cb 0x7f34d80056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a2410 0x5650104b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a2410 0x5650104b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41604/0 (socket says 172.21.15.137:41604) 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 -- 172.21.15.137:0/2230989368 learned_addr learned my addr 172.21.15.137:0/2230989368 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34e77fe640 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34e7fff640 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104accb0 0x5650104b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 -- 172.21.15.137:0/2230989368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104accb0 msgr2=0x5650104b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104accb0 0x5650104b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 -- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 msgr2=0x5650104b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 -- 172.21.15.137:0/2230989368 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650104bf320 con 0x5650104a2410 2024-09-12T12:17:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34dffff640 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a2410 0x5650104b5520 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005940 tx=0x7f34d002aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb308bcc73bacfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2230989368 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0035040 con 0x5650104a2410 2024-09-12T12:17:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2230989368 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f34d0005030 con 0x5650104a2410 2024-09-12T12:17:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.064+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e0003970 con 0x5650104a2410 2024-09-12T12:17:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2230989368 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f34d0051050 con 0x5650104a2410 2024-09-12T12:17:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 10 client.15942 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565010527280 con 0x5650104a2410 2024-09-12T12:17:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2230989368 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e0003970 con 0x5650104a2410 2024-09-12T12:17:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2230989368 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f34d00035f0 con 0x5650104a2410 2024-09-12T12:17:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 10 client.15942 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 20 client.15942 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565010527a30 0x565010529e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 4 client.15942 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56501040dd70 con 0x565010527a30 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.065+0000 7f34e77fe640 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565010527a30 0x565010529e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.066+0000 7f34c6ffd640 10 client.15942 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.066+0000 7f34e77fe640 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565010527a30 0x565010529e50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319329b222e0d99a server_cookie=f793db6bba09814f in_seq=0 out_seq=0 2024-09-12T12:17:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34c6ffd640 1 -- 172.21.15.137:0/2230989368 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56501040dd70 con 0x565010527a30 2024-09-12T12:17:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34c6ffd640 10 client.15942 handle_command_reply: tid=0 2024-09-12T12:17:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 client.15942 shutdown 2024-09-12T12:17:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a2410 msgr2=0x5650104b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a2410 0x5650104b5520 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f34d0005940 tx=0x7f34d002aa20 comp rx=0 tx=0).stop 2024-09-12T12:17:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 shutdown_connections 2024-09-12T12:17:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5650104accb0 0x5650104b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x565010527a30 0x565010529e50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5650104a2410 0x5650104b5520 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 --2- 172.21.15.137:0/2230989368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650104a1af0 0x5650104b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34c6ffd640 0 client.15942 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 >> 172.21.15.137:0/2230989368 conn(0x56501049e060 msgr2=0x5650104a3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 shutdown_connections 2024-09-12T12:17:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 1 -- 172.21.15.137:0/2230989368 wait complete. 2024-09-12T12:17:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.096+0000 7f34f105c740 20 client.15942 trim_cache size 0 max 16384 2024-09-12T12:17:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:17:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:17:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:17:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:17:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34d40a2da0 msgr2=0x7f34d40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34d40a2da0 0x7f34d40a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f34e0022000 tx=0x7f34e0020000 comp rx=0 tx=0).stop 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8102ce0 msgr2=0x7f34e8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8102ce0 0x7f34e8198360 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34d800d770 tx=0x7f34d800dc30 comp rx=0 tx=0).stop 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 shutdown_connections 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34d40a2da0 0x7f34d40a5260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34e810c0b0 0x7f34e819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34e8103600 0x7f34e81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 --2- 172.21.15.137:0/3235502606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34e8102ce0 0x7f34e8198360 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 >> 172.21.15.137:0/3235502606 conn(0x7f34e8077990 msgr2=0x7f34e80fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 shutdown_connections 2024-09-12T12:17:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:39.097+0000 7f34ee1e4640 1 -- 172.21.15.137:0/3235502606 wait complete. 2024-09-12T12:17:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:39 smithi189 ceph-mon[35475]: pgmap v136: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-12T12:17:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:39 smithi171 ceph-mon[35098]: pgmap v136: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-12T12:17:39.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.70 MB/sec execute 74 sec latency 14398.301 ms 2024-09-12T12:17:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:39 smithi137 ceph-mon[30950]: pgmap v136: 97 pgs: 97 active+clean; 79 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 1.7 MiB/s wr, 104 op/s 2024-09-12T12:17:40.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.61 MB/sec execute 75 sec latency 15398.367 ms 2024-09-12T12:17:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:41 smithi189 ceph-mon[35475]: pgmap v137: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-09-12T12:17:41.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:41 smithi171 ceph-mon[35098]: pgmap v137: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-09-12T12:17:41.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.53 MB/sec execute 76 sec latency 16398.459 ms 2024-09-12T12:17:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:41 smithi137 ceph-mon[30950]: pgmap v137: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-09-12T12:17:42.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.44 MB/sec execute 77 sec latency 17398.531 ms 2024-09-12T12:17:42.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:17:43.062 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:17:43.490 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:43 smithi137 ceph-mon[30950]: pgmap v138: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-09-12T12:17:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:43 smithi189 ceph-mon[35475]: pgmap v138: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-09-12T12:17:43.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 -- 172.21.15.137:0/3585216602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c103660 msgr2=0x7fa97c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 --2- 172.21.15.137:0/3585216602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c103660 0x7fa97c10bb50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa970009900 tx=0x7fa97002a480 comp rx=0 tx=0).stop 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 -- 172.21.15.137:0/3585216602 shutdown_connections 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 --2- 172.21.15.137:0/3585216602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97c10c170 0x7fa97c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 --2- 172.21.15.137:0/3585216602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c103660 0x7fa97c10bb50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 --2- 172.21.15.137:0/3585216602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa97c102d40 0x7fa97c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 -- 172.21.15.137:0/3585216602 >> 172.21.15.137:0/3585216602 conn(0x7fa97c0fc7a0 msgr2=0x7fa97c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 -- 172.21.15.137:0/3585216602 shutdown_connections 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.570+0000 7fa984561640 1 -- 172.21.15.137:0/3585216602 wait complete. 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 Processor -- start 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 -- start start 2024-09-12T12:17:43.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c102d40 0x7fa97c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa97c103660 0x7fa97c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97c10c170 0x7fa97c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c111fb0 con 0x7fa97c102d40 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c111e30 con 0x7fa97c10c170 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa984561640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa97c112130 con 0x7fa97c103660 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa9822fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c102d40 0x7fa97c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa9822fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c102d40 0x7fa97c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34620/0 (socket says 172.21.15.137:34620) 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa9822fd640 1 -- 172.21.15.137:0/1296021533 learned_addr learned my addr 172.21.15.137:0/1296021533 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa981afc640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa97c103660 0x7fa97c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.571+0000 7fa982afe640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97c10c170 0x7fa97c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa9822fd640 1 -- 172.21.15.137:0/1296021533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa97c103660 msgr2=0x7fa97c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa9822fd640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa97c103660 0x7fa97c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa9822fd640 1 -- 172.21.15.137:0/1296021533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97c10c170 msgr2=0x7fa97c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa9822fd640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97c10c170 0x7fa97c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa9822fd640 1 -- 172.21.15.137:0/1296021533 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97c1a6a20 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa9822fd640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c102d40 0x7fa97c1009a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa97400b4a0 tx=0x7fa97400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa97b7fe640 1 -- 172.21.15.137:0/1296021533 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974014070 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa97b7fe640 1 -- 172.21.15.137:0/1296021533 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa974010af0 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97c1a6d10 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa97b7fe640 1 -- 172.21.15.137:0/1296021533 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974017d10 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.572+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97c1a71c0 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.573+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97c103120 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.574+0000 7fa97b7fe640 1 -- 172.21.15.137:0/1296021533 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa974017070 con 0x7fa97c102d40 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.575+0000 7fa97b7fe640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa9640a2cf0 0x7fa9640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:43.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.575+0000 7fa981afc640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa9640a2cf0 0x7fa9640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:43.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.575+0000 7fa97b7fe640 1 -- 172.21.15.137:0/1296021533 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fa9740ce5b0 con 0x7fa97c102d40 2024-09-12T12:17:43.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.575+0000 7fa981afc640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa9640a2cf0 0x7fa9640a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa970009230 tx=0x7fa9700058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:43.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.579+0000 7fa97b7fe640 1 -- 172.21.15.137:0/1296021533 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa97409ace0 con 0x7fa97c102d40 2024-09-12T12:17:43.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:43 smithi171 ceph-mon[35098]: pgmap v138: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 1.1 MiB/s wr, 68 op/s 2024-09-12T12:17:43.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.36 MB/sec execute 78 sec latency 18398.605 ms 2024-09-12T12:17:43.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.778+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa97c107210 con 0x7fa97c102d40 2024-09-12T12:17:43.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.779+0000 7fa97b7fe640 1 -- 172.21.15.137:0/1296021533 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fa97409efe0 con 0x7fa97c102d40 2024-09-12T12:17:43.780 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:17:43.780 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":29,"btime":"2024-09-12T12:17:07:265882+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:17:06.970365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:17:43.780 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 29 2024-09-12T12:17:43.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa9640a2cf0 msgr2=0x7fa9640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa9640a2cf0 0x7fa9640a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa970009230 tx=0x7fa9700058d0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c102d40 msgr2=0x7fa97c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c102d40 0x7fa97c1009a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa97400b4a0 tx=0x7fa97400b960 comp rx=0 tx=0).stop 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 shutdown_connections 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa9640a2cf0 0x7fa9640a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97c10c170 0x7fa97c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa97c103660 0x7fa97c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 --2- 172.21.15.137:0/1296021533 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa97c102d40 0x7fa97c1009a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 >> 172.21.15.137:0/1296021533 conn(0x7fa97c0fc7a0 msgr2=0x7fa97c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 shutdown_connections 2024-09-12T12:17:43.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:43.782+0000 7fa984561640 1 -- 172.21.15.137:0/1296021533 wait complete. 2024-09-12T12:17:44.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:44 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1296021533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:44.607 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 3a364b90 --release --await' 2024-09-12T12:17:44.608 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 3a364b90 --release --await 2024-09-12T12:17:44.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:44 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1296021533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:44.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 302435 6.28 MB/sec execute 79 sec latency 19398.661 ms 2024-09-12T12:17:44.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:44 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1296021533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:17:44.856 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:17:45.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.328+0000 7fd41f77f640 1 -- 172.21.15.137:0/644655199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4181055b0 msgr2=0x7fd41810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.328+0000 7fd41f77f640 1 --2- 172.21.15.137:0/644655199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4181055b0 0x7fd41810d130 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd41000adb0 tx=0x7fd41002b7a0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.328+0000 7fd41f77f640 1 -- 172.21.15.137:0/644655199 shutdown_connections 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.328+0000 7fd41f77f640 1 --2- 172.21.15.137:0/644655199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4181055b0 0x7fd41810d130 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.328+0000 7fd41f77f640 1 --2- 172.21.15.137:0/644655199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd418100e30 0x7fd418104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.328+0000 7fd41f77f640 1 --2- 172.21.15.137:0/644655199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd418100480 0x7fd418100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.328+0000 7fd41f77f640 1 -- 172.21.15.137:0/644655199 >> 172.21.15.137:0/644655199 conn(0x7fd418077210 msgr2=0x7fd4180fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 -- 172.21.15.137:0/644655199 shutdown_connections 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 -- 172.21.15.137:0/644655199 wait complete. 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 Processor -- start 2024-09-12T12:17:45.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 -- start start 2024-09-12T12:17:45.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd418100480 0x7fd418198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd418100e30 0x7fd418198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4181055b0 0x7fd41819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810fdd0 con 0x7fd418100480 2024-09-12T12:17:45.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810fc50 con 0x7fd418100e30 2024-09-12T12:17:45.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41f77f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd41810ff50 con 0x7fd4181055b0 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41d51b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd418100480 0x7fd418198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41d51b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd418100480 0x7fd418198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34634/0 (socket says 172.21.15.137:34634) 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.329+0000 7fd41d51b640 1 -- 172.21.15.137:0/3793795631 learned_addr learned my addr 172.21.15.137:0/3793795631 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4181055b0 0x7fd41819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd418100e30 0x7fd418198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41d51b640 1 -- 172.21.15.137:0/3793795631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4181055b0 msgr2=0x7fd41819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41d51b640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4181055b0 0x7fd41819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41d51b640 1 -- 172.21.15.137:0/3793795631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd418100e30 msgr2=0x7fd418198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41d51b640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd418100e30 0x7fd418198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41d51b640 1 -- 172.21.15.137:0/3793795631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41819d4c0 con 0x7fd418100480 2024-09-12T12:17:45.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41d51b640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd418100480 0x7fd418198510 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c970 tx=0x7fd41400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:45.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd40e7fc640 1 -- 172.21.15.137:0/3793795631 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414007bb0 con 0x7fd418100480 2024-09-12T12:17:45.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd40e7fc640 1 -- 172.21.15.137:0/3793795631 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd414007d50 con 0x7fd418100480 2024-09-12T12:17:45.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41819d7b0 con 0x7fd418100480 2024-09-12T12:17:45.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd40e7fc640 1 -- 172.21.15.137:0/3793795631 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41400f450 con 0x7fd418100480 2024-09-12T12:17:45.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.330+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a5090 con 0x7fd418100480 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.332+0000 7fd40e7fc640 1 -- 172.21.15.137:0/3793795631 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd414004f40 con 0x7fd418100480 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 Processor -- start 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 -- start start 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b625440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b62fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b632630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55572b58a620 con 0x55572b625060 2024-09-12T12:17:45.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55572b531e50 con 0x55572b630240 2024-09-12T12:17:45.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd421b58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55572b5865e0 con 0x55572b625980 2024-09-12T12:17:45.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd41dd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b625440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd40e7fc640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd4040a2cf0 0x7fd4040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd41dd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b625440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34638/0 (socket says 172.21.15.137:34638) 2024-09-12T12:17:45.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd41dd1c640 1 -- 172.21.15.137:0/130358854 learned_addr learned my addr 172.21.15.137:0/130358854 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:45.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.333+0000 7fd41d51b640 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b62fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b632630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd4040a2cf0 0x7fd4040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd40e7fc640 1 -- 172.21.15.137:0/3793795631 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd4140cf0d0 con 0x7fd418100480 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41cd1a640 1 -- 172.21.15.137:0/130358854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 msgr2=0x55572b62fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b62fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41cd1a640 1 -- 172.21.15.137:0/130358854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 msgr2=0x55572b625440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b625440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41cd1a640 1 -- 172.21.15.137:0/130358854 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55572b522d40 con 0x55572b630240 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd4040a2cf0 0x7fd4040a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd41000a540 tx=0x7fd410005f50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b632630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd408009970 tx=0x7fd408009e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8978e3ac9c0c12 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd4037fe640 1 -- 172.21.15.137:0/130358854 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408017070 con 0x55572b630240 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd4037fe640 1 -- 172.21.15.137:0/130358854 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd408013470 con 0x55572b630240 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.334+0000 7fd4037fe640 1 -- 172.21.15.137:0/130358854 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408012610 con 0x55572b630240 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 -- 172.21.15.137:0/130358854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 msgr2=0x55572b632630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b632630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd408009970 tx=0x7fd408009e30 comp rx=0 tx=0).stop 2024-09-12T12:17:45.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 -- 172.21.15.137:0/130358854 shutdown_connections 2024-09-12T12:17:45.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b632630 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b62fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 --2- 172.21.15.137:0/130358854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b625440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 -- 172.21.15.137:0/130358854 >> 172.21.15.137:0/130358854 conn(0x55572b6215f0 msgr2=0x55572b6219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:45.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 -- 172.21.15.137:0/130358854 shutdown_connections 2024-09-12T12:17:45.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 -- 172.21.15.137:0/130358854 wait complete. 2024-09-12T12:17:45.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 -- start start 2024-09-12T12:17:45.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.335+0000 7fd421b58740 1 Processor -- start 2024-09-12T12:17:45.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd421b58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b62d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd421b58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625980 0x55572b62d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd421b58740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b62db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd421b58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55572b632e10 con 0x55572b625980 2024-09-12T12:17:45.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd421b58740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55572b512f10 con 0x55572b630240 2024-09-12T12:17:45.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd421b58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55572b632f90 con 0x55572b625060 2024-09-12T12:17:45.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd421b58740 10 client.0 _ll_register_callbacks cb 0x55572b536db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:45.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625980 0x55572b62d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625980 0x55572b62d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34648/0 (socket says 172.21.15.137:34648) 2024-09-12T12:17:45.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 -- 172.21.15.137:0/2361413150 learned_addr learned my addr 172.21.15.137:0/2361413150 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:45.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b62db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41d51b640 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b62d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 -- 172.21.15.137:0/2361413150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 msgr2=0x55572b62d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b62d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 -- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 msgr2=0x55572b62db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b62db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.336+0000 7fd41cd1a640 1 -- 172.21.15.137:0/2361413150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572b299580 con 0x55572b625980 2024-09-12T12:17:45.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625980 0x55572b62d5c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd408004810 tx=0x7fd408006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795ba4bc79cbd8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:45.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd400ff9640 1 -- 172.21.15.137:0/2361413150 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408017050 con 0x55572b625980 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd400ff9640 1 -- 172.21.15.137:0/2361413150 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd40801c450 con 0x55572b625980 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd408013470 con 0x55572b625980 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd400ff9640 1 -- 172.21.15.137:0/2361413150 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd40801c650 con 0x55572b625980 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 10 client.15960 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55572b363960 con 0x55572b625980 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd400ff9640 1 -- 172.21.15.137:0/2361413150 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd408013470 con 0x55572b625980 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd400ff9640 1 -- 172.21.15.137:0/2361413150 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd4080308c0 con 0x55572b625980 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 10 client.15960 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 10 client.15960 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:45.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 20 client.15960 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b63fc50 0x55572b642040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 4 client.15960 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.337+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55572b597b90 con 0x55572b63fc50 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.338+0000 7fd41d51b640 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b63fc50 0x55572b642040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.338+0000 7fd400ff9640 10 client.15960 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.338+0000 7fd41d51b640 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b63fc50 0x55572b642040 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc6c30ea7457eeb server_cookie=9593c83e5c32428a in_seq=0 out_seq=0 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd400ff9640 1 -- 172.21.15.137:0/2361413150 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55572b597b90 con 0x55572b63fc50 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd400ff9640 10 client.15960 handle_command_reply: tid=0 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 client.15960 shutdown 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625980 msgr2=0x55572b62d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625980 0x55572b62d5c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd408004810 tx=0x7fd408006e40 comp rx=0 tx=0).stop 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 shutdown_connections 2024-09-12T12:17:45.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b62db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b63fc50 0x55572b642040 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625980 0x55572b62d5c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 --2- 172.21.15.137:0/2361413150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b62d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 >> 172.21.15.137:0/2361413150 conn(0x55572b6215f0 msgr2=0x55572b627320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd400ff9640 0 client.15960 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 shutdown_connections 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 1 -- 172.21.15.137:0/2361413150 wait complete. 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.339+0000 7fd421b58740 20 client.15960 trim_cache size 0 max 16384 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 Processor -- start 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 -- start start 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b620640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b6476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b647be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181b0150 con 0x55572b625060 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181ada20 con 0x55572b630240 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd421b58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181ab870 con 0x55572b625980 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd41d51b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b620640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd41d51b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b620640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34652/0 (socket says 172.21.15.137:34652) 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.356+0000 7fd41d51b640 1 -- 172.21.15.137:0/2400167203 learned_addr learned my addr 172.21.15.137:0/2400167203 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41cd1a640 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b6476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b647be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41d51b640 1 -- 172.21.15.137:0/2400167203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 msgr2=0x55572b6476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41d51b640 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b6476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41d51b640 1 -- 172.21.15.137:0/2400167203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 msgr2=0x55572b647be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41d51b640 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b647be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41d51b640 1 -- 172.21.15.137:0/2400167203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55572b621420 con 0x55572b625060 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd41d51b640 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b620640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd4140cce70 tx=0x7fd41406e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71d13262f37ba2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd402ffd640 1 -- 172.21.15.137:0/2400167203 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41401d040 con 0x55572b625060 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd402ffd640 1 -- 172.21.15.137:0/2400167203 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41409bd40 con 0x55572b625060 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.357+0000 7fd402ffd640 1 -- 172.21.15.137:0/2400167203 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4140ccaf0 con 0x55572b625060 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 -- 172.21.15.137:0/2400167203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 msgr2=0x55572b620640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b620640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd4140cce70 tx=0x7fd41406e6b0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 -- 172.21.15.137:0/2400167203 shutdown_connections 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b630240 0x55572b647be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625980 0x55572b6476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 --2- 172.21.15.137:0/2400167203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b625060 0x55572b620640 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 -- 172.21.15.137:0/2400167203 >> 172.21.15.137:0/2400167203 conn(0x55572b6215f0 msgr2=0x55572b61ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 -- 172.21.15.137:0/2400167203 shutdown_connections 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 -- 172.21.15.137:0/2400167203 wait complete. 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 -- start start 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.358+0000 7fd421b58740 1 Processor -- start 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd421b58740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b6381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd421b58740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b625980 0x55572b638720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd421b58740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b630240 0x55572b6351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd421b58740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181abee0 con 0x55572b630240 2024-09-12T12:17:45.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd421b58740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181ad350 con 0x55572b625980 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd421b58740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181ac5b0 con 0x55572b625060 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd421b58740 10 client.0 _ll_register_callbacks cb 0x7fd4080097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b630240 0x55572b6351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b630240 0x55572b6351b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34662/0 (socket says 172.21.15.137:34662) 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 -- 172.21.15.137:0/3029427031 learned_addr learned my addr 172.21.15.137:0/3029427031 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41d51b640 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b6381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 -- 172.21.15.137:0/3029427031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 msgr2=0x55572b6381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b6381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 -- 172.21.15.137:0/3029427031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b625980 msgr2=0x55572b638720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b625980 0x55572b638720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 -- 172.21.15.137:0/3029427031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55572b622280 con 0x55572b630240 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd41dd1c640 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b630240 0x55572b6351b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd410005fe0 tx=0x7fd410049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43c9b1bdce79449 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.359+0000 7fd400ff9640 1 -- 172.21.15.137:0/3029427031 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100319a0 con 0x55572b630240 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd400ff9640 1 -- 172.21.15.137:0/3029427031 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd410048040 con 0x55572b630240 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41409bd40 con 0x55572b630240 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd400ff9640 1 -- 172.21.15.137:0/3029427031 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd410031b40 con 0x55572b630240 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 10 client.15966 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:45.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55572b6aacd0 con 0x55572b630240 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd400ff9640 1 -- 172.21.15.137:0/3029427031 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd41409bd40 con 0x55572b630240 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd400ff9640 1 -- 172.21.15.137:0/3029427031 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fd41004d440 con 0x55572b630240 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 10 client.15966 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 10 client.15966 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 20 client.15966 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b6ab4d0 0x55572b6ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 4 client.15966 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3a364b90", "release": true, "await": true}] 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3a364b90", "release": true, "await": true}) -- 0x55572b61c950 con 0x55572b6ab4d0 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.360+0000 7fd41d51b640 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b6ab4d0 0x55572b6ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:45.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.361+0000 7fd41d51b640 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b6ab4d0 0x55572b6ad8f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a9c18bbf1f6114 server_cookie=ad04916359baafed in_seq=0 out_seq=0 2024-09-12T12:17:45.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.361+0000 7fd400ff9640 10 client.15966 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:45.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd400ff9640 1 -- 172.21.15.137:0/3029427031 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55572b61c950 con 0x55572b6ab4d0 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd400ff9640 10 client.15966 handle_command_reply: tid=0 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 client.15966 shutdown 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b630240 msgr2=0x55572b6351b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b630240 0x55572b6351b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd410005fe0 tx=0x7fd410049000 comp rx=0 tx=0).stop 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 shutdown_connections 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55572b6ab4d0 0x55572b6ad8f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55572b630240 0x55572b6351b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55572b625980 0x55572b638720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd400ff9640 0 client.15966 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 --2- 172.21.15.137:0/3029427031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55572b625060 0x55572b6381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 >> 172.21.15.137:0/3029427031 conn(0x55572b6215f0 msgr2=0x55572b62f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:45.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 shutdown_connections 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 1 -- 172.21.15.137:0/3029427031 wait complete. 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.367+0000 7fd421b58740 20 client.15966 trim_cache size 0 max 16384 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 29, 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 20, 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stdout: "3a364b90": { 2024-09-12T12:17:45.387 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 20, 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 140.2, 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:17:45.388 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:45.389 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:45.389 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:45.389 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:45.389 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:17:45.389 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd4040a2cf0 msgr2=0x7fd4040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd4040a2cf0 0x7fd4040a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd41000a540 tx=0x7fd410005f50 comp rx=0 tx=0).stop 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd418100480 msgr2=0x7fd418198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd418100480 0x7fd418198510 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c970 tx=0x7fd41400cd70 comp rx=0 tx=0).stop 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 shutdown_connections 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd4040a2cf0 0x7fd4040a51b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4181055b0 0x7fd41819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd418100e30 0x7fd418198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 --2- 172.21.15.137:0/3793795631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd418100480 0x7fd418198510 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 >> 172.21.15.137:0/3793795631 conn(0x7fd418077210 msgr2=0x7fd4180feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 shutdown_connections 2024-09-12T12:17:45.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:17:45.368+0000 7fd41f77f640 1 -- 172.21.15.137:0/3793795631 wait complete. 2024-09-12T12:17:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:45 smithi137 ceph-mon[30950]: pgmap v139: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 823 KiB/s wr, 48 op/s 2024-09-12T12:17:45.710 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:45 smithi171 ceph-mon[35098]: pgmap v139: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 823 KiB/s wr, 48 op/s 2024-09-12T12:17:45.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 303416 6.26 MB/sec execute 80 sec latency 20041.722 ms 2024-09-12T12:17:45.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:45 smithi189 ceph-mon[35475]: pgmap v139: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 823 KiB/s wr, 48 op/s 2024-09-12T12:17:46.236 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3a364b90', total seconds elapsed: 23.6 2024-09-12T12:17:46.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 305685 6.35 MB/sec execute 81 sec latency 12.307 ms 2024-09-12T12:17:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:47 smithi171 ceph-mon[35098]: pgmap v140: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 825 KiB/s wr, 51 op/s 2024-09-12T12:17:47.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 308574 6.48 MB/sec execute 82 sec latency 11.310 ms 2024-09-12T12:17:47.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:47 smithi137 ceph-mon[30950]: pgmap v140: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 825 KiB/s wr, 51 op/s 2024-09-12T12:17:47.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:47 smithi189 ceph-mon[35475]: pgmap v140: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 825 KiB/s wr, 51 op/s 2024-09-12T12:17:48.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 310251 6.49 MB/sec execute 83 sec latency 12.888 ms 2024-09-12T12:17:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:49 smithi171 ceph-mon[35098]: pgmap v141: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 242 KiB/s wr, 22 op/s 2024-09-12T12:17:49.727 INFO:tasks.workunit.client.0.smithi137.stdout: 1 312609 6.58 MB/sec execute 84 sec latency 12.140 ms 2024-09-12T12:17:49.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:49 smithi137 ceph-mon[30950]: pgmap v141: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 242 KiB/s wr, 22 op/s 2024-09-12T12:17:49.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:49 smithi189 ceph-mon[35475]: pgmap v141: 97 pgs: 97 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 242 KiB/s wr, 22 op/s 2024-09-12T12:17:50.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 315643 6.71 MB/sec execute 85 sec latency 11.361 ms 2024-09-12T12:17:51.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:51 smithi171 ceph-mon[35098]: pgmap v142: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 855 KiB/s wr, 104 op/s 2024-09-12T12:17:51.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 317326 6.71 MB/sec execute 86 sec latency 12.264 ms 2024-09-12T12:17:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:51 smithi137 ceph-mon[30950]: pgmap v142: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 855 KiB/s wr, 104 op/s 2024-09-12T12:17:51.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:51 smithi189 ceph-mon[35475]: pgmap v142: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 855 KiB/s wr, 104 op/s 2024-09-12T12:17:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 -- 172.21.15.137:0/1218904891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 msgr2=0x7f311c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1218904891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c10d880 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3114009cd0 tx=0x7f311402a8e0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 -- 172.21.15.137:0/1218904891 shutdown_connections 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1218904891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c10d880 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1218904891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f311c1009b0 0x7f311c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1218904891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f311c1069c0 0x7f311c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 -- 172.21.15.137:0/1218904891 >> 172.21.15.137:0/1218904891 conn(0x7f311c077990 msgr2=0x7f311c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 -- 172.21.15.137:0/1218904891 shutdown_connections 2024-09-12T12:17:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 -- 172.21.15.137:0/1218904891 wait complete. 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 Processor -- start 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.186+0000 7f3122e7b640 1 -- start start 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3122e7b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f311c1009b0 0x7f311c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3122e7b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3122e7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f311c1069c0 0x7f311c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3122e7b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f311c110e00 con 0x7f311c101350 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3122e7b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f311c110c80 con 0x7f311c1009b0 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3122e7b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f311c110f80 con 0x7f311c1069c0 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47116/0 (socket says 172.21.15.137:47116) 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 -- 172.21.15.137:0/1288317538 learned_addr learned my addr 172.21.15.137:0/1288317538 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3120c17640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f311c1009b0 0x7f311c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3121418640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f311c1069c0 0x7f311c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 -- 172.21.15.137:0/1288317538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f311c1069c0 msgr2=0x7f311c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f311c1069c0 0x7f311c19bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 -- 172.21.15.137:0/1288317538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f311c1009b0 msgr2=0x7f311c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f311c1009b0 0x7f311c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 -- 172.21.15.137:0/1288317538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311c19c200 con 0x7f311c101350 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f311bfff640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c197750 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f310800eb00 tx=0x7f310800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3119ffb640 1 -- 172.21.15.137:0/1288317538 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310800bc30 con 0x7f311c101350 2024-09-12T12:17:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3119ffb640 1 -- 172.21.15.137:0/1288317538 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310800bdd0 con 0x7f311c101350 2024-09-12T12:17:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311c19c4f0 con 0x7f311c101350 2024-09-12T12:17:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.187+0000 7f3119ffb640 1 -- 172.21.15.137:0/1288317538 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108017410 con 0x7f311c101350 2024-09-12T12:17:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.188+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311c0698a0 con 0x7f311c101350 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.190+0000 7f3119ffb640 1 -- 172.21.15.137:0/1288317538 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3108005020 con 0x7f311c101350 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.190+0000 7f3125c18740 1 Processor -- start 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.190+0000 7f3125c18740 1 -- start start 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3125c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128adaf0 0x5640128aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3125c18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128ae410 0x5640128b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3125c18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128bb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3125c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5640128122f0 con 0x5640128adaf0 2024-09-12T12:17:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3125c18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5640127ba270 con 0x5640128b8cb0 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3125c18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56401280e2b0 con 0x5640128ae410 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3119ffb640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f310c0a2e20 0x7f310c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128ae410 0x5640128b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3121418640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128adaf0 0x5640128aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128ae410 0x5640128b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:38260/0 (socket says 172.21.15.137:38260) 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 -- 172.21.15.137:0/2607236451 learned_addr learned my addr 172.21.15.137:0/2607236451 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3121418640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128adaf0 0x5640128aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47118/0 (socket says 172.21.15.137:47118) 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f311bfff640 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128bb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3121418640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f310c0a2e20 0x7f310c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3119ffb640 1 -- 172.21.15.137:0/1288317538 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f31080d0c80 con 0x7f311c101350 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 -- 172.21.15.137:0/2607236451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 msgr2=0x5640128bb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128bb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 -- 172.21.15.137:0/2607236451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128adaf0 msgr2=0x5640128aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128adaf0 0x5640128aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 -- 172.21.15.137:0/2607236451 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640127b9340 con 0x5640128ae410 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3120c17640 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128ae410 0x5640128b8770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3110009c70 tx=0x7f3110011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e7ea4ad1ee2d0c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.191+0000 7f3121418640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f310c0a2e20 0x7f310c0a52e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3114004ab0 tx=0x7f3114004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3106ffd640 1 -- 172.21.15.137:0/2607236451 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110004230 con 0x5640128ae410 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3106ffd640 1 -- 172.21.15.137:0/2607236451 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31100043d0 con 0x5640128ae410 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3106ffd640 1 -- 172.21.15.137:0/2607236451 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100197c0 con 0x5640128ae410 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 -- 172.21.15.137:0/2607236451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128ae410 msgr2=0x5640128b8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128ae410 0x5640128b8770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3110009c70 tx=0x7f3110011810 comp rx=0 tx=0).stop 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 -- 172.21.15.137:0/2607236451 shutdown_connections 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128bb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128ae410 0x5640128b8770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 --2- 172.21.15.137:0/2607236451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128adaf0 0x5640128aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 -- 172.21.15.137:0/2607236451 >> 172.21.15.137:0/2607236451 conn(0x5640128aa060 msgr2=0x5640128aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 -- 172.21.15.137:0/2607236451 shutdown_connections 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.192+0000 7f3125c18740 1 -- 172.21.15.137:0/2607236451 wait complete. 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 -- start start 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 Processor -- start 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5640128bb880 con 0x5640128ae410 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56401279b290 con 0x5640128b8cb0 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5640128bba00 con 0x5640128adaf0 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3125c18740 10 client.0 _ll_register_callbacks cb 0x5640127cc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3120c17640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47132/0 (socket says 172.21.15.137:47132) 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f311bfff640 1 -- 172.21.15.137:0/707969522 learned_addr learned my addr 172.21.15.137:0/707969522 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f3121418640 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f311bfff640 1 -- 172.21.15.137:0/707969522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 msgr2=0x5640128b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f311bfff640 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.193+0000 7f311bfff640 1 -- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 msgr2=0x5640128b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f311bfff640 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f311bfff640 1 -- 172.21.15.137:0/707969522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564012525210 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3120c17640 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f311bfff640 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128b6080 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3108099c10 tx=0x7f310806c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465eb8304fa31ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f30fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f30fbfff640 1 -- 172.21.15.137:0/707969522 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080ce840 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f30fbfff640 1 -- 172.21.15.137:0/707969522 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f31080a1130 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31100043d0 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f30fbfff640 1 -- 172.21.15.137:0/707969522 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31080a1540 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 10 client.15978 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640125f1620 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f30fbfff640 1 -- 172.21.15.137:0/707969522 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31100043d0 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f30fbfff640 1 -- 172.21.15.137:0/707969522 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f310809c690 con 0x5640128ae410 2024-09-12T12:17:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 10 client.15978 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 20 client.15978 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5640128c8bb0 0x5640128cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 4 client.15978 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.194+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56401281fa20 con 0x5640128c8bb0 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.195+0000 7f3120c17640 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5640128c8bb0 0x5640128cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.195+0000 7f30fbfff640 10 client.15978 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.195+0000 7f3120c17640 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5640128c8bb0 0x5640128cafc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95f5232bca05505 server_cookie=3e48fdf3db0e07b8 in_seq=0 out_seq=0 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f30fbfff640 1 -- 172.21.15.137:0/707969522 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56401281fa20 con 0x5640128c8bb0 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f30fbfff640 10 client.15978 handle_command_reply: tid=0 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 client.15978 shutdown 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 msgr2=0x5640128b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128b6080 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3108099c10 tx=0x7f310806c170 comp rx=0 tx=0).stop 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 shutdown_connections 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5640128c8bb0 0x5640128cafc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128b6080 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 --2- 172.21.15.137:0/707969522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 >> 172.21.15.137:0/707969522 conn(0x5640128aa060 msgr2=0x5640128afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f30fbfff640 0 client.15978 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 shutdown_connections 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 1 -- 172.21.15.137:0/707969522 wait complete. 2024-09-12T12:17:52.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.196+0000 7f3125c18740 20 client.15978 trim_cache size 0 max 16384 2024-09-12T12:17:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 Processor -- start 2024-09-12T12:17:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 -- start start 2024-09-12T12:17:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1add30 con 0x5640128ae410 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1ab600 con 0x5640128b8cb0 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3125c18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1a9450 con 0x5640128adaf0 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47134/0 (socket says 172.21.15.137:47134) 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 -- 172.21.15.137:0/470342829 learned_addr learned my addr 172.21.15.137:0/470342829 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3120c17640 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f3121418640 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 -- 172.21.15.137:0/470342829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 msgr2=0x5640128b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 -- 172.21.15.137:0/470342829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 msgr2=0x5640128d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 -- 172.21.15.137:0/470342829 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640128ae240 con 0x5640128ae410 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.211+0000 7f311bfff640 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128d0880 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3108099bc0 tx=0x7f310809d140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6ef80d770569da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f31067fc640 1 -- 172.21.15.137:0/470342829 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080a5e00 con 0x5640128ae410 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f31067fc640 1 -- 172.21.15.137:0/470342829 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310809c690 con 0x5640128ae410 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f31067fc640 1 -- 172.21.15.137:0/470342829 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310809c100 con 0x5640128ae410 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 -- 172.21.15.137:0/470342829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 msgr2=0x5640128d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128d0880 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3108099bc0 tx=0x7f310809d140 comp rx=0 tx=0).stop 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 -- 172.21.15.137:0/470342829 shutdown_connections 2024-09-12T12:17:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128b8cb0 0x5640128d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128d0880 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 --2- 172.21.15.137:0/470342829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128adaf0 0x5640128b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 -- 172.21.15.137:0/470342829 >> 172.21.15.137:0/470342829 conn(0x5640128aa060 msgr2=0x5640128b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 -- 172.21.15.137:0/470342829 shutdown_connections 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.212+0000 7f3125c18740 1 -- 172.21.15.137:0/470342829 wait complete. 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 -- start start 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 Processor -- start 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128adaf0 0x5640128c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128b8cb0 0x5640128b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1a9ac0 con 0x5640128ae410 2024-09-12T12:17:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1aaf30 con 0x5640128adaf0 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1aa190 con 0x5640128b8cb0 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3125c18740 10 client.0 _ll_register_callbacks cb 0x7f310809d090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3120c17640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128adaf0 0x5640128c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3120c17640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128adaf0 0x5640128c0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:47688/0 (socket says 172.21.15.137:47688) 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f311bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128c13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47148/0 (socket says 172.21.15.137:47148) 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3120c17640 1 -- 172.21.15.137:0/628386362 learned_addr learned my addr 172.21.15.137:0/628386362 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f3121418640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128b8cb0 0x5640128b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f311bfff640 1 -- 172.21.15.137:0/628386362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128b8cb0 msgr2=0x5640128b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f311bfff640 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128b8cb0 0x5640128b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f311bfff640 1 -- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128adaf0 msgr2=0x5640128c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.213+0000 7f311bfff640 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128adaf0 0x5640128c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f311bfff640 1 -- 172.21.15.137:0/628386362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640128b2a30 con 0x5640128ae410 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3120c17640 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128adaf0 0x5640128c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f311bfff640 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128c13c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3108099bc0 tx=0x7f310809caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116404813537a3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f30fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f30fbfff640 1 -- 172.21.15.137:0/628386362 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080ce850 con 0x5640128ae410 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f30fbfff640 1 -- 172.21.15.137:0/628386362 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f31080a5b70 con 0x5640128ae410 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310809c690 con 0x5640128ae410 2024-09-12T12:17:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f30fbfff640 1 -- 172.21.15.137:0/628386362 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31080a5e00 con 0x5640128ae410 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 10 client.15984 fetch_fsmap learned FSMap version 29 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640129375d0 con 0x5640128ae410 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f30fbfff640 1 -- 172.21.15.137:0/628386362 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310809c690 con 0x5640128ae410 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f30fbfff640 1 -- 172.21.15.137:0/628386362 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f31080a28e0 con 0x5640128ae410 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 10 client.15984 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 20 client.15984 populate_metadata read hostname 'smithi137' 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564012937aa0 0x564012939e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 4 client.15984 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564012819d70 con 0x564012937aa0 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.214+0000 7f3120c17640 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564012937aa0 0x564012939e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.215+0000 7f30fbfff640 10 client.15984 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.215+0000 7f3120c17640 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564012937aa0 0x564012939e90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698c901e7062f8d4 server_cookie=6fb2ff98628aa667 in_seq=0 out_seq=0 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.264+0000 7f30fbfff640 1 -- 172.21.15.137:0/628386362 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564012819d70 con 0x564012937aa0 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.264+0000 7f30fbfff640 10 client.15984 handle_command_reply: tid=0 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.264+0000 7f3125c18740 1 client.15984 shutdown 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 msgr2=0x5640128c13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128c13c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3108099bc0 tx=0x7f310809caf0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 shutdown_connections 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640128b8cb0 0x5640128b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564012937aa0 0x564012939e90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5640128ae410 0x5640128c13c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 --2- 172.21.15.137:0/628386362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5640128adaf0 0x5640128c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f30fbfff640 0 client.15984 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 >> 172.21.15.137:0/628386362 conn(0x5640128aa060 msgr2=0x5640128af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 shutdown_connections 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 1 -- 172.21.15.137:0/628386362 wait complete. 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3125c18740 20 client.15984 trim_cache size 0 max 16384 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:17:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:17:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:17:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f310c0a2e20 msgr2=0x7f310c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f310c0a2e20 0x7f310c0a52e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3114004ab0 tx=0x7f3114004a40 comp rx=0 tx=0).stop 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 msgr2=0x7f311c197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c197750 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f310800eb00 tx=0x7f310800ef00 comp rx=0 tx=0).stop 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 shutdown_connections 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f310c0a2e20 0x7f310c0a52e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f311c1069c0 0x7f311c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f311c101350 0x7f311c197750 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 --2- 172.21.15.137:0/1288317538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f311c1009b0 0x7f311c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.265+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 >> 172.21.15.137:0/1288317538 conn(0x7f311c077990 msgr2=0x7f311c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.266+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 shutdown_connections 2024-09-12T12:17:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:17:52.266+0000 7f3122e7b640 1 -- 172.21.15.137:0/1288317538 wait complete. 2024-09-12T12:17:52.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 319711 6.79 MB/sec execute 87 sec latency 45.653 ms 2024-09-12T12:17:53.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:53 smithi171 ceph-mon[35098]: pgmap v143: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 649 KiB/s wr, 90 op/s 2024-09-12T12:17:53.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 322781 6.91 MB/sec execute 88 sec latency 14.521 ms 2024-09-12T12:17:53.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:53 smithi137 ceph-mon[30950]: pgmap v143: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 649 KiB/s wr, 90 op/s 2024-09-12T12:17:53.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:53 smithi189 ceph-mon[35475]: pgmap v143: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 649 KiB/s wr, 90 op/s 2024-09-12T12:17:54.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 324454 6.91 MB/sec execute 89 sec latency 11.677 ms 2024-09-12T12:17:55.642 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:55 smithi189 ceph-mon[35475]: pgmap v144: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 615 KiB/s wr, 84 op/s 2024-09-12T12:17:55.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:55 smithi171 ceph-mon[35098]: pgmap v144: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 615 KiB/s wr, 84 op/s 2024-09-12T12:17:55.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 326814 6.99 MB/sec execute 90 sec latency 9.888 ms 2024-09-12T12:17:55.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:55 smithi137 ceph-mon[30950]: pgmap v144: 97 pgs: 97 active+clean; 85 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 615 KiB/s wr, 84 op/s 2024-09-12T12:17:56.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 329945 7.11 MB/sec execute 91 sec latency 16.461 ms 2024-09-12T12:17:57.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:57 smithi171 ceph-mon[35098]: pgmap v145: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-09-12T12:17:57.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 331544 7.10 MB/sec execute 92 sec latency 21.734 ms 2024-09-12T12:17:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:57 smithi137 ceph-mon[30950]: pgmap v145: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-09-12T12:17:57.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:57 smithi189 ceph-mon[35475]: pgmap v145: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 238 op/s 2024-09-12T12:17:58.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 333981 7.18 MB/sec execute 93 sec latency 8.391 ms 2024-09-12T12:17:59.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:17:59 smithi171 ceph-mon[35098]: pgmap v146: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-09-12T12:17:59.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 337062 7.29 MB/sec execute 94 sec latency 14.277 ms 2024-09-12T12:17:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:17:59 smithi137 ceph-mon[30950]: pgmap v146: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-09-12T12:17:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:17:59 smithi189 ceph-mon[35475]: pgmap v146: 97 pgs: 97 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-09-12T12:18:00.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 338011 7.24 MB/sec execute 95 sec latency 21.727 ms 2024-09-12T12:18:01.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:01 smithi171 ceph-mon[35098]: pgmap v147: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 361 op/s 2024-09-12T12:18:01.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 340742 7.35 MB/sec execute 96 sec latency 40.560 ms 2024-09-12T12:18:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:01 smithi137 ceph-mon[30950]: pgmap v147: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 361 op/s 2024-09-12T12:18:01.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:01 smithi189 ceph-mon[35475]: pgmap v147: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 361 op/s 2024-09-12T12:18:02.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 342379 7.33 MB/sec execute 97 sec latency 14.528 ms 2024-09-12T12:18:03.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:03 smithi171 ceph-mon[35098]: pgmap v148: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:03.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 344602 7.39 MB/sec execute 98 sec latency 12.076 ms 2024-09-12T12:18:03.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:03 smithi137 ceph-mon[30950]: pgmap v148: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:03.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:03 smithi189 ceph-mon[35475]: pgmap v148: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:04.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 347428 7.47 MB/sec execute 99 sec latency 14.757 ms 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 -- 172.21.15.137:0/3699366722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978105a00 msgr2=0x7f0978105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 --2- 172.21.15.137:0/3699366722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978105a00 0x7f0978105de0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0968009900 tx=0x7f096802a410 comp rx=0 tx=0).stop 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 -- 172.21.15.137:0/3699366722 shutdown_connections 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 --2- 172.21.15.137:0/3699366722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0978101a80 0x7f097810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 --2- 172.21.15.137:0/3699366722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 0x7f0978101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 --2- 172.21.15.137:0/3699366722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978105a00 0x7f0978105de0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 -- 172.21.15.137:0/3699366722 >> 172.21.15.137:0/3699366722 conn(0x7f0978077160 msgr2=0x7f09780fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 -- 172.21.15.137:0/3699366722 shutdown_connections 2024-09-12T12:18:05.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 -- 172.21.15.137:0/3699366722 wait complete. 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 Processor -- start 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 -- start start 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 0x7f09781990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978101a80 0x7f0978199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.355+0000 7f097d465640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0978105a00 0x7f09781a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f097d465640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0978110e10 con 0x7f0978101a80 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f097d465640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0978110c90 con 0x7f09781010e0 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f097d465640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0978110f90 con 0x7f0978105a00 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978101a80 0x7f0978199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 0x7f09781990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978101a80 0x7f0978199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38638/0 (socket says 172.21.15.137:38638) 2024-09-12T12:18:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 0x7f09781990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:38680/0 (socket says 172.21.15.137:38680) 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 -- 172.21.15.137:0/1251414681 learned_addr learned my addr 172.21.15.137:0/1251414681 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f09777fe640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0978105a00 0x7f09781a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 -- 172.21.15.137:0/1251414681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0978105a00 msgr2=0x7f09781a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0978105a00 0x7f09781a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 -- 172.21.15.137:0/1251414681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 msgr2=0x7f09781990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 0x7f09781990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 -- 172.21.15.137:0/1251414681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781a46a0 con 0x7f0978101a80 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f0976ffd640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 0x7f09781990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f096ffff640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978101a80 0x7f0978199620 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f096000ea60 tx=0x7f096000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:05.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f0974ff9640 1 -- 172.21.15.137:0/1251414681 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096000cde0 con 0x7f0978101a80 2024-09-12T12:18:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f0974ff9640 1 -- 172.21.15.137:0/1251414681 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09600035f0 con 0x7f0978101a80 2024-09-12T12:18:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781a4870 con 0x7f0978101a80 2024-09-12T12:18:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f0974ff9640 1 -- 172.21.15.137:0/1251414681 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960010670 con 0x7f0978101a80 2024-09-12T12:18:05.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.356+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781a4de0 con 0x7f0978101a80 2024-09-12T12:18:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0974ff9640 1 -- 172.21.15.137:0/1251414681 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f09600037a0 con 0x7f0978101a80 2024-09-12T12:18:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 Processor -- start 2024-09-12T12:18:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 -- start start 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b629d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b638b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b63b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559c0b58e2f0 con 0x559c0b62a2d0 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559c0b536270 con 0x559c0b6299b0 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.359+0000 7f0980206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c0b58a2b0 con 0x559c0b6390c0 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b638b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f096ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b63b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b638b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38642/0 (socket says 172.21.15.137:38642) 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 -- 172.21.15.137:0/361363654 learned_addr learned my addr 172.21.15.137:0/361363654 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f09777fe640 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b629d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0974ff9640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09640a2e20 0x7f09640a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 -- 172.21.15.137:0/361363654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 msgr2=0x559c0b63b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b63b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 -- 172.21.15.137:0/361363654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 msgr2=0x559c0b629d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b629d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 -- 172.21.15.137:0/361363654 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0b535340 con 0x559c0b62a2d0 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f096ffff640 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b63b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f09777fe640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09640a2e20 0x7f09640a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0974ff9640 1 -- 172.21.15.137:0/1251414681 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f09600cf670 con 0x7f0978101a80 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f0976ffd640 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b638b80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f096802a930 tx=0x7f0968005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90e5f13d8d28963 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f096d7fa640 1 -- 172.21.15.137:0/361363654 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968035030 con 0x559c0b62a2d0 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f096d7fa640 1 -- 172.21.15.137:0/361363654 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0968040070 con 0x559c0b62a2d0 2024-09-12T12:18:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.360+0000 7f096d7fa640 1 -- 172.21.15.137:0/361363654 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968031c70 con 0x559c0b62a2d0 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 -- 172.21.15.137:0/361363654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 msgr2=0x559c0b638b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f09777fe640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09640a2e20 0x7f09640a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0970005fd0 tx=0x7f0970004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b638b80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f096802a930 tx=0x7f0968005e80 comp rx=0 tx=0).stop 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 -- 172.21.15.137:0/361363654 shutdown_connections 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b63b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b638b80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 --2- 172.21.15.137:0/361363654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b629d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 -- 172.21.15.137:0/361363654 >> 172.21.15.137:0/361363654 conn(0x559c0b626060 msgr2=0x559c0b626440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 -- 172.21.15.137:0/361363654 shutdown_connections 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 -- 172.21.15.137:0/361363654 wait complete. 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.361+0000 7f0980206740 1 -- start start 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 1 Processor -- start 2024-09-12T12:18:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b636010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b636550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b636a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559c0b63bc30 con 0x559c0b62a2d0 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559c0b517290 con 0x559c0b6299b0 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559c0b63bdb0 con 0x559c0b6390c0 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0980206740 10 client.0 _ll_register_callbacks cb 0x559c0b548140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f096ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b636550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b636010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b636010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:38696/0 (socket says 172.21.15.137:38696) 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 -- 172.21.15.137:0/3552348746 learned_addr learned my addr 172.21.15.137:0/3552348746 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f09777fe640 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b636a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 -- 172.21.15.137:0/3552348746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 msgr2=0x559c0b636a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b636a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 -- 172.21.15.137:0/3552348746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 msgr2=0x559c0b636550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b636550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f0976ffd640 1 -- 172.21.15.137:0/3552348746 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0b2a1210 con 0x559c0b6299b0 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.362+0000 7f096ffff640 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b636550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f0976ffd640 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b636010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f096802a450 tx=0x7f0968031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97d0fd57d438f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f095e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f095e7fc640 1 -- 172.21.15.137:0/3552348746 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096804d070 con 0x559c0b6299b0 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f095e7fc640 1 -- 172.21.15.137:0/3552348746 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f0968005020 con 0x559c0b6299b0 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0968040070 con 0x559c0b6299b0 2024-09-12T12:18:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f095e7fc640 1 -- 172.21.15.137:0/3552348746 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0968058050 con 0x559c0b6299b0 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f0980206740 10 client.25433 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c0b36d620 con 0x559c0b6299b0 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.363+0000 7f095e7fc640 1 -- 172.21.15.137:0/3552348746 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0968040070 con 0x559c0b6299b0 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f095e7fc640 1 -- 172.21.15.137:0/3552348746 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f09680035f0 con 0x559c0b6299b0 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f0980206740 10 client.25433 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f0980206740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f0980206740 20 client.25433 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f0980206740 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b648eb0 0x559c0b64b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f0980206740 4 client.25433 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c0b59ba20 con 0x559c0b648eb0 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f096ffff640 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b648eb0 0x559c0b64b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f096ffff640 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b648eb0 0x559c0b64b2a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51537ce7bb0e2d15 server_cookie=9eaa56d207ad3ea5 in_seq=0 out_seq=0 2024-09-12T12:18:05.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.364+0000 7f095e7fc640 10 client.25433 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f095e7fc640 1 -- 172.21.15.137:0/3552348746 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c0b59ba20 con 0x559c0b648eb0 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f095e7fc640 10 client.25433 handle_command_reply: tid=0 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 client.25433 shutdown 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 msgr2=0x559c0b636010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b636010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f096802a450 tx=0x7f0968031770 comp rx=0 tx=0).stop 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 shutdown_connections 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6390c0 0x559c0b636a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b648eb0 0x559c0b64b2a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b636550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 --2- 172.21.15.137:0/3552348746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6299b0 0x559c0b636010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f095e7fc640 0 client.25433 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 >> 172.21.15.137:0/3552348746 conn(0x559c0b626060 msgr2=0x559c0b62bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 shutdown_connections 2024-09-12T12:18:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 1 -- 172.21.15.137:0/3552348746 wait complete. 2024-09-12T12:18:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.365+0000 7f0980206740 20 client.25433 trim_cache size 0 max 16384 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.379+0000 7f0980206740 1 Processor -- start 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.379+0000 7f0980206740 1 -- start start 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0980206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b637f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0980206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b650900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0980206740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b650e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0980206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09781add60 con 0x559c0b62a2d0 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0980206740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f09781ab630 con 0x559c0b6390c0 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0980206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09781a94a0 con 0x559c0b6299b0 2024-09-12T12:18:05.385 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b650900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b650900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38652/0 (socket says 172.21.15.137:38652) 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 -- 172.21.15.137:0/3132735643 learned_addr learned my addr 172.21.15.137:0/3132735643 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0976ffd640 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b637f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f09777fe640 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b650e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 -- 172.21.15.137:0/3132735643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 msgr2=0x559c0b637f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b637f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 -- 172.21.15.137:0/3132735643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 msgr2=0x559c0b650e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b650e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 -- 172.21.15.137:0/3132735643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0b63b850 con 0x559c0b62a2d0 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f0976ffd640 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b637f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f09777fe640 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b650e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:18:05.386 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.380+0000 7f096ffff640 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b650900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0960003f70 tx=0x7f09600095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c1c7cac143642e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f096cff9640 1 -- 172.21.15.137:0/3132735643 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096009b540 con 0x559c0b62a2d0 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f096cff9640 1 -- 172.21.15.137:0/3132735643 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096009c2e0 con 0x559c0b62a2d0 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f096cff9640 1 -- 172.21.15.137:0/3132735643 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09600cd090 con 0x559c0b62a2d0 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 -- 172.21.15.137:0/3132735643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 msgr2=0x559c0b650900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b650900 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0960003f70 tx=0x7f09600095b0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 -- 172.21.15.137:0/3132735643 shutdown_connections 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b650e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b650900 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 --2- 172.21.15.137:0/3132735643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b637f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 -- 172.21.15.137:0/3132735643 >> 172.21.15.137:0/3132735643 conn(0x559c0b626060 msgr2=0x559c0b637870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 -- 172.21.15.137:0/3132735643 shutdown_connections 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.381+0000 7f0980206740 1 -- 172.21.15.137:0/3132735643 wait complete. 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 -- start start 2024-09-12T12:18:05.387 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 Processor -- start 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b641610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b632c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b633150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f09781a9af0 con 0x559c0b62a2d0 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f09781aaf60 con 0x559c0b6390c0 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09781aa1c0 con 0x559c0b6299b0 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0980206740 10 client.0 _ll_register_callbacks cb 0x7f09600d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f096ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b632c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b641610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0976ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b641610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54326/0 (socket says 172.21.15.137:54326) 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f0976ffd640 1 -- 172.21.15.137:0/1222392890 learned_addr learned my addr 172.21.15.137:0/1222392890 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f09777fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b633150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f096ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b632c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38668/0 (socket says 172.21.15.137:38668) 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.382+0000 7f096ffff640 1 -- 172.21.15.137:0/1222392890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 msgr2=0x559c0b641610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f096ffff640 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b641610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f096ffff640 1 -- 172.21.15.137:0/1222392890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 msgr2=0x559c0b633150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f096ffff640 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b633150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f096ffff640 1 -- 172.21.15.137:0/1222392890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0b6337f0 con 0x559c0b62a2d0 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0976ffd640 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b641610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f096ffff640 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b632c10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x559c0b637230 tx=0x7f09600a0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95183a9c7f016b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f095e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f095e7fc640 1 -- 172.21.15.137:0/1222392890 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096009bbe0 con 0x559c0b62a2d0 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f095e7fc640 1 -- 172.21.15.137:0/1222392890 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f09600cd090 con 0x559c0b62a2d0 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096009c2e0 con 0x559c0b62a2d0 2024-09-12T12:18:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f095e7fc640 1 -- 172.21.15.137:0/1222392890 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0960098250 con 0x559c0b62a2d0 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 10 client.16002 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c0b6b3750 con 0x559c0b62a2d0 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f095e7fc640 1 -- 172.21.15.137:0/1222392890 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096009c2e0 con 0x559c0b62a2d0 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f095e7fc640 1 -- 172.21.15.137:0/1222392890 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f0960097930 con 0x559c0b62a2d0 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 10 client.16002 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 20 client.16002 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b6b3db0 0x559c0b6b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 4 client.16002 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c0b595d70 con 0x559c0b6b3db0 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.383+0000 7f0976ffd640 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b6b3db0 0x559c0b6b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:05.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.384+0000 7f0976ffd640 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b6b3db0 0x559c0b6b61a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=213a9d131952c5e0 server_cookie=3b98d24fedcc7b8e in_seq=0 out_seq=0 2024-09-12T12:18:05.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.384+0000 7f095e7fc640 10 client.16002 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f095e7fc640 1 -- 172.21.15.137:0/1222392890 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c0b595d70 con 0x559c0b6b3db0 2024-09-12T12:18:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f095e7fc640 10 client.16002 handle_command_reply: tid=0 2024-09-12T12:18:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 client.16002 shutdown 2024-09-12T12:18:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 msgr2=0x559c0b632c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b632c10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x559c0b637230 tx=0x7f09600a0000 comp rx=0 tx=0).stop 2024-09-12T12:18:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 shutdown_connections 2024-09-12T12:18:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559c0b6390c0 0x559c0b633150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559c0b6b3db0 0x559c0b6b61a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559c0b62a2d0 0x559c0b632c10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 --2- 172.21.15.137:0/1222392890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559c0b6299b0 0x559c0b641610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 >> 172.21.15.137:0/1222392890 conn(0x559c0b626060 msgr2=0x559c0b62bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:05.460 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.458+0000 7f095e7fc640 0 client.16002 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.459+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 shutdown_connections 2024-09-12T12:18:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.459+0000 7f0980206740 1 -- 172.21.15.137:0/1222392890 wait complete. 2024-09-12T12:18:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.459+0000 7f0980206740 20 client.16002 trim_cache size 0 max 16384 2024-09-12T12:18:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:18:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:18:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:18:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:18:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09640a2e20 msgr2=0x7f09640a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09640a2e20 0x7f09640a52e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0970005fd0 tx=0x7f0970004a40 comp rx=0 tx=0).stop 2024-09-12T12:18:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978101a80 msgr2=0x7f0978199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978101a80 0x7f0978199620 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f096000ea60 tx=0x7f096000ef20 comp rx=0 tx=0).stop 2024-09-12T12:18:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 shutdown_connections 2024-09-12T12:18:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09640a2e20 0x7f09640a52e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0978105a00 0x7f09781a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0978101a80 0x7f0978199620 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 --2- 172.21.15.137:0/1251414681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09781010e0 0x7f09781990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 >> 172.21.15.137:0/1251414681 conn(0x7f0978077160 msgr2=0x7f0978103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 shutdown_connections 2024-09-12T12:18:05.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:05.460+0000 7f097d465640 1 -- 172.21.15.137:0/1251414681 wait complete. 2024-09-12T12:18:05.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:05 smithi171 ceph-mon[35098]: pgmap v149: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:05.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 348995 7.49 MB/sec execute 100 sec latency 24.958 ms 2024-09-12T12:18:05.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:05 smithi137 ceph-mon[30950]: pgmap v149: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:05.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:05 smithi189 ceph-mon[35475]: pgmap v149: 97 pgs: 97 active+clean; 95 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:06.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 351576 7.55 MB/sec execute 101 sec latency 11.320 ms 2024-09-12T12:18:07.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:07 smithi171 ceph-mon[35098]: pgmap v150: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2024-09-12T12:18:07.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 354353 7.61 MB/sec execute 102 sec latency 14.127 ms 2024-09-12T12:18:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:07 smithi137 ceph-mon[30950]: pgmap v150: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2024-09-12T12:18:07.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:07 smithi189 ceph-mon[35475]: pgmap v150: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 407 op/s 2024-09-12T12:18:08.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 356169 7.64 MB/sec execute 103 sec latency 14.583 ms 2024-09-12T12:18:09.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:09 smithi171 ceph-mon[35098]: pgmap v151: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 253 op/s 2024-09-12T12:18:09.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 358672 7.70 MB/sec execute 104 sec latency 9.979 ms 2024-09-12T12:18:09.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:09 smithi137 ceph-mon[30950]: pgmap v151: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 253 op/s 2024-09-12T12:18:09.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:09 smithi189 ceph-mon[35475]: pgmap v151: 97 pgs: 97 active+clean; 102 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 253 op/s 2024-09-12T12:18:10.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 361606 7.77 MB/sec execute 105 sec latency 28.272 ms 2024-09-12T12:18:11.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:11 smithi171 ceph-mon[35098]: pgmap v152: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 401 op/s 2024-09-12T12:18:11.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 363263 7.79 MB/sec execute 106 sec latency 13.562 ms 2024-09-12T12:18:11.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:11 smithi137 ceph-mon[30950]: pgmap v152: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 401 op/s 2024-09-12T12:18:11.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:11 smithi189 ceph-mon[35475]: pgmap v152: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 401 op/s 2024-09-12T12:18:12.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 365707 7.84 MB/sec execute 107 sec latency 28.916 ms 2024-09-12T12:18:13.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:13 smithi171 ceph-mon[35098]: pgmap v153: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-09-12T12:18:13.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 368106 7.87 MB/sec execute 108 sec latency 14.883 ms 2024-09-12T12:18:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:13 smithi137 ceph-mon[30950]: pgmap v153: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-09-12T12:18:13.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:13 smithi189 ceph-mon[35475]: pgmap v153: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-09-12T12:18:14.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 370142 7.90 MB/sec execute 109 sec latency 11.918 ms 2024-09-12T12:18:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:15 smithi171 ceph-mon[35098]: pgmap v154: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-09-12T12:18:15.728 INFO:tasks.workunit.client.0.smithi137.stdout: 1 372816 7.98 MB/sec execute 110 sec latency 10.740 ms 2024-09-12T12:18:15.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:15 smithi137 ceph-mon[30950]: pgmap v154: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-09-12T12:18:15.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:15 smithi189 ceph-mon[35475]: pgmap v154: 97 pgs: 97 active+clean; 99 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 275 op/s 2024-09-12T12:18:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:16 smithi171 ceph-mon[35098]: pgmap v155: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 384 op/s 2024-09-12T12:18:16.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 375441 8.02 MB/sec execute 111 sec latency 23.779 ms 2024-09-12T12:18:16.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:16 smithi137 ceph-mon[30950]: pgmap v155: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 384 op/s 2024-09-12T12:18:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:16 smithi189 ceph-mon[35475]: pgmap v155: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 384 op/s 2024-09-12T12:18:17.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 377427 8.05 MB/sec execute 112 sec latency 13.181 ms 2024-09-12T12:18:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 -- 172.21.15.137:0/2790761584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240102d20 msgr2=0x7f8240103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 --2- 172.21.15.137:0/2790761584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240102d20 0x7f8240103100 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f823c009900 tx=0x7f823c02a480 comp rx=0 tx=0).stop 2024-09-12T12:18:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 -- 172.21.15.137:0/2790761584 shutdown_connections 2024-09-12T12:18:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 --2- 172.21.15.137:0/2790761584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824010c0f0 0x7f824010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 --2- 172.21.15.137:0/2790761584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8240103640 0x7f824010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 --2- 172.21.15.137:0/2790761584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240102d20 0x7f8240103100 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 -- 172.21.15.137:0/2790761584 >> 172.21.15.137:0/2790761584 conn(0x7f8240077990 msgr2=0x7f82400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 -- 172.21.15.137:0/2790761584 shutdown_connections 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.558+0000 7f824872c640 1 -- 172.21.15.137:0/2790761584 wait complete. 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 Processor -- start 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 -- start start 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8240102d20 0x7f8240198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240103640 0x7f8240198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824010c0f0 0x7f824019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.561 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8240111ec0 con 0x7f8240103640 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8240111d40 con 0x7f8240102d20 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f824872c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8240112040 con 0x7f824010c0f0 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f8245cc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240103640 0x7f8240198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f8245cc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240103640 0x7f8240198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60558/0 (socket says 172.21.15.137:60558) 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f8245cc7640 1 -- 172.21.15.137:0/3775611555 learned_addr learned my addr 172.21.15.137:0/3775611555 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f8246cc9640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824010c0f0 0x7f824019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.559+0000 7f82464c8640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8240102d20 0x7f8240198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f8245cc7640 1 -- 172.21.15.137:0/3775611555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824010c0f0 msgr2=0x7f824019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f8245cc7640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824010c0f0 0x7f824019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f8245cc7640 1 -- 172.21.15.137:0/3775611555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8240102d20 msgr2=0x7f8240198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f8245cc7640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8240102d20 0x7f8240198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f8245cc7640 1 -- 172.21.15.137:0/3775611555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824019d2f0 con 0x7f8240103640 2024-09-12T12:18:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f8245cc7640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240103640 0x7f8240198840 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f823000e9f0 tx=0x7f823000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f82377fe640 1 -- 172.21.15.137:0/3775611555 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823000cd80 con 0x7f8240103640 2024-09-12T12:18:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f82377fe640 1 -- 172.21.15.137:0/3775611555 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8230010910 con 0x7f8240103640 2024-09-12T12:18:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824019d5e0 con 0x7f8240103640 2024-09-12T12:18:18.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f82377fe640 1 -- 172.21.15.137:0/3775611555 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230018990 con 0x7f8240103640 2024-09-12T12:18:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.560+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82401a4e70 con 0x7f8240103640 2024-09-12T12:18:18.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.562+0000 7f82377fe640 1 -- 172.21.15.137:0/3775611555 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8230010ac0 con 0x7f8240103640 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 Processor -- start 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f82377fe640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f822c0a2e20 0x7f822c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f8246cc9640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f822c0a2e20 0x7f822c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f82377fe640 1 -- 172.21.15.137:0/3775611555 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f82300cf720 con 0x7f8240103640 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 -- start start 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642cd22a2f0 con 0x5642cd2d0cb0 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5642cd1d2270 con 0x5642cd2c5af0 2024-09-12T12:18:18.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.563+0000 7f824b4c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642cd2262b0 con 0x5642cd2c6410 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:58692/0 (socket says 172.21.15.137:58692) 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 -- 172.21.15.137:0/2677005617 learned_addr learned my addr 172.21.15.137:0/2677005617 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f8246cc9640 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f8245cc7640 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f8246cc9640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f822c0a2e20 0x7f822c0a52e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8238005fd0 tx=0x7f8238009600 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 -- 172.21.15.137:0/2677005617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 msgr2=0x5642cd2d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 -- 172.21.15.137:0/2677005617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2d0cb0 msgr2=0x5642cd2d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 -- 172.21.15.137:0/2677005617 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cd1d1340 con 0x5642cd2c5af0 2024-09-12T12:18:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f82464c8640 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2c5ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f823c009900 tx=0x7f823c005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e70f72a1b239942 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f8234ff9640 1 -- 172.21.15.137:0/2677005617 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c03f070 con 0x5642cd2c5af0 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f8234ff9640 1 -- 172.21.15.137:0/2677005617 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c004030 con 0x5642cd2c5af0 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.564+0000 7f8234ff9640 1 -- 172.21.15.137:0/2677005617 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c035070 con 0x5642cd2c5af0 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 -- 172.21.15.137:0/2677005617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 msgr2=0x5642cd2c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2c5ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f823c009900 tx=0x7f823c005dd0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 -- 172.21.15.137:0/2677005617 shutdown_connections 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2677005617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2c5ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 -- 172.21.15.137:0/2677005617 >> 172.21.15.137:0/2677005617 conn(0x5642cd2c2060 msgr2=0x5642cd2c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 -- 172.21.15.137:0/2677005617 shutdown_connections 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.565+0000 7f824b4c0740 1 -- 172.21.15.137:0/2677005617 wait complete. 2024-09-12T12:18:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 -- start start 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 Processor -- start 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2d0cb0 0x5642cd2ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5642cd2d3880 con 0x5642cd2c6410 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5642cd1b3290 con 0x5642cd2c5af0 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5642cd2d3a00 con 0x5642cd2d0cb0 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f824b4c0740 10 client.0 _ll_register_callbacks cb 0x5642cd1e4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f82464c8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8245cc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8246cc9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2d0cb0 0x5642cd2ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f82464c8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:58706/0 (socket says 172.21.15.137:58706) 2024-09-12T12:18:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f82464c8640 1 -- 172.21.15.137:0/3139538339 learned_addr learned my addr 172.21.15.137:0/3139538339 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8245cc7640 1 -- 172.21.15.137:0/3139538339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2d0cb0 msgr2=0x5642cd2ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8245cc7640 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2d0cb0 0x5642cd2ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8245cc7640 1 -- 172.21.15.137:0/3139538339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 msgr2=0x5642cd2cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8245cc7640 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8245cc7640 1 -- 172.21.15.137:0/3139538339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ccf3d210 con 0x5642cd2c6410 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8245cc7640 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2ce150 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f82300ccec0 tx=0x7f823009c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee8725941b5f043 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.566+0000 7f8229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f8229ffb640 1 -- 172.21.15.137:0/3139538339 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230018b30 con 0x5642cd2c6410 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f8229ffb640 1 -- 172.21.15.137:0/3139538339 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f8230018cd0 con 0x5642cd2c6410 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823c004030 con 0x5642cd2c6410 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f8229ffb640 1 -- 172.21.15.137:0/3139538339 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f82300a4b50 con 0x5642cd2c6410 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 10 client.16014 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642cd009620 con 0x5642cd2c6410 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f8229ffb640 1 -- 172.21.15.137:0/3139538339 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c004030 con 0x5642cd2c6410 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f8229ffb640 1 -- 172.21.15.137:0/3139538339 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f8230097b00 con 0x5642cd2c6410 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 10 client.16014 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 20 client.16014 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd2e0cc0 0x5642cd2e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 4 client.16014 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cd237a20 con 0x5642cd2e0cc0 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.567+0000 7f82464c8640 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd2e0cc0 0x5642cd2e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.568+0000 7f8229ffb640 10 client.16014 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.568+0000 7f82464c8640 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd2e0cc0 0x5642cd2e30f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f1416e843e1bc server_cookie=96c7d55857f25b4a in_seq=0 out_seq=0 2024-09-12T12:18:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f8229ffb640 1 -- 172.21.15.137:0/3139538339 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642cd237a20 con 0x5642cd2e0cc0 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f8229ffb640 10 client.16014 handle_command_reply: tid=0 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 client.16014 shutdown 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 msgr2=0x5642cd2ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2ce150 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f82300ccec0 tx=0x7f823009c4d0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 shutdown_connections 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2d0cb0 0x5642cd2ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd2e0cc0 0x5642cd2e30f0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2ce150 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 --2- 172.21.15.137:0/3139538339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2c5af0 0x5642cd2cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f8229ffb640 0 client.16014 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 >> 172.21.15.137:0/3139538339 conn(0x5642cd2c2060 msgr2=0x5642cd2c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 shutdown_connections 2024-09-12T12:18:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 1 -- 172.21.15.137:0/3139538339 wait complete. 2024-09-12T12:18:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.569+0000 7f824b4c0740 20 client.16014 trim_cache size 0 max 16384 2024-09-12T12:18:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 Processor -- start 2024-09-12T12:18:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 -- start start 2024-09-12T12:18:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c5af0 0x5642cd2ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2e8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82401afe90 con 0x5642cd2c5af0 2024-09-12T12:18:18.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f82401ad760 con 0x5642cd2d0cb0 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f824b4c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82401ab5b0 con 0x5642cd2c6410 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c5af0 0x5642cd2ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f8245cc7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c5af0 0x5642cd2ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60580/0 (socket says 172.21.15.137:60580) 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f8245cc7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2e8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54906/0 (socket says 172.21.15.137:54906) 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 -- 172.21.15.137:0/2493943983 learned_addr learned my addr 172.21.15.137:0/2493943983 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f8246cc9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 -- 172.21.15.137:0/2493943983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 msgr2=0x5642cd2e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 --2- 172.21.15.137:0/2493943983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2e8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 -- 172.21.15.137:0/2493943983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 msgr2=0x5642cd2e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 --2- 172.21.15.137:0/2493943983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 -- 172.21.15.137:0/2493943983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cd2c2cf0 con 0x5642cd2c5af0 2024-09-12T12:18:18.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f82464c8640 1 --2- 172.21.15.137:0/2493943983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c5af0 0x5642cd2ce4c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f823c009900 tx=0x7f823c02ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb4f7f6c53581c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.584+0000 7f822bfff640 1 -- 172.21.15.137:0/2493943983 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c035050 con 0x5642cd2c5af0 2024-09-12T12:18:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f822bfff640 1 -- 172.21.15.137:0/2493943983 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c031be0 con 0x5642cd2c5af0 2024-09-12T12:18:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f822bfff640 1 -- 172.21.15.137:0/2493943983 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c03f040 con 0x5642cd2c5af0 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 -- 172.21.15.137:0/2493943983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c5af0 msgr2=0x5642cd2ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2493943983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c5af0 0x5642cd2ce4c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f823c009900 tx=0x7f823c02ab00 comp rx=0 tx=0).stop 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 -- 172.21.15.137:0/2493943983 shutdown_connections 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2493943983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2493943983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c6410 0x5642cd2e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2493943983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c5af0 0x5642cd2ce4c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 -- 172.21.15.137:0/2493943983 >> 172.21.15.137:0/2493943983 conn(0x5642cd2c2060 msgr2=0x5642cd2cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:18.589 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 -- 172.21.15.137:0/2493943983 shutdown_connections 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.585+0000 7f824b4c0740 1 -- 172.21.15.137:0/2493943983 wait complete. 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 -- start start 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 Processor -- start 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c5af0 0x5642cd2d4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f82401abc20 con 0x5642cd2c6410 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f82401ad090 con 0x5642cd2d0cb0 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82401ac2f0 con 0x5642cd2c5af0 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f824b4c0740 10 client.0 _ll_register_callbacks cb 0x7f823000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2d4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60592/0 (socket says 172.21.15.137:60592) 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 -- 172.21.15.137:0/2743297704 learned_addr learned my addr 172.21.15.137:0/2743297704 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f82464c8640 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c5af0 0x5642cd2d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8246cc9640 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 -- 172.21.15.137:0/2743297704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c5af0 msgr2=0x5642cd2d4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c5af0 0x5642cd2d4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 -- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 msgr2=0x5642cd2d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 -- 172.21.15.137:0/2743297704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cd34a930 con 0x5642cd2c6410 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8245cc7640 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2d4e50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f823009c540 tx=0x7f8230097c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c9cec279271e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.586+0000 7f8229ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f8229ffb640 1 -- 172.21.15.137:0/2743297704 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823009b2f0 con 0x5642cd2c6410 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f8229ffb640 1 -- 172.21.15.137:0/2743297704 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f8230010430 con 0x5642cd2c6410 2024-09-12T12:18:18.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f823c031be0 con 0x5642cd2c6410 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f8229ffb640 1 -- 172.21.15.137:0/2743297704 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8230010630 con 0x5642cd2c6410 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 10 client.16020 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642cd34bb60 con 0x5642cd2c6410 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f8229ffb640 1 -- 172.21.15.137:0/2743297704 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f823c031be0 con 0x5642cd2c6410 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f8229ffb640 1 -- 172.21.15.137:0/2743297704 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f8230018b30 con 0x5642cd2c6410 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 10 client.16020 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 10 client.16020 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 20 client.16020 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd34c310 0x5642cd34e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 4 client.16020 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642cd231d70 con 0x5642cd34c310 2024-09-12T12:18:18.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.587+0000 7f82464c8640 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd34c310 0x5642cd34e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.588+0000 7f8229ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:18.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.588+0000 7f82464c8640 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd34c310 0x5642cd34e730 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79643064dbbb70f5 server_cookie=c96fe84a7e3aa426 in_seq=0 out_seq=0 2024-09-12T12:18:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f8229ffb640 1 -- 172.21.15.137:0/2743297704 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642cd231d70 con 0x5642cd34c310 2024-09-12T12:18:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f8229ffb640 10 client.16020 handle_command_reply: tid=0 2024-09-12T12:18:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 client.16020 shutdown 2024-09-12T12:18:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 msgr2=0x5642cd2d4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2d4e50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f823009c540 tx=0x7f8230097c10 comp rx=0 tx=0).stop 2024-09-12T12:18:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 shutdown_connections 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5642cd2d0cb0 0x5642cd2d5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5642cd34c310 0x5642cd34e730 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5642cd2c6410 0x5642cd2d4e50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 --2- 172.21.15.137:0/2743297704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5642cd2c5af0 0x5642cd2d4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 >> 172.21.15.137:0/2743297704 conn(0x5642cd2c2060 msgr2=0x5642cd2c7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f8229ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 shutdown_connections 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.652+0000 7f824b4c0740 1 -- 172.21.15.137:0/2743297704 wait complete. 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824b4c0740 20 client.16020 trim_cache size 0 max 16384 2024-09-12T12:18:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:18:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:18:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:18:18.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f822c0a2e20 msgr2=0x7f822c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f822c0a2e20 0x7f822c0a52e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8238005fd0 tx=0x7f8238009600 comp rx=0 tx=0).stop 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240103640 msgr2=0x7f8240198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240103640 0x7f8240198840 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f823000e9f0 tx=0x7f823000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 shutdown_connections 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f822c0a2e20 0x7f822c0a52e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f824010c0f0 0x7f824019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8240103640 0x7f8240198840 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 --2- 172.21.15.137:0/3775611555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8240102d20 0x7f8240198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:18.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.653+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 >> 172.21.15.137:0/3775611555 conn(0x7f8240077990 msgr2=0x7f82400fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.654+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 shutdown_connections 2024-09-12T12:18:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:18.654+0000 7f824872c640 1 -- 172.21.15.137:0/3775611555 wait complete. 2024-09-12T12:18:18.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 380004 8.10 MB/sec execute 113 sec latency 62.302 ms 2024-09-12T12:18:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:18 smithi137 ceph-mon[30950]: pgmap v156: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-12T12:18:18.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:18 smithi189 ceph-mon[35475]: pgmap v156: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-12T12:18:18.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:18 smithi171 ceph-mon[35098]: pgmap v156: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 257 op/s 2024-09-12T12:18:19.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 382757 8.15 MB/sec execute 114 sec latency 17.962 ms 2024-09-12T12:18:20.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 384560 8.17 MB/sec execute 115 sec latency 13.587 ms 2024-09-12T12:18:20.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:20 smithi137 ceph-mon[30950]: pgmap v157: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 427 op/s 2024-09-12T12:18:20.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:20 smithi189 ceph-mon[35475]: pgmap v157: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 427 op/s 2024-09-12T12:18:20.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:20 smithi171 ceph-mon[35098]: pgmap v157: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 427 op/s 2024-09-12T12:18:21.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 387045 8.23 MB/sec execute 116 sec latency 35.701 ms 2024-09-12T12:18:21.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:21 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:18:21.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:21 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:18:21.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:21 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:18:22.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 389844 8.27 MB/sec execute 117 sec latency 35.705 ms 2024-09-12T12:18:22.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:22 smithi137 ceph-mon[30950]: pgmap v158: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:22.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:18:22.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:18:22.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:18:22.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:22 smithi189 ceph-mon[35475]: pgmap v158: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:18:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:18:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:18:22.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:22 smithi171 ceph-mon[35098]: pgmap v158: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:18:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:18:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:18:23.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 391715 8.29 MB/sec execute 118 sec latency 10.426 ms 2024-09-12T12:18:24.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 394158 8.34 MB/sec execute 119 sec latency 10.069 ms 2024-09-12T12:18:24.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:24 smithi137 ceph-mon[30950]: pgmap v159: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:24.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:24 smithi189 ceph-mon[35475]: pgmap v159: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:24.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:24 smithi171 ceph-mon[35098]: pgmap v159: 97 pgs: 97 active+clean; 106 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 279 op/s 2024-09-12T12:18:25.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 396645 8.36 MB/sec execute 120 sec latency 12.525 ms 2024-09-12T12:18:26.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 398560 8.38 MB/sec execute 121 sec latency 10.779 ms 2024-09-12T12:18:26.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:26 smithi137 ceph-mon[30950]: pgmap v160: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-09-12T12:18:26.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:26 smithi189 ceph-mon[35475]: pgmap v160: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-09-12T12:18:26.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:26 smithi171 ceph-mon[35098]: pgmap v160: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-09-12T12:18:27.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 401154 8.45 MB/sec execute 122 sec latency 45.052 ms 2024-09-12T12:18:28.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 403261 8.44 MB/sec execute 123 sec latency 17.476 ms 2024-09-12T12:18:28.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:28 smithi137 ceph-mon[30950]: pgmap v161: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-12T12:18:28.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:28 smithi189 ceph-mon[35475]: pgmap v161: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-12T12:18:28.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:28 smithi171 ceph-mon[35098]: pgmap v161: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-12T12:18:29.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 405140 8.47 MB/sec execute 124 sec latency 13.643 ms 2024-09-12T12:18:30.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 408160 8.55 MB/sec execute 125 sec latency 18.807 ms 2024-09-12T12:18:30.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:30 smithi137 ceph-mon[30950]: pgmap v162: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 439 op/s 2024-09-12T12:18:30.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:30 smithi189 ceph-mon[35475]: pgmap v162: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 439 op/s 2024-09-12T12:18:30.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:30 smithi171 ceph-mon[35098]: pgmap v162: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 439 op/s 2024-09-12T12:18:31.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 410040 8.53 MB/sec execute 126 sec latency 30.265 ms 2024-09-12T12:18:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 -- 172.21.15.137:0/1846558817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c01035c0 msgr2=0x7f03c010baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 --2- 172.21.15.137:0/1846558817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c01035c0 0x7f03c010baa0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009900 tx=0x7f03b002a480 comp rx=0 tx=0).stop 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 -- 172.21.15.137:0/1846558817 shutdown_connections 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 --2- 172.21.15.137:0/1846558817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c010c0c0 0x7f03c010e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 --2- 172.21.15.137:0/1846558817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c01035c0 0x7f03c010baa0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 --2- 172.21.15.137:0/1846558817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03c0102ca0 0x7f03c0103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 -- 172.21.15.137:0/1846558817 >> 172.21.15.137:0/1846558817 conn(0x7f03c00fc6f0 msgr2=0x7f03c00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 -- 172.21.15.137:0/1846558817 shutdown_connections 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.741+0000 7f03c8227640 1 -- 172.21.15.137:0/1846558817 wait complete. 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 Processor -- start 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 -- start start 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c0102ca0 0x7f03c0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03c01035c0 0x7f03c0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c010c0c0 0x7f03c019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0111ee0 con 0x7f03c0102ca0 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0111d60 con 0x7f03c010c0c0 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c8227640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0112060 con 0x7f03c01035c0 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c0102ca0 0x7f03c0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c0102ca0 0x7f03c0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58030/0 (socket says 172.21.15.137:58030) 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 -- 172.21.15.137:0/1153809822 learned_addr learned my addr 172.21.15.137:0/1153809822 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c57c2640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03c01035c0 0x7f03c0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c67c4640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c010c0c0 0x7f03c019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 -- 172.21.15.137:0/1153809822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03c01035c0 msgr2=0x7f03c0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03c01035c0 0x7f03c0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 -- 172.21.15.137:0/1153809822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c010c0c0 msgr2=0x7f03c019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c010c0c0 0x7f03c019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.742+0000 7f03c5fc3640 1 -- 172.21.15.137:0/1153809822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c019d320 con 0x7f03c0102ca0 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.743+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c0102ca0 0x7f03c0198330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00e9f0 tx=0x7f03bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.743+0000 7f03b6ffd640 1 -- 172.21.15.137:0/1153809822 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc00cd80 con 0x7f03c0102ca0 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.743+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c019d610 con 0x7f03c0102ca0 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.743+0000 7f03b6ffd640 1 -- 172.21.15.137:0/1153809822 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03bc005500 con 0x7f03c0102ca0 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.743+0000 7f03b6ffd640 1 -- 172.21.15.137:0/1153809822 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc010690 con 0x7f03c0102ca0 2024-09-12T12:18:31.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.743+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c01a4ea0 con 0x7f03c0102ca0 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.745+0000 7f03b6ffd640 1 -- 172.21.15.137:0/1153809822 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f03bc0046c0 con 0x7f03c0102ca0 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.745+0000 7f03cafc6740 1 Processor -- start 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03cafc6740 1 -- start start 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03b6ffd640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03ac0a2da0 0x7f03ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e6eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e79770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e7c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03cafc6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5580c8dd32f0 con 0x5580c8e6eaf0 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03cafc6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5580c8d7b270 con 0x5580c8e6f410 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03cafc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580c8dcf2b0 con 0x5580c8e79cb0 2024-09-12T12:18:31.752 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e6eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c67c4640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03ac0a2da0 0x7f03ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e6eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58042/0 (socket says 172.21.15.137:58042) 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 -- 172.21.15.137:0/2412951870 learned_addr learned my addr 172.21.15.137:0/2412951870 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03b6ffd640 1 -- 172.21.15.137:0/1153809822 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f03bc0cf880 con 0x7f03c0102ca0 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c57c2640 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e79770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c67c4640 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e7c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 -- 172.21.15.137:0/2412951870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 msgr2=0x5580c8e7c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e7c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 -- 172.21.15.137:0/2412951870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 msgr2=0x5580c8e79770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e79770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 -- 172.21.15.137:0/2412951870 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c8d7a340 con 0x5580c8e6eaf0 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c57c2640 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e6eed0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f03bc098760 tx=0x7f03bc097a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a092ad89f9363824 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03c67c4640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03ac0a2da0 0x7f03ac0a5260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005fd0 tx=0x7f03b8009e90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.746+0000 7f03abfff640 1 -- 172.21.15.137:0/2412951870 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc09b6f0 con 0x5580c8e6eaf0 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03abfff640 1 -- 172.21.15.137:0/2412951870 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03bc09c490 con 0x5580c8e6eaf0 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03abfff640 1 -- 172.21.15.137:0/2412951870 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc06c0a0 con 0x5580c8e6eaf0 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03cafc6740 1 -- 172.21.15.137:0/2412951870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 msgr2=0x5580c8e6eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03cafc6740 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e6eed0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f03bc098760 tx=0x7f03bc097a10 comp rx=0 tx=0).stop 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03cafc6740 1 -- 172.21.15.137:0/2412951870 shutdown_connections 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03cafc6740 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e7c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03cafc6740 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e79770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03cafc6740 1 --2- 172.21.15.137:0/2412951870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e6eed0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.747+0000 7f03cafc6740 1 -- 172.21.15.137:0/2412951870 >> 172.21.15.137:0/2412951870 conn(0x5580c8e6b060 msgr2=0x5580c8e6b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 -- 172.21.15.137:0/2412951870 shutdown_connections 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 -- 172.21.15.137:0/2412951870 wait complete. 2024-09-12T12:18:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 -- start start 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 Processor -- start 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e76bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e77100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e77640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5580c8e7c880 con 0x5580c8e6eaf0 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5580c8d5c290 con 0x5580c8e6f410 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.748+0000 7f03cafc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5580c8e7ca00 con 0x5580c8e79cb0 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03cafc6740 10 client.0 _ll_register_callbacks cb 0x5580c8d8d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e76bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c57c2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e77100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e76bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58052/0 (socket says 172.21.15.137:58052) 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c67c4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e77640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c57c2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e77100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:46256/0 (socket says 172.21.15.137:46256) 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 -- 172.21.15.137:0/3471202682 learned_addr learned my addr 172.21.15.137:0/3471202682 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 -- 172.21.15.137:0/3471202682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 msgr2=0x5580c8e77640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e77640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 -- 172.21.15.137:0/3471202682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 msgr2=0x5580c8e77100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e77100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 -- 172.21.15.137:0/3471202682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c8ae6210 con 0x5580c8e6eaf0 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e76bc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0cd020 tx=0x7f03bc0cd470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbc4d785417788 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03a97fa640 1 -- 172.21.15.137:0/3471202682 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc09bd90 con 0x5580c8e6eaf0 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03a97fa640 1 -- 172.21.15.137:0/3471202682 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f03bc09c200 con 0x5580c8e6eaf0 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc09c490 con 0x5580c8e6eaf0 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03a97fa640 1 -- 172.21.15.137:0/3471202682 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03bc0a0140 con 0x5580c8e6eaf0 2024-09-12T12:18:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03cafc6740 10 client.16032 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5580c8bb2620 con 0x5580c8e6eaf0 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.749+0000 7f03a97fa640 1 -- 172.21.15.137:0/3471202682 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03bc09c490 con 0x5580c8e6eaf0 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03a97fa640 1 -- 172.21.15.137:0/3471202682 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f03bc09ac50 con 0x5580c8e6eaf0 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03cafc6740 10 client.16032 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03cafc6740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03cafc6740 20 client.16032 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03cafc6740 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8e89ab0 0x5580c8e8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03cafc6740 4 client.16032 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c8de0a20 con 0x5580c8e89ab0 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03c57c2640 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8e89ab0 0x5580c8e8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03a97fa640 10 client.16032 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.750+0000 7f03c57c2640 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8e89ab0 0x5580c8e8bea0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70d492ebe4a2f833 server_cookie=28fad1a131c6a8df in_seq=0 out_seq=0 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03a97fa640 1 -- 172.21.15.137:0/3471202682 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580c8de0a20 con 0x5580c8e89ab0 2024-09-12T12:18:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03a97fa640 10 client.16032 handle_command_reply: tid=0 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 client.16032 shutdown 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 msgr2=0x5580c8e76bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e76bc0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0cd020 tx=0x7f03bc0cd470 comp rx=0 tx=0).stop 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 shutdown_connections 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e79cb0 0x5580c8e77640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e77100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8e89ab0 0x5580c8e8bea0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 --2- 172.21.15.137:0/3471202682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e6eaf0 0x5580c8e76bc0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 >> 172.21.15.137:0/3471202682 conn(0x5580c8e6b060 msgr2=0x5580c8e70dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03a97fa640 0 client.16032 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 shutdown_connections 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 1 -- 172.21.15.137:0/3471202682 wait complete. 2024-09-12T12:18:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.751+0000 7f03cafc6740 20 client.16032 trim_cache size 0 max 16384 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 Processor -- start 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 -- start start 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6eaf0 0x5580c8e77440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6f410 0x5580c8e8eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01afef0 con 0x5580c8e79cb0 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01ad7c0 con 0x5580c8e6eaf0 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03cafc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01ab610 con 0x5580c8e6f410 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03c67c4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6eaf0 0x5580c8e77440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03c67c4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58056/0 (socket says 172.21.15.137:58056) 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03c67c4640 1 -- 172.21.15.137:0/549291079 learned_addr learned my addr 172.21.15.137:0/549291079 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.767+0000 7f03c57c2640 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6f410 0x5580c8e8eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03c67c4640 1 -- 172.21.15.137:0/549291079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6f410 msgr2=0x5580c8e8eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03c67c4640 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6f410 0x5580c8e8eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03c67c4640 1 -- 172.21.15.137:0/549291079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6eaf0 msgr2=0x5580c8e77440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03c67c4640 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6eaf0 0x5580c8e77440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03c67c4640 1 -- 172.21.15.137:0/549291079 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c8e6f240 con 0x5580c8e79cb0 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6eaf0 0x5580c8e77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03c67c4640 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8f400 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f03b80116d0 tx=0x7f03b8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e1068687640ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03ab7fe640 1 -- 172.21.15.137:0/549291079 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8004b20 con 0x5580c8e79cb0 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03ab7fe640 1 -- 172.21.15.137:0/549291079 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b8004cc0 con 0x5580c8e79cb0 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.768+0000 7f03ab7fe640 1 -- 172.21.15.137:0/549291079 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80054a0 con 0x5580c8e79cb0 2024-09-12T12:18:31.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 -- 172.21.15.137:0/549291079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 msgr2=0x5580c8e8f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8f400 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f03b80116d0 tx=0x7f03b8021000 comp rx=0 tx=0).stop 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 -- 172.21.15.137:0/549291079 shutdown_connections 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8f400 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6f410 0x5580c8e8eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 --2- 172.21.15.137:0/549291079 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6eaf0 0x5580c8e77440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 -- 172.21.15.137:0/549291079 >> 172.21.15.137:0/549291079 conn(0x5580c8e6b060 msgr2=0x5580c8e78c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 -- 172.21.15.137:0/549291079 shutdown_connections 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 -- 172.21.15.137:0/549291079 wait complete. 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 -- start start 2024-09-12T12:18:31.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 Processor -- start 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6eaf0 0x5580c8e8f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e8c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.769+0000 7f03cafc6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01abc80 con 0x5580c8e79cb0 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03cafc6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01ad0f0 con 0x5580c8e6f410 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03cafc6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01ac350 con 0x5580c8e6eaf0 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03cafc6740 10 client.0 _ll_register_callbacks cb 0x7f03bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c57c2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e8c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c5fc3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6eaf0 0x5580c8e8f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58060/0 (socket says 172.21.15.137:58060) 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 -- 172.21.15.137:0/414508416 learned_addr learned my addr 172.21.15.137:0/414508416 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 -- 172.21.15.137:0/414508416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6eaf0 msgr2=0x5580c8e8f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6eaf0 0x5580c8e8f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 -- 172.21.15.137:0/414508416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 msgr2=0x5580c8e8c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e8c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 -- 172.21.15.137:0/414508416 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c8e8d330 con 0x5580c8e79cb0 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6eaf0 0x5580c8e8f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03c67c4640 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8ccf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f03b8004a80 tx=0x7f03b8004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b645e5d674d3fba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03a97fa640 1 -- 172.21.15.137:0/414508416 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8037020 con 0x5580c8e79cb0 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03a97fa640 1 -- 172.21.15.137:0/414508416 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f03b8004640 con 0x5580c8e79cb0 2024-09-12T12:18:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b8004cc0 con 0x5580c8e79cb0 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03a97fa640 1 -- 172.21.15.137:0/414508416 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03b8042050 con 0x5580c8e79cb0 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03cafc6740 10 client.16038 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.770+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5580c8ef4250 con 0x5580c8e79cb0 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03a97fa640 1 -- 172.21.15.137:0/414508416 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03b8004cc0 con 0x5580c8e79cb0 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03a97fa640 1 -- 172.21.15.137:0/414508416 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f03b80335f0 con 0x5580c8e79cb0 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03cafc6740 10 client.16038 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03cafc6740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03cafc6740 20 client.16038 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03cafc6740 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8ef4960 0x5580c8ef6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03cafc6740 4 client.16038 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580c8ddad70 con 0x5580c8ef4960 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8ef4960 0x5580c8ef6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03a97fa640 10 client.16038 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.771+0000 7f03c5fc3640 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8ef4960 0x5580c8ef6d80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9b4a93f60d7daa8 server_cookie=3b39757885b95e61 in_seq=0 out_seq=0 2024-09-12T12:18:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.846+0000 7f03a97fa640 1 -- 172.21.15.137:0/414508416 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580c8ddad70 con 0x5580c8ef4960 2024-09-12T12:18:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.846+0000 7f03a97fa640 10 client.16038 handle_command_reply: tid=0 2024-09-12T12:18:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 client.16038 shutdown 2024-09-12T12:18:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 msgr2=0x5580c8e8ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8ccf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f03b8004a80 tx=0x7f03b8004550 comp rx=0 tx=0).stop 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 shutdown_connections 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5580c8ef4960 0x5580c8ef6d80 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5580c8e79cb0 0x5580c8e8ccf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5580c8e6f410 0x5580c8e8c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 --2- 172.21.15.137:0/414508416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5580c8e6eaf0 0x5580c8e8f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 >> 172.21.15.137:0/414508416 conn(0x5580c8e6b060 msgr2=0x5580c8e6a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03a97fa640 0 client.16038 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 shutdown_connections 2024-09-12T12:18:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 1 -- 172.21.15.137:0/414508416 wait complete. 2024-09-12T12:18:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.847+0000 7f03cafc6740 20 client.16038 trim_cache size 0 max 16384 2024-09-12T12:18:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:18:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:18:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:18:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:18:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03ac0a2da0 msgr2=0x7f03ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03ac0a2da0 0x7f03ac0a5260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005fd0 tx=0x7f03b8009e90 comp rx=0 tx=0).stop 2024-09-12T12:18:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c0102ca0 msgr2=0x7f03c0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c0102ca0 0x7f03c0198330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00e9f0 tx=0x7f03bc00edf0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 shutdown_connections 2024-09-12T12:18:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03ac0a2da0 0x7f03ac0a5260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03c010c0c0 0x7f03c019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03c01035c0 0x7f03c0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 --2- 172.21.15.137:0/1153809822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03c0102ca0 0x7f03c0198330 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.848+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 >> 172.21.15.137:0/1153809822 conn(0x7f03c00fc6f0 msgr2=0x7f03c00fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.849+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 shutdown_connections 2024-09-12T12:18:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:31.849+0000 7f03c8227640 1 -- 172.21.15.137:0/1153809822 wait complete. 2024-09-12T12:18:32.729 INFO:tasks.workunit.client.0.smithi137.stdout: 1 412044 8.57 MB/sec execute 127 sec latency 48.202 ms 2024-09-12T12:18:32.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:32 smithi137 ceph-mon[30950]: pgmap v163: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:18:32.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:32 smithi189 ceph-mon[35475]: pgmap v163: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:18:32.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:32 smithi171 ceph-mon[35098]: pgmap v163: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:18:33.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 415038 8.64 MB/sec execute 128 sec latency 17.110 ms 2024-09-12T12:18:34.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 416559 8.62 MB/sec execute 129 sec latency 12.379 ms 2024-09-12T12:18:34.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:34 smithi137 ceph-mon[30950]: pgmap v164: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:18:34.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:34 smithi189 ceph-mon[35475]: pgmap v164: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:18:34.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:34 smithi171 ceph-mon[35098]: pgmap v164: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:18:35.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 419017 8.66 MB/sec execute 130 sec latency 8.951 ms 2024-09-12T12:18:36.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 421592 8.69 MB/sec execute 131 sec latency 14.364 ms 2024-09-12T12:18:36.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:36 smithi137 ceph-mon[30950]: pgmap v165: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 452 op/s 2024-09-12T12:18:36.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:36 smithi189 ceph-mon[35475]: pgmap v165: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 452 op/s 2024-09-12T12:18:36.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:36 smithi171 ceph-mon[35098]: pgmap v165: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 452 op/s 2024-09-12T12:18:37.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 423296 8.69 MB/sec execute 132 sec latency 39.228 ms 2024-09-12T12:18:38.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 426037 8.76 MB/sec execute 133 sec latency 25.924 ms 2024-09-12T12:18:38.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:38 smithi137 ceph-mon[30950]: pgmap v166: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-12T12:18:38.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:38 smithi189 ceph-mon[35475]: pgmap v166: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-12T12:18:38.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:38 smithi171 ceph-mon[35098]: pgmap v166: 97 pgs: 97 active+clean; 115 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 317 op/s 2024-09-12T12:18:39.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 428552 8.77 MB/sec execute 134 sec latency 10.212 ms 2024-09-12T12:18:40.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 430406 8.78 MB/sec execute 135 sec latency 27.987 ms 2024-09-12T12:18:40.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:40 smithi137 ceph-mon[30950]: pgmap v167: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-09-12T12:18:40.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:40 smithi189 ceph-mon[35475]: pgmap v167: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-09-12T12:18:40.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:40 smithi171 ceph-mon[35098]: pgmap v167: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 448 op/s 2024-09-12T12:18:41.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 433152 8.84 MB/sec execute 136 sec latency 10.718 ms 2024-09-12T12:18:42.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 435664 8.87 MB/sec execute 137 sec latency 76.488 ms 2024-09-12T12:18:42.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:42 smithi137 ceph-mon[30950]: pgmap v168: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-09-12T12:18:42.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:42 smithi189 ceph-mon[35475]: pgmap v168: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-09-12T12:18:42.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:42 smithi171 ceph-mon[35098]: pgmap v168: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-09-12T12:18:43.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 437411 8.87 MB/sec execute 138 sec latency 25.335 ms 2024-09-12T12:18:44.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 440194 8.93 MB/sec execute 139 sec latency 13.576 ms 2024-09-12T12:18:44.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:44 smithi137 ceph-mon[30950]: pgmap v169: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-09-12T12:18:44.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:44 smithi189 ceph-mon[35475]: pgmap v169: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-09-12T12:18:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 -- 172.21.15.137:0/2336040331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 msgr2=0x7f712c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 --2- 172.21.15.137:0/2336040331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c106da0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f711c009930 tx=0x7f711c02a420 comp rx=0 tx=0).stop 2024-09-12T12:18:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 -- 172.21.15.137:0/2336040331 shutdown_connections 2024-09-12T12:18:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 --2- 172.21.15.137:0/2336040331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f712c101350 0x7f712c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 --2- 172.21.15.137:0/2336040331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f712c1009b0 0x7f712c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 --2- 172.21.15.137:0/2336040331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c106da0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 -- 172.21.15.137:0/2336040331 >> 172.21.15.137:0/2336040331 conn(0x7f712c077990 msgr2=0x7f712c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 -- 172.21.15.137:0/2336040331 shutdown_connections 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.942+0000 7f71326f8640 1 -- 172.21.15.137:0/2336040331 wait complete. 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 Processor -- start 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 -- start start 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f712c1009b0 0x7f712c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f712c101350 0x7f712c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f712c110e00 con 0x7f712c1069c0 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f712c110c80 con 0x7f712c1009b0 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f71326f8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f712c110f80 con 0x7f712c101350 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46666/0 (socket says 172.21.15.137:46666) 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7123fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f712c101350 0x7f712c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 -- 172.21.15.137:0/3123807936 learned_addr learned my addr 172.21.15.137:0/3123807936 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f712bfff640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f712c1009b0 0x7f712c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 -- 172.21.15.137:0/3123807936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f712c101350 msgr2=0x7f712c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f712c101350 0x7f712c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 -- 172.21.15.137:0/3123807936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f712c1009b0 msgr2=0x7f712c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f712c1009b0 0x7f712c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 -- 172.21.15.137:0/3123807936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c19c260 con 0x7f712c1069c0 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.943+0000 7f7130c95640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c19bb30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f712400eb10 tx=0x7f712400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.944+0000 7f7129ffb640 1 -- 172.21.15.137:0/3123807936 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124004b20 con 0x7f712c1069c0 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.944+0000 7f7129ffb640 1 -- 172.21.15.137:0/3123807936 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7124004cc0 con 0x7f712c1069c0 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.944+0000 7f7129ffb640 1 -- 172.21.15.137:0/3123807936 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124010760 con 0x7f712c1069c0 2024-09-12T12:18:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.944+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c19c550 con 0x7f712c1069c0 2024-09-12T12:18:44.948 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.944+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1a3de0 con 0x7f712c1069c0 2024-09-12T12:18:44.954 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:44 smithi171 ceph-mon[35098]: pgmap v169: 97 pgs: 97 active+clean; 122 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-09-12T12:18:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.946+0000 7f7135497740 1 Processor -- start 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.946+0000 7f7129ffb640 1 -- 172.21.15.137:0/3123807936 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7124005020 con 0x7f712c1069c0 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.946+0000 7f7135497740 1 -- start start 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7135497740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70112fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7135497740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7135497740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b70113f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7135497740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b7010942f0 con 0x55b70112f970 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7135497740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b70103c270 con 0x55b70113cec0 2024-09-12T12:18:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7135497740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7010902b0 con 0x55b701130290 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70112fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70112fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46668/0 (socket says 172.21.15.137:46668) 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f712bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 -- 172.21.15.137:0/2560698282 learned_addr learned my addr 172.21.15.137:0/2560698282 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b70113f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7129ffb640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71180a2e20 0x7f71180a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 -- 172.21.15.137:0/2560698282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 msgr2=0x55b70113c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 -- 172.21.15.137:0/2560698282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 msgr2=0x55b70113f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b70113f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 -- 172.21.15.137:0/2560698282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70103b340 con 0x55b70112f970 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7130c95640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71180a2e20 0x7f71180a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7123fff640 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70112fd50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7114013630 tx=0x7f7114009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca53cc575e06677 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.947+0000 7f7129ffb640 1 -- 172.21.15.137:0/3123807936 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f71240cf890 con 0x7f712c1069c0 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f71227fc640 1 -- 172.21.15.137:0/2560698282 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114004050 con 0x55b70112f970 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f71227fc640 1 -- 172.21.15.137:0/2560698282 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71140041f0 con 0x55b70112f970 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f71227fc640 1 -- 172.21.15.137:0/2560698282 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711400f7e0 con 0x55b70112f970 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 -- 172.21.15.137:0/2560698282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 msgr2=0x55b70112fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7130c95640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71180a2e20 0x7f71180a52e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f712409bb00 tx=0x7f712406ddc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70112fd50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7114013630 tx=0x7f7114009780 comp rx=0 tx=0).stop 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 -- 172.21.15.137:0/2560698282 shutdown_connections 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b70113f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 --2- 172.21.15.137:0/2560698282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70112fd50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 -- 172.21.15.137:0/2560698282 >> 172.21.15.137:0/2560698282 conn(0x55b70112c060 msgr2=0x55b70112c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 -- 172.21.15.137:0/2560698282 shutdown_connections 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.948+0000 7f7135497740 1 -- 172.21.15.137:0/2560698282 wait complete. 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 -- start start 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 Processor -- start 2024-09-12T12:18:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70112f970 0x55b701139e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b70113a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70113cec0 0x55b70113a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b70113faf0 con 0x55b70113cec0 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b70101d290 con 0x55b701130290 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b70113fc70 con 0x55b70112f970 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7135497740 10 client.0 _ll_register_callbacks cb 0x55b70104e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7123fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70113cec0 0x55b70113a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f712bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b70113a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f7123fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70112f970 0x55b701139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f712bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b70113a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37612/0 (socket says 172.21.15.137:37612) 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.949+0000 7f712bfff640 1 -- 172.21.15.137:0/1796162933 learned_addr learned my addr 172.21.15.137:0/1796162933 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f712bfff640 1 -- 172.21.15.137:0/1796162933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70112f970 msgr2=0x55b701139e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f712bfff640 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70112f970 0x55b701139e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f712bfff640 1 -- 172.21.15.137:0/1796162933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70113cec0 msgr2=0x55b70113a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f712bfff640 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70113cec0 0x55b70113a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f712bfff640 1 -- 172.21.15.137:0/1796162933 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b700da7210 con 0x55b701130290 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f712bfff640 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b70113a340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f711c009930 tx=0x7f711c0317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9113b08a030a063f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f710b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:18:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f710b7fe640 1 -- 172.21.15.137:0/1796162933 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c035040 con 0x55b701130290 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71140041f0 con 0x55b701130290 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f710b7fe640 1 -- 172.21.15.137:0/1796162933 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f711c004460 con 0x55b701130290 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f710b7fe640 1 -- 172.21.15.137:0/1796162933 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f711c031a10 con 0x55b701130290 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f7135497740 10 client.25487 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.950+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b700e73620 con 0x55b701130290 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f710b7fe640 1 -- 172.21.15.137:0/1796162933 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71140041f0 con 0x55b701130290 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f710b7fe640 1 -- 172.21.15.137:0/1796162933 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f711c04f880 con 0x55b701130290 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7135497740 10 client.25487 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7135497740 10 client.25487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7135497740 20 client.25487 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7135497740 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b70114f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7135497740 4 client.25487 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7010a1a20 con 0x55b70114ce10 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7123fff640 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b70114f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f710b7fe640 10 client.25487 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.951+0000 7f7123fff640 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b70114f200 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd9cc0a2a5f6e51 server_cookie=453cca361f0d633a in_seq=0 out_seq=0 2024-09-12T12:18:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f710b7fe640 1 -- 172.21.15.137:0/1796162933 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7010a1a20 con 0x55b70114ce10 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f710b7fe640 10 client.25487 handle_command_reply: tid=0 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 client.25487 shutdown 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 msgr2=0x55b70113a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b70113a340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f711c009930 tx=0x7f711c0317e0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 shutdown_connections 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b70114f200 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70113cec0 0x55b70113a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f710b7fe640 0 client.25487 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b70113a340 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 --2- 172.21.15.137:0/1796162933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70112f970 0x55b701139e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 >> 172.21.15.137:0/1796162933 conn(0x55b70112c060 msgr2=0x55b701131dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 shutdown_connections 2024-09-12T12:18:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.952+0000 7f7135497740 1 -- 172.21.15.137:0/1796162933 wait complete. 2024-09-12T12:18:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.953+0000 7f7135497740 20 client.25487 trim_cache size 0 max 16384 2024-09-12T12:18:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 Processor -- start 2024-09-12T12:18:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 -- start start 2024-09-12T12:18:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70113c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b701154af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1adcf0 con 0x55b70112f970 2024-09-12T12:18:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1ab5c0 con 0x55b70113cec0 2024-09-12T12:18:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.971+0000 7f7135497740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1a9410 con 0x55b701130290 2024-09-12T12:18:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:49316/0 (socket says 172.21.15.137:49316) 2024-09-12T12:18:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 -- 172.21.15.137:0/414529279 learned_addr learned my addr 172.21.15.137:0/414529279 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f7123fff640 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70113c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f7123fff640 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b701154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 -- 172.21.15.137:0/414529279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 msgr2=0x55b701154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b701154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 -- 172.21.15.137:0/414529279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 msgr2=0x55b70113c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70113c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f712bfff640 1 -- 172.21.15.137:0/414529279 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7011300c0 con 0x55b701130290 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.972+0000 7f7123fff640 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70113c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.973+0000 7f712bfff640 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113a6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f711c04fc80 tx=0x7f711c02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8cf8108db06567c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.973+0000 7f7121ffb640 1 -- 172.21.15.137:0/414529279 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c035030 con 0x55b701130290 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.973+0000 7f7121ffb640 1 -- 172.21.15.137:0/414529279 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711c041070 con 0x55b701130290 2024-09-12T12:18:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.973+0000 7f7121ffb640 1 -- 172.21.15.137:0/414529279 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c04f6e0 con 0x55b701130290 2024-09-12T12:18:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.974+0000 7f7135497740 1 -- 172.21.15.137:0/414529279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 msgr2=0x55b70113a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.974+0000 7f7135497740 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113a6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f711c04fc80 tx=0x7f711c02a420 comp rx=0 tx=0).stop 2024-09-12T12:18:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 -- 172.21.15.137:0/414529279 shutdown_connections 2024-09-12T12:18:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b70113cec0 0x55b701154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b701130290 0x55b70113a6b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 --2- 172.21.15.137:0/414529279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b70113c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 -- 172.21.15.137:0/414529279 >> 172.21.15.137:0/414529279 conn(0x55b70112c060 msgr2=0x55b70112b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 -- 172.21.15.137:0/414529279 shutdown_connections 2024-09-12T12:18:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 -- 172.21.15.137:0/414529279 wait complete. 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.975+0000 7f7135497740 1 -- start start 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 1 Processor -- start 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b701133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b701144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70113cec0 0x55b701145400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1a9a80 con 0x55b70112f970 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1aaef0 con 0x55b701130290 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f712c1aa150 con 0x55b70113cec0 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7135497740 10 client.0 _ll_register_callbacks cb 0x7f711c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7123fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b701133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f712bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b701144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f712bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b701144ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37624/0 (socket says 172.21.15.137:37624) 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7123fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b701133fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46704/0 (socket says 172.21.15.137:46704) 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f712bfff640 1 -- 172.21.15.137:0/4046799552 learned_addr learned my addr 172.21.15.137:0/4046799552 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.976+0000 7f7123fff640 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70113cec0 0x55b701145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f712bfff640 1 -- 172.21.15.137:0/4046799552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70113cec0 msgr2=0x55b701145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f712bfff640 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70113cec0 0x55b701145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f712bfff640 1 -- 172.21.15.137:0/4046799552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 msgr2=0x55b701133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f712bfff640 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b701133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f712bfff640 1 -- 172.21.15.137:0/4046799552 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7011459c0 con 0x55b701130290 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f7123fff640 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b701133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f712bfff640 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b701144ec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f711c005d00 tx=0x7f711c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e220e8bd349fb7b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f710b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f710b7fe640 1 -- 172.21.15.137:0/4046799552 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c05e070 con 0x55b701130290 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f710b7fe640 1 -- 172.21.15.137:0/4046799552 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f711c03d920 con 0x55b701130290 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f711c041070 con 0x55b701130290 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f710b7fe640 1 -- 172.21.15.137:0/4046799552 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f711c06a050 con 0x55b701130290 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f7135497740 10 client.25493 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.977+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b7011cf2d0 con 0x55b701130290 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f710b7fe640 1 -- 172.21.15.137:0/4046799552 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711c0035f0 con 0x55b701130290 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f710b7fe640 1 -- 172.21.15.137:0/4046799552 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f711c03c470 con 0x55b701130290 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7135497740 10 client.25493 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7135497740 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7135497740 20 client.25493 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7135497740 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b7011d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7135497740 4 client.25493 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:18:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f711c041070 con 0x55b70114ce10 2024-09-12T12:18:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7123fff640 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b7011d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f710b7fe640 10 client.25493 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:44.978+0000 7f7123fff640 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b7011d18f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5de746447df883c server_cookie=abdfb8a52a2a08f in_seq=0 out_seq=0 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.064+0000 7f710b7fe640 1 -- 172.21.15.137:0/4046799552 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f711c041070 con 0x55b70114ce10 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.064+0000 7f710b7fe640 10 client.25493 handle_command_reply: tid=0 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.064+0000 7f7135497740 1 client.25493 shutdown 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.064+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 msgr2=0x55b701144ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.064+0000 7f7135497740 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b701144ec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f711c005d00 tx=0x7f711c0058d0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 shutdown_connections 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b70113cec0 0x55b701145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b701130290 0x55b701144ec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b70114ce10 0x55b7011d18f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 --2- 172.21.15.137:0/4046799552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b70112f970 0x55b701133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f710b7fe640 0 client.25493 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 >> 172.21.15.137:0/4046799552 conn(0x55b70112c060 msgr2=0x55b701147d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 shutdown_connections 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 1 -- 172.21.15.137:0/4046799552 wait complete. 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f7135497740 20 client.25493 trim_cache size 0 max 16384 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:18:45.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:18:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71180a2e20 msgr2=0x7f71180a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71180a2e20 0x7f71180a52e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f712409bb00 tx=0x7f712406ddc0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 msgr2=0x7f712c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:45.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c19bb30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f712400eb10 tx=0x7f712400c2d0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 shutdown_connections 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71180a2e20 0x7f71180a52e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f712c1069c0 0x7f712c19bb30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f712c101350 0x7f712c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 --2- 172.21.15.137:0/3123807936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f712c1009b0 0x7f712c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 >> 172.21.15.137:0/3123807936 conn(0x7f712c077990 msgr2=0x7f712c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 shutdown_connections 2024-09-12T12:18:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:45.065+0000 7f71326f8640 1 -- 172.21.15.137:0/3123807936 wait complete. 2024-09-12T12:18:45.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 442116 8.91 MB/sec execute 140 sec latency 35.610 ms 2024-09-12T12:18:46.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 444024 8.94 MB/sec execute 141 sec latency 12.896 ms 2024-09-12T12:18:46.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:46 smithi137 ceph-mon[30950]: pgmap v170: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-09-12T12:18:46.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:46 smithi189 ceph-mon[35475]: pgmap v170: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-09-12T12:18:46.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:46 smithi171 ceph-mon[35098]: pgmap v170: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 452 op/s 2024-09-12T12:18:47.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 446912 9.00 MB/sec execute 142 sec latency 52.342 ms 2024-09-12T12:18:48.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 448434 8.98 MB/sec execute 143 sec latency 25.163 ms 2024-09-12T12:18:48.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:48 smithi137 ceph-mon[30950]: pgmap v171: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-12T12:18:48.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:48 smithi189 ceph-mon[35475]: pgmap v171: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-12T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:48 smithi171 ceph-mon[35098]: pgmap v171: 97 pgs: 97 active+clean; 129 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 268 op/s 2024-09-12T12:18:49.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 450945 9.02 MB/sec execute 144 sec latency 9.923 ms 2024-09-12T12:18:50.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 453517 9.04 MB/sec execute 145 sec latency 10.965 ms 2024-09-12T12:18:50.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:50 smithi137 ceph-mon[30950]: pgmap v172: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 443 op/s 2024-09-12T12:18:50.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:50 smithi189 ceph-mon[35475]: pgmap v172: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 443 op/s 2024-09-12T12:18:50.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:50 smithi171 ceph-mon[35098]: pgmap v172: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 443 op/s 2024-09-12T12:18:51.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 455485 9.06 MB/sec execute 146 sec latency 18.590 ms 2024-09-12T12:18:52.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 457958 9.09 MB/sec execute 147 sec latency 34.690 ms 2024-09-12T12:18:52.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:52 smithi137 ceph-mon[30950]: pgmap v173: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-12T12:18:52.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:52 smithi189 ceph-mon[35475]: pgmap v173: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-12T12:18:52.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:52 smithi171 ceph-mon[35098]: pgmap v173: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-12T12:18:53.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 459236 9.18 MB/sec execute 148 sec latency 38.548 ms 2024-09-12T12:18:54.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 462229 9.24 MB/sec execute 149 sec latency 13.627 ms 2024-09-12T12:18:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:54 smithi137 ceph-mon[30950]: pgmap v174: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-12T12:18:54.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:54 smithi189 ceph-mon[35475]: pgmap v174: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-12T12:18:54.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:54 smithi171 ceph-mon[35098]: pgmap v174: 97 pgs: 97 active+clean; 123 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-12T12:18:55.730 INFO:tasks.workunit.client.0.smithi137.stdout: 1 464696 9.25 MB/sec execute 150 sec latency 13.145 ms 2024-09-12T12:18:56.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 466703 9.26 MB/sec execute 151 sec latency 18.675 ms 2024-09-12T12:18:56.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:56 smithi137 ceph-mon[30950]: pgmap v175: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 449 op/s 2024-09-12T12:18:56.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:56 smithi189 ceph-mon[35475]: pgmap v175: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 449 op/s 2024-09-12T12:18:56.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:56 smithi171 ceph-mon[35098]: pgmap v175: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 449 op/s 2024-09-12T12:18:57.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 469361 9.31 MB/sec execute 152 sec latency 46.159 ms 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 -- 172.21.15.137:0/2754723830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01009b0 msgr2=0x7f06f0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 --2- 172.21.15.137:0/2754723830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01009b0 0x7f06f0100e10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009930 tx=0x7f06e002a420 comp rx=0 tx=0).stop 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 -- 172.21.15.137:0/2754723830 shutdown_connections 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 --2- 172.21.15.137:0/2754723830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f0101350 0x7f06f010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 --2- 172.21.15.137:0/2754723830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01009b0 0x7f06f0100e10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 --2- 172.21.15.137:0/2754723830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f01069c0 0x7f06f0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 -- 172.21.15.137:0/2754723830 >> 172.21.15.137:0/2754723830 conn(0x7f06f0077990 msgr2=0x7f06f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:58.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 -- 172.21.15.137:0/2754723830 shutdown_connections 2024-09-12T12:18:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 -- 172.21.15.137:0/2754723830 wait complete. 2024-09-12T12:18:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 Processor -- start 2024-09-12T12:18:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.159+0000 7f06f7482640 1 -- start start 2024-09-12T12:18:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f7482640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f01009b0 0x7f06f01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f7482640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f0101350 0x7f06f01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f7482640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01069c0 0x7f06f01a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f7482640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0110e00 con 0x7f06f01069c0 2024-09-12T12:18:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f7482640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0110c80 con 0x7f06f0101350 2024-09-12T12:18:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f7482640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0110f80 con 0x7f06f01009b0 2024-09-12T12:18:58.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f4a1d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f0101350 0x7f06f01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f521e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f01009b0 0x7f06f01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01069c0 0x7f06f01a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f4a1d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f0101350 0x7f06f01996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:58366/0 (socket says 172.21.15.137:58366) 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f4a1d640 1 -- 172.21.15.137:0/3991463381 learned_addr learned my addr 172.21.15.137:0/3991463381 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f521e640 1 -- 172.21.15.137:0/3991463381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f0101350 msgr2=0x7f06f01996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f521e640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f0101350 0x7f06f01996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f521e640 1 -- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01069c0 msgr2=0x7f06f01a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f521e640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01069c0 0x7f06f01a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f521e640 1 -- 172.21.15.137:0/3991463381 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f01a4760 con 0x7f06f01009b0 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.160+0000 7f06f521e640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f01009b0 0x7f06f01991a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00eb00 tx=0x7f06ec00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.161+0000 7f06e67fc640 1 -- 172.21.15.137:0/3991463381 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec010430 con 0x7f06f01009b0 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.161+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f0196f90 con 0x7f06f01009b0 2024-09-12T12:18:58.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.161+0000 7f06e67fc640 1 -- 172.21.15.137:0/3991463381 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec010ab0 con 0x7f06f01009b0 2024-09-12T12:18:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.161+0000 7f06e67fc640 1 -- 172.21.15.137:0/3991463381 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec017bb0 con 0x7f06f01009b0 2024-09-12T12:18:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.161+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f0197530 con 0x7f06f01009b0 2024-09-12T12:18:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 Processor -- start 2024-09-12T12:18:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 -- start start 2024-09-12T12:18:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddee7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a0dde4c2f0 con 0x55a0ddee7af0 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a0dddf4270 con 0x55a0ddef2cb0 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06fa221740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0dde482b0 con 0x55a0ddee8410 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06e67fc640 1 -- 172.21.15.137:0/3991463381 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f06ec010c60 con 0x7f06f01009b0 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddee7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.163+0000 7f06f4a1d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddee7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49292/0 (socket says 172.21.15.137:49292) 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 -- 172.21.15.137:0/2283819526 learned_addr learned my addr 172.21.15.137:0/2283819526 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f521e640 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 -- 172.21.15.137:0/2283819526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 msgr2=0x55a0ddef2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 -- 172.21.15.137:0/2283819526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 msgr2=0x55a0ddef50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 -- 172.21.15.137:0/2283819526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0dddf3340 con 0x55a0ddee7af0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddee7ed0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009ca0 tx=0x7f06e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e183f899b88699a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06e67fc640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06db7fe640 1 -- 172.21.15.137:0/2283819526 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e801e070 con 0x55a0ddee7af0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06db7fe640 1 -- 172.21.15.137:0/2283819526 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8009e50 con 0x55a0ddee7af0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06db7fe640 1 -- 172.21.15.137:0/2283819526 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e800ea80 con 0x55a0ddee7af0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06f521e640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.164+0000 7f06e67fc640 1 -- 172.21.15.137:0/3991463381 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f06ec0cfd50 con 0x7f06f01009b0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06f521e640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55a0ddee4ec0 tx=0x7f06ec00cf10 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 -- 172.21.15.137:0/2283819526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 msgr2=0x55a0ddee7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddee7ed0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009ca0 tx=0x7f06e8011810 comp rx=0 tx=0).stop 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 -- 172.21.15.137:0/2283819526 shutdown_connections 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 --2- 172.21.15.137:0/2283819526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddee7ed0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 -- 172.21.15.137:0/2283819526 >> 172.21.15.137:0/2283819526 conn(0x55a0ddee4060 msgr2=0x55a0ddee4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 -- 172.21.15.137:0/2283819526 shutdown_connections 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 -- 172.21.15.137:0/2283819526 wait complete. 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.165+0000 7f06fa221740 1 -- start start 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 1 Processor -- start 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee7af0 0x55a0ddeefc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddef5880 con 0x55a0ddef2cb0 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a0dddd5290 con 0x55a0ddee7af0 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a0ddef5a00 con 0x55a0ddee8410 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06fa221740 10 client.0 _ll_register_callbacks cb 0x55a0dde06140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49308/0 (socket says 172.21.15.137:49308) 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f5a1f640 1 -- 172.21.15.137:0/2577116860 learned_addr learned my addr 172.21.15.137:0/2577116860 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f4a1d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee7af0 0x55a0ddeefc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f4a1d640 1 -- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee7af0 msgr2=0x55a0ddeefc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f4a1d640 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee7af0 0x55a0ddeefc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f4a1d640 1 -- 172.21.15.137:0/2577116860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 msgr2=0x55a0ddef06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.175 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f4a1d640 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f4a1d640 1 -- 172.21.15.137:0/2577116860 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ddb5f210 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.166+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06f4a1d640 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef0180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009900 tx=0x7f06e0004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36775b95525eb4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06d8ff9640 1 -- 172.21.15.137:0/2577116860 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0035040 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e8009e50 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06d8ff9640 1 -- 172.21.15.137:0/2577116860 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f06e0005020 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06d8ff9640 1 -- 172.21.15.137:0/2577116860 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06e0005d50 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 10 client.25567 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0ddc2b620 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06d8ff9640 1 -- 172.21.15.137:0/2577116860 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8009e50 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06d8ff9640 1 -- 172.21.15.137:0/2577116860 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f06e003a670 con 0x55a0ddee8410 2024-09-12T12:18:58.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 10 client.25567 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 20 client.25567 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf02c90 0x55a0ddf05080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 4 client.25567 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.167+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0dde59a20 con 0x55a0ddf02c90 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.168+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf02c90 0x55a0ddf05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.168+0000 7f06d8ff9640 10 client.25567 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.168+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf02c90 0x55a0ddf05080 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb8c3a40f85d77e server_cookie=482aad2b32f4600d in_seq=0 out_seq=0 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06d8ff9640 1 -- 172.21.15.137:0/2577116860 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0dde59a20 con 0x55a0ddf02c90 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06d8ff9640 10 client.25567 handle_command_reply: tid=0 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 client.25567 shutdown 2024-09-12T12:18:58.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 msgr2=0x55a0ddef0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef0180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f06e0009900 tx=0x7f06e0004250 comp rx=0 tx=0).stop 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 shutdown_connections 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf02c90 0x55a0ddf05080 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddef06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddef0180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 --2- 172.21.15.137:0/2577116860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee7af0 0x55a0ddeefc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 >> 172.21.15.137:0/2577116860 conn(0x55a0ddee4060 msgr2=0x55a0ddee31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06d8ff9640 0 client.25567 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 shutdown_connections 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 1 -- 172.21.15.137:0/2577116860 wait complete. 2024-09-12T12:18:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.169+0000 7f06fa221740 20 client.25567 trim_cache size 0 max 16384 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 Processor -- start 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 -- start start 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee7af0 0x55a0ddef04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee8410 0x55a0ddf0a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddf0ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01add10 con 0x55a0ddef2cb0 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ab5e0 con 0x55a0ddee8410 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06fa221740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01a9450 con 0x55a0ddee7af0 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.187+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddf0ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddf0ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49312/0 (socket says 172.21.15.137:49312) 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f4a1d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee8410 0x55a0ddf0a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 -- 172.21.15.137:0/818290972 learned_addr learned my addr 172.21.15.137:0/818290972 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee7af0 0x55a0ddef04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 -- 172.21.15.137:0/818290972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee7af0 msgr2=0x55a0ddef04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee7af0 0x55a0ddef04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 -- 172.21.15.137:0/818290972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee8410 msgr2=0x55a0ddf0a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee8410 0x55a0ddf0a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 -- 172.21.15.137:0/818290972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ddeff660 con 0x55a0ddef2cb0 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddf0ae60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009230 tx=0x7f06e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7840b5fc5fd69415 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06daffd640 1 -- 172.21.15.137:0/818290972 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e801e040 con 0x55a0ddef2cb0 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06daffd640 1 -- 172.21.15.137:0/818290972 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e801f040 con 0x55a0ddef2cb0 2024-09-12T12:18:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.188+0000 7f06daffd640 1 -- 172.21.15.137:0/818290972 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8003a10 con 0x55a0ddef2cb0 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 -- 172.21.15.137:0/818290972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 msgr2=0x55a0ddf0ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddf0ae60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009230 tx=0x7f06e8005f90 comp rx=0 tx=0).stop 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 -- 172.21.15.137:0/818290972 shutdown_connections 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddef2cb0 0x55a0ddf0ae60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddee8410 0x55a0ddf0a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 --2- 172.21.15.137:0/818290972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee7af0 0x55a0ddef04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 -- 172.21.15.137:0/818290972 >> 172.21.15.137:0/818290972 conn(0x55a0ddee4060 msgr2=0x55a0ddeeaef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 -- 172.21.15.137:0/818290972 shutdown_connections 2024-09-12T12:18:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 -- 172.21.15.137:0/818290972 wait complete. 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 -- start start 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.189+0000 7f06fa221740 1 Processor -- start 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddefaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddefb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddeec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01a9aa0 con 0x55a0ddee7af0 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aaf10 con 0x55a0ddef2cb0 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aa170 con 0x55a0ddee8410 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 10 client.0 _ll_register_callbacks cb 0x7f06e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddefaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddefaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49324/0 (socket says 172.21.15.137:49324) 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 -- 172.21.15.137:0/554286284 learned_addr learned my addr 172.21.15.137:0/554286284 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:18:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f4a1d640 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddefb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddeec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 -- 172.21.15.137:0/554286284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 msgr2=0x55a0ddefb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddefb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 -- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 msgr2=0x55a0ddeec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddeec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 -- 172.21.15.137:0/554286284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ddeeccc0 con 0x55a0ddee7af0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06f5a1f640 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddefaf30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06e80059d0 tx=0x7f06e80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd7ed961300ce73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06d8ff9640 1 -- 172.21.15.137:0/554286284 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e803c060 con 0x55a0ddee7af0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06d8ff9640 1 -- 172.21.15.137:0/554286284 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f06e800ea50 con 0x55a0ddee7af0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.190+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e801f040 con 0x55a0ddee7af0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06d8ff9640 1 -- 172.21.15.137:0/554286284 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06e801e040 con 0x55a0ddee7af0 2024-09-12T12:18:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 10 client.16074 fetch_fsmap learned FSMap version 29 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0ddf6d440 con 0x55a0ddee7af0 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06d8ff9640 1 -- 172.21.15.137:0/554286284 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e801f040 con 0x55a0ddee7af0 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06d8ff9640 1 -- 172.21.15.137:0/554286284 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f06e8019430 con 0x55a0ddee7af0 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 10 client.16074 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 20 client.16074 populate_metadata read hostname 'smithi137' 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf6d8c0 0x55a0ddf6fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 4 client.16074 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0dde53d70 con 0x55a0ddf6d8c0 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06f4a1d640 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf6d8c0 0x55a0ddf6fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.192+0000 7f06d8ff9640 10 client.16074 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.191+0000 7f06f4a1d640 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf6d8c0 0x55a0ddf6fcb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee8c61c6cac37df server_cookie=a786cfa1693a6237 in_seq=0 out_seq=0 2024-09-12T12:18:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.260+0000 7f06d8ff9640 1 -- 172.21.15.137:0/554286284 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0dde53d70 con 0x55a0ddf6d8c0 2024-09-12T12:18:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.260+0000 7f06d8ff9640 10 client.16074 handle_command_reply: tid=0 2024-09-12T12:18:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.260+0000 7f06fa221740 1 client.16074 shutdown 2024-09-12T12:18:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.260+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 msgr2=0x55a0ddefaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddefaf30 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06e80059d0 tx=0x7f06e80035f0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 shutdown_connections 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a0ddef2cb0 0x55a0ddeec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a0ddee8410 0x55a0ddefb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a0ddf6d8c0 0x55a0ddf6fcb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06d8ff9640 0 client.16074 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 --2- 172.21.15.137:0/554286284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0ddee7af0 0x55a0ddefaf30 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 >> 172.21.15.137:0/554286284 conn(0x55a0ddee4060 msgr2=0x55a0ddee9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 shutdown_connections 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 1 -- 172.21.15.137:0/554286284 wait complete. 2024-09-12T12:18:58.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.261+0000 7f06fa221740 20 client.16074 trim_cache size 0 max 16384 2024-09-12T12:18:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:18:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:18:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:18:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:18:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.263+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 msgr2=0x7f06dc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.263+0000 7f06f7482640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55a0ddee4ec0 tx=0x7f06ec00cf10 comp rx=0 tx=0).stop 2024-09-12T12:18:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.263+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f01009b0 msgr2=0x7f06f01991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.263+0000 7f06f7482640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f01009b0 0x7f06f01991a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00eb00 tx=0x7f06ec00c2d0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 shutdown_connections 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f01069c0 0x7f06f01a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f0101350 0x7f06f01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 --2- 172.21.15.137:0/3991463381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f01009b0 0x7f06f01991a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 >> 172.21.15.137:0/3991463381 conn(0x7f06f0077990 msgr2=0x7f06f01037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 shutdown_connections 2024-09-12T12:18:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:18:58.264+0000 7f06f7482640 1 -- 172.21.15.137:0/3991463381 wait complete. 2024-09-12T12:18:58.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 471356 9.29 MB/sec execute 153 sec latency 40.442 ms 2024-09-12T12:18:58.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:18:58 smithi137 ceph-mon[30950]: pgmap v176: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-09-12T12:18:58.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:18:58 smithi189 ceph-mon[35475]: pgmap v176: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-09-12T12:18:58.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:18:58 smithi171 ceph-mon[35098]: pgmap v176: 97 pgs: 97 active+clean; 130 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-09-12T12:18:59.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 473329 9.32 MB/sec execute 154 sec latency 11.589 ms 2024-09-12T12:19:00.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 475301 9.32 MB/sec execute 155 sec latency 18.955 ms 2024-09-12T12:19:00.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:00 smithi137 ceph-mon[30950]: pgmap v177: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 490 op/s 2024-09-12T12:19:00.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:00 smithi189 ceph-mon[35475]: pgmap v177: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 490 op/s 2024-09-12T12:19:00.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:00 smithi171 ceph-mon[35098]: pgmap v177: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 490 op/s 2024-09-12T12:19:01.081 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T12:19:01.082 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:19:01.343 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:19:01.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 476961 9.32 MB/sec execute 156 sec latency 45.747 ms 2024-09-12T12:19:01.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 -- 172.21.15.137:0/969983278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660100e20 msgr2=0x7fd660101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 --2- 172.21.15.137:0/969983278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660100e20 0x7fd660101200 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009900 tx=0x7fd65c02a480 comp rx=0 tx=0).stop 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 -- 172.21.15.137:0/969983278 shutdown_connections 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 --2- 172.21.15.137:0/969983278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660102ef0 0x7fd66010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 --2- 172.21.15.137:0/969983278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd660102570 0x7fd6601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 --2- 172.21.15.137:0/969983278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660100e20 0x7fd660101200 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 -- 172.21.15.137:0/969983278 >> 172.21.15.137:0/969983278 conn(0x7fd6600fc7d0 msgr2=0x7fd6600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 -- 172.21.15.137:0/969983278 shutdown_connections 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.856+0000 7fd6678b9640 1 -- 172.21.15.137:0/969983278 wait complete. 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 Processor -- start 2024-09-12T12:19:01.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 -- start start 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660100e20 0x7fd6601984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660102570 0x7fd660198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd660102ef0 0x7fd66019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd660110f80 con 0x7fd660102570 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd660110e00 con 0x7fd660102ef0 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd6678b9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd660111100 con 0x7fd660100e20 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd664e54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660102570 0x7fd660198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd665655640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660100e20 0x7fd6601984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd664e54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660102570 0x7fd660198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38494/0 (socket says 172.21.15.137:38494) 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd665655640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660100e20 0x7fd6601984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:36330/0 (socket says 172.21.15.137:36330) 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd664e54640 1 -- 172.21.15.137:0/4215305649 learned_addr learned my addr 172.21.15.137:0/4215305649 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:01.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.857+0000 7fd665e56640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd660102ef0 0x7fd66019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd665e56640 1 -- 172.21.15.137:0/4215305649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660100e20 msgr2=0x7fd6601984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd665e56640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660100e20 0x7fd6601984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd665e56640 1 -- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660102570 msgr2=0x7fd660198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd665e56640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660102570 0x7fd660198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd665e56640 1 -- 172.21.15.137:0/4215305649 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66019d4a0 con 0x7fd660102ef0 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd665655640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660100e20 0x7fd6601984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd665e56640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd660102ef0 0x7fd66019cdc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd658009fd0 tx=0x7fd65800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd6567fc640 1 -- 172.21.15.137:0/4215305649 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65800ce20 con 0x7fd660102ef0 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66019d790 con 0x7fd660102ef0 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd6567fc640 1 -- 172.21.15.137:0/4215305649 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd658005020 con 0x7fd660102ef0 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd6567fc640 1 -- 172.21.15.137:0/4215305649 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658010720 con 0x7fd660102ef0 2024-09-12T12:19:01.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.858+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6601a5070 con 0x7fd660102ef0 2024-09-12T12:19:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.859+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd660101200 con 0x7fd660102ef0 2024-09-12T12:19:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.861+0000 7fd6567fc640 1 -- 172.21.15.137:0/4215305649 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd658005540 con 0x7fd660102ef0 2024-09-12T12:19:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.861+0000 7fd6567fc640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd64c0a2cf0 0x7fd64c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.861+0000 7fd665655640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd64c0a2cf0 0x7fd64c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.861+0000 7fd6567fc640 1 -- 172.21.15.137:0/4215305649 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd6580cfad0 con 0x7fd660102ef0 2024-09-12T12:19:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.862+0000 7fd665655640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd64c0a2cf0 0x7fd64c0a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009230 tx=0x7fd65c0058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:01.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:01.867+0000 7fd6567fc640 1 -- 172.21.15.137:0/4215305649 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd65809c6e0 con 0x7fd660102ef0 2024-09-12T12:19:02.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.081+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6601a5240 con 0x7fd660102ef0 2024-09-12T12:19:02.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.082+0000 7fd6567fc640 1 -- 172.21.15.137:0/4215305649 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fd65809bfe0 con 0x7fd660102ef0 2024-09-12T12:19:02.083 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:19:02.083 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":29,"btime":"2024-09-12T12:17:07:265882+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:17:06.970365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:19:02.083 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 29 2024-09-12T12:19:02.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd64c0a2cf0 msgr2=0x7fd64c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:02.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd64c0a2cf0 0x7fd64c0a51b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009230 tx=0x7fd65c0058d0 comp rx=0 tx=0).stop 2024-09-12T12:19:02.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd660102ef0 msgr2=0x7fd66019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:02.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd660102ef0 0x7fd66019cdc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd658009fd0 tx=0x7fd65800ef10 comp rx=0 tx=0).stop 2024-09-12T12:19:02.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 shutdown_connections 2024-09-12T12:19:02.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd64c0a2cf0 0x7fd64c0a51b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:02.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd660102ef0 0x7fd66019cdc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:02.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd660102570 0x7fd660198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:02.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 --2- 172.21.15.137:0/4215305649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd660100e20 0x7fd6601984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:02.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 >> 172.21.15.137:0/4215305649 conn(0x7fd6600fc7d0 msgr2=0x7fd66006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:02.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 shutdown_connections 2024-09-12T12:19:02.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:02.084+0000 7fd6678b9640 1 -- 172.21.15.137:0/4215305649 wait complete. 2024-09-12T12:19:02.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 479810 9.37 MB/sec execute 157 sec latency 32.285 ms 2024-09-12T12:19:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:02 smithi137 ceph-mon[30950]: pgmap v178: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-12T12:19:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:02 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4215305649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:19:02.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:02 smithi189 ceph-mon[35475]: pgmap v178: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-12T12:19:02.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:02 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4215305649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:19:02.940 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-12T12:19:02.940 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-12T12:19:02.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:02 smithi171 ceph-mon[35098]: pgmap v178: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-12T12:19:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:02 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4215305649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:19:03.225 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:19:03.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481174 9.35 MB/sec execute 158 sec latency 38.803 ms 2024-09-12T12:19:03.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.758+0000 7fbdb0355640 1 -- 172.21.15.137:0/1461186050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8102570 msgr2=0x7fbda81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.758+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1461186050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8102570 0x7fbda81029b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009900 tx=0x7fbd9802a480 comp rx=0 tx=0).stop 2024-09-12T12:19:03.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- 172.21.15.137:0/1461186050 shutdown_connections 2024-09-12T12:19:03.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1461186050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8102ef0 0x7fbda810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1461186050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8102570 0x7fbda81029b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1461186050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbda8100e20 0x7fbda8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- 172.21.15.137:0/1461186050 >> 172.21.15.137:0/1461186050 conn(0x7fbda80fc7d0 msgr2=0x7fbda80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- 172.21.15.137:0/1461186050 shutdown_connections 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- 172.21.15.137:0/1461186050 wait complete. 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 Processor -- start 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- start start 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8100e20 0x7fbda81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbda8102570 0x7fbda8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8102ef0 0x7fbda819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8110f80 con 0x7fbda8100e20 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8110e00 con 0x7fbda8102ef0 2024-09-12T12:19:03.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.759+0000 7fbdb0355640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbda8111100 con 0x7fbda8102570 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8100e20 0x7fbda81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8100e20 0x7fbda81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38518/0 (socket says 172.21.15.137:38518) 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 -- 172.21.15.137:0/1018614007 learned_addr learned my addr 172.21.15.137:0/1018614007 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbda8102570 0x7fbda8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae8f2640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8102ef0 0x7fbda819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 -- 172.21.15.137:0/1018614007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbda8102570 msgr2=0x7fbda8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbda8102570 0x7fbda8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 -- 172.21.15.137:0/1018614007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8102ef0 msgr2=0x7fbda819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8102ef0 0x7fbda819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 -- 172.21.15.137:0/1018614007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda819d480 con 0x7fbda8100e20 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbda8102570 0x7fbda8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8100e20 0x7fbda81984d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbda400e9f0 tx=0x7fbda400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:03.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbd9f7fe640 1 -- 172.21.15.137:0/1018614007 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda400cd80 con 0x7fbda8100e20 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda819d770 con 0x7fbda8100e20 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbd9f7fe640 1 -- 172.21.15.137:0/1018614007 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda4005500 con 0x7fbda8100e20 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbd9f7fe640 1 -- 172.21.15.137:0/1018614007 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4010690 con 0x7fbda8100e20 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.760+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a5050 con 0x7fbda8100e20 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbd9f7fe640 1 -- 172.21.15.137:0/1018614007 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbda40046c0 con 0x7fbda8100e20 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 Processor -- start 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 -- start start 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbd9f7fe640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbd940a2cf0 0x7fbd940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a299b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564c1a1ff2a0 con 0x564c1a2a4940 2024-09-12T12:19:03.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564c1a1a6b10 con 0x564c1a299760 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.763+0000 7fbdb272e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564c1a1fb260 con 0x564c1a29a080 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a299b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae8f2640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbd940a2cf0 0x7fbd940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a299b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:35182/0 (socket says 172.21.15.137:35182) 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 -- 172.21.15.137:0/249889467 learned_addr learned my addr 172.21.15.137:0/249889467 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbd9f7fe640 1 -- 172.21.15.137:0/1018614007 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fbda40cf960 con 0x7fbda8100e20 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae8f2640 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 -- 172.21.15.137:0/249889467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 msgr2=0x564c1a2a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 -- 172.21.15.137:0/249889467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 msgr2=0x564c1a2a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 -- 172.21.15.137:0/249889467 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c19f70b90 con 0x564c1a299760 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae8f2640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbd940a2cf0 0x7fbd940a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbda0005fd0 tx=0x7fbda0012a30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a299b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbda400e9c0 tx=0x7fbda4004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=918422fd2e1bf4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbd9cdea640 1 -- 172.21.15.137:0/249889467 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda409b830 con 0x564c1a299760 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbd9cdea640 1 -- 172.21.15.137:0/249889467 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda409c5d0 con 0x564c1a299760 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.764+0000 7fbd9cdea640 1 -- 172.21.15.137:0/249889467 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4008220 con 0x564c1a299760 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 -- 172.21.15.137:0/249889467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 msgr2=0x564c1a299b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a299b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbda400e9c0 tx=0x7fbda4004b60 comp rx=0 tx=0).stop 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 -- 172.21.15.137:0/249889467 shutdown_connections 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 --2- 172.21.15.137:0/249889467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a299b40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 -- 172.21.15.137:0/249889467 >> 172.21.15.137:0/249889467 conn(0x564c1a295cf0 msgr2=0x564c1a2960d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:03.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 -- 172.21.15.137:0/249889467 shutdown_connections 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 -- 172.21.15.137:0/249889467 wait complete. 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.765+0000 7fbdb272e740 1 -- start start 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 1 Processor -- start 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2a1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564c1a2a7510 con 0x564c1a2a4940 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564c1a1876e0 con 0x564c1a299760 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564c1a2a7690 con 0x564c1a29a080 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdb272e740 10 client.0 _ll_register_callbacks cb 0x564c1a1aba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdae8f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdad8f0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:35190/0 (socket says 172.21.15.137:35190) 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdae8f2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38540/0 (socket says 172.21.15.137:38540) 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdae0f1640 1 -- 172.21.15.137:0/3537347123 learned_addr learned my addr 172.21.15.137:0/3537347123 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdad8f0640 1 -- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 msgr2=0x564c1a2a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdad8f0640 1 -- 172.21.15.137:0/3537347123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 msgr2=0x564c1a2a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdad8f0640 1 -- 172.21.15.137:0/3537347123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c1a2b0a50 con 0x564c1a29a080 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdae8f2640 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.766+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a1dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd980052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641c471f68931ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbd91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:19:03.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbd91ffb640 1 -- 172.21.15.137:0/3537347123 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98035030 con 0x564c1a29a080 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbd91ffb640 1 -- 172.21.15.137:0/3537347123 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fbd980048c0 con 0x564c1a29a080 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda409c5d0 con 0x564c1a29a080 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbd91ffb640 1 -- 172.21.15.137:0/3537347123 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd98005d50 con 0x564c1a29a080 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 10 client.25591 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c1a2b33e0 con 0x564c1a29a080 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbd91ffb640 1 -- 172.21.15.137:0/3537347123 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda409c5d0 con 0x564c1a29a080 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbd91ffb640 1 -- 172.21.15.137:0/3537347123 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbd98031540 con 0x564c1a29a080 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 10 client.25591 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 10 client.25591 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 20 client.25591 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a2b47f0 0x564c1a2b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 4 client.25591 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:19:03.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.767+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c1a20c9d0 con 0x564c1a2b47f0 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.768+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a2b47f0 0x564c1a2b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.768+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a2b47f0 0x564c1a2b6be0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23916845b0645c5f server_cookie=12ad35fc08b1e3a7 in_seq=0 out_seq=0 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.768+0000 7fbd91ffb640 10 client.25591 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbd91ffb640 1 -- 172.21.15.137:0/3537347123 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c1a20c9d0 con 0x564c1a2b47f0 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbd91ffb640 10 client.25591 handle_command_reply: tid=0 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 client.25591 shutdown 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 msgr2=0x564c1a2a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a1dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009230 tx=0x7fbd980052f0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 shutdown_connections 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a2b47f0 0x564c1a2b6be0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2a1dc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 --2- 172.21.15.137:0/3537347123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbd91ffb640 0 client.25591 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:03.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 >> 172.21.15.137:0/3537347123 conn(0x564c1a295cf0 msgr2=0x564c1a29ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:03.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 shutdown_connections 2024-09-12T12:19:03.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 1 -- 172.21.15.137:0/3537347123 wait complete. 2024-09-12T12:19:03.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.769+0000 7fbdb272e740 20 client.25591 trim_cache size 0 max 16384 2024-09-12T12:19:03.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 Processor -- start 2024-09-12T12:19:03.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 -- start start 2024-09-12T12:19:03.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a299760 0x564c1a29c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a29a080 0x564c1a2bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a2a4940 0x564c1a2bc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81affe0 con 0x564c1a299760 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ad8b0 con 0x564c1a29a080 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdb272e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ab700 con 0x564c1a2a4940 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdad8f0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a29a080 0x564c1a2bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a299760 0x564c1a29c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdad8f0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a29a080 0x564c1a2bc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:35196/0 (socket says 172.21.15.137:35196) 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdad8f0640 1 -- 172.21.15.137:0/2793557492 learned_addr learned my addr 172.21.15.137:0/2793557492 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdae8f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a2a4940 0x564c1a2bc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.786+0000 7fbdad8f0640 1 -- 172.21.15.137:0/2793557492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a2a4940 msgr2=0x564c1a2bc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/2793557492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a2a4940 0x564c1a2bc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbdad8f0640 1 -- 172.21.15.137:0/2793557492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a299760 msgr2=0x564c1a29c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/2793557492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a299760 0x564c1a29c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbdad8f0640 1 -- 172.21.15.137:0/2793557492 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c1a291260 con 0x564c1a29a080 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/2793557492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a29a080 0x564c1a2bc4b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd98006010 tx=0x7fbd9802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355ea529dbe0f62f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbd93fff640 1 -- 172.21.15.137:0/2793557492 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98035070 con 0x564c1a29a080 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbd93fff640 1 -- 172.21.15.137:0/2793557492 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd98053070 con 0x564c1a29a080 2024-09-12T12:19:03.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.787+0000 7fbd93fff640 1 -- 172.21.15.137:0/2793557492 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd980035f0 con 0x564c1a29a080 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 -- 172.21.15.137:0/2793557492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a29a080 msgr2=0x564c1a2bc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 --2- 172.21.15.137:0/2793557492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a29a080 0x564c1a2bc4b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbd98006010 tx=0x7fbd9802a480 comp rx=0 tx=0).stop 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 -- 172.21.15.137:0/2793557492 shutdown_connections 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 --2- 172.21.15.137:0/2793557492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a2a4940 0x564c1a2bc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 --2- 172.21.15.137:0/2793557492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a29a080 0x564c1a2bc4b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 --2- 172.21.15.137:0/2793557492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a299760 0x564c1a29c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 -- 172.21.15.137:0/2793557492 >> 172.21.15.137:0/2793557492 conn(0x564c1a295cf0 msgr2=0x564c1a2a30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 -- 172.21.15.137:0/2793557492 shutdown_connections 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 -- 172.21.15.137:0/2793557492 wait complete. 2024-09-12T12:19:03.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 -- start start 2024-09-12T12:19:03.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.788+0000 7fbdb272e740 1 Processor -- start 2024-09-12T12:19:03.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdb272e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdae0f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:35210/0 (socket says 172.21.15.137:35210) 2024-09-12T12:19:03.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81abd70 con 0x564c1a2a4940 2024-09-12T12:19:03.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdae0f1640 1 -- 172.21.15.137:0/599957592 learned_addr learned my addr 172.21.15.137:0/599957592 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ad1e0 con 0x564c1a299760 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbda81ac440 con 0x564c1a29a080 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdb272e740 10 client.0 _ll_register_callbacks cb 0x7fbd980044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdae0f1640 1 -- 172.21.15.137:0/599957592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 msgr2=0x564c1a2ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdae0f1640 1 -- 172.21.15.137:0/599957592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 msgr2=0x564c1a2a98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.789+0000 7fbdae8f2640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbdae0f1640 1 -- 172.21.15.137:0/599957592 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c1a2a9e20 con 0x564c1a299760 2024-09-12T12:19:03.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbdae8f2640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbdae0f1640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2ac8e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbda409be60 tx=0x7fbda409bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f672b72470e46cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbd91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbd91ffb640 1 -- 172.21.15.137:0/599957592 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4066060 con 0x564c1a299760 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbd91ffb640 1 -- 172.21.15.137:0/599957592 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fbda40a1650 con 0x564c1a299760 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd98053070 con 0x564c1a299760 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbd91ffb640 1 -- 172.21.15.137:0/599957592 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbda40b7070 con 0x564c1a299760 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbdb272e740 10 client.25535 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c1a324140 con 0x564c1a299760 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.790+0000 7fbd91ffb640 1 -- 172.21.15.137:0/599957592 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd98053070 con 0x564c1a299760 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbd91ffb640 1 -- 172.21.15.137:0/599957592 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fbda4098540 con 0x564c1a299760 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdb272e740 10 client.25535 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:03.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdb272e740 10 client.25535 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:19:03.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdb272e740 20 client.25535 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:03.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdb272e740 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a324480 0x564c1a3268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:03.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdb272e740 4 client.25535 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-12T12:19:03.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x564c1a206d20 con 0x564c1a324480 2024-09-12T12:19:03.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a324480 0x564c1a3268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:03.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbd91ffb640 10 client.25535 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:03.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.791+0000 7fbdad8f0640 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a324480 0x564c1a3268a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d687a2076b24eb6 server_cookie=7ed4af78ff476e51 in_seq=0 out_seq=0 2024-09-12T12:19:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.893+0000 7fbd91ffb640 1 -- 172.21.15.137:0/599957592 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564c1a206d20 con 0x564c1a324480 2024-09-12T12:19:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbd91ffb640 10 client.25535 handle_command_reply: tid=0 2024-09-12T12:19:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 client.25535 shutdown 2024-09-12T12:19:03.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 msgr2=0x564c1a2ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2ac8e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbda409be60 tx=0x7fbda409bda0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 shutdown_connections 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564c1a324480 0x564c1a3268a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564c1a2a4940 0x564c1a2a98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564c1a29a080 0x564c1a2ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbd91ffb640 0 client.25535 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 --2- 172.21.15.137:0/599957592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564c1a299760 0x564c1a2ac8e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 >> 172.21.15.137:0/599957592 conn(0x564c1a295cf0 msgr2=0x564c1a294650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:03.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 shutdown_connections 2024-09-12T12:19:03.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 1 -- 172.21.15.137:0/599957592 wait complete. 2024-09-12T12:19:03.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.894+0000 7fbdb272e740 20 client.25535 trim_cache size 0 max 16384 2024-09-12T12:19:03.897 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:19:03.897 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 29, 2024-09-12T12:19:03.897 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:19:03.897 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 22, 2024-09-12T12:19:03.897 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:19:03.898 INFO:teuthology.orchestra.run.smithi137.stdout: "7de7e21a": { 2024-09-12T12:19:03.898 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 22, 2024-09-12T12:19:03.898 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:19:03.898 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:19:03.899 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:19:03.899 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:19:03.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:19:03.899 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:19:03.900 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 140.1, 2024-09-12T12:19:03.900 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:19:03.900 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:19:03.900 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:19:03.901 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:19:03.901 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:19:03.901 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:19:03.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:03.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:03.902 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:03.902 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:03.902 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:03.902 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:19:03.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbd940a2cf0 msgr2=0x7fbd940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbd940a2cf0 0x7fbd940a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbda0005fd0 tx=0x7fbda0012a30 comp rx=0 tx=0).stop 2024-09-12T12:19:03.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8100e20 msgr2=0x7fbda81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:03.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8100e20 0x7fbda81984d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbda400e9f0 tx=0x7fbda400edf0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 shutdown_connections 2024-09-12T12:19:03.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbd940a2cf0 0x7fbd940a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8102ef0 0x7fbda819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbda8102570 0x7fbda8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 --2- 172.21.15.137:0/1018614007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbda8100e20 0x7fbda81984d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:03.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 >> 172.21.15.137:0/1018614007 conn(0x7fbda80fc7d0 msgr2=0x7fbda806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:03.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 shutdown_connections 2024-09-12T12:19:03.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:03.895+0000 7fbdb0355640 1 -- 172.21.15.137:0/1018614007 wait complete. 2024-09-12T12:19:04.609 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7de7e21a', quiesce took 3.5 seconds. Will release after: 16.6 2024-09-12T12:19:04.731 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 9.29 MB/sec execute 159 sec latency 938.853 ms 2024-09-12T12:19:04.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:04 smithi189 ceph-mon[35475]: pgmap v179: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-12T12:19:04.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:04 smithi171 ceph-mon[35098]: pgmap v179: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-12T12:19:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:04 smithi137 ceph-mon[30950]: pgmap v179: 97 pgs: 97 active+clean; 124 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 315 op/s 2024-09-12T12:19:05.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 9.24 MB/sec execute 160 sec latency 1938.972 ms 2024-09-12T12:19:06.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 9.18 MB/sec execute 161 sec latency 2939.010 ms 2024-09-12T12:19:06.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:06 smithi189 ceph-mon[35475]: pgmap v180: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 461 op/s 2024-09-12T12:19:06.852 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:06 smithi137 ceph-mon[30950]: pgmap v180: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 461 op/s 2024-09-12T12:19:06.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:06 smithi171 ceph-mon[35098]: pgmap v180: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 461 op/s 2024-09-12T12:19:07.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 9.12 MB/sec execute 162 sec latency 3939.101 ms 2024-09-12T12:19:08.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 9.07 MB/sec execute 163 sec latency 4939.150 ms 2024-09-12T12:19:08.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:08 smithi189 ceph-mon[35475]: pgmap v181: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-09-12T12:19:08.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:08 smithi171 ceph-mon[35098]: pgmap v181: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-09-12T12:19:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:08 smithi137 ceph-mon[30950]: pgmap v181: 97 pgs: 97 active+clean; 136 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-09-12T12:19:09.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 9.01 MB/sec execute 164 sec latency 5939.303 ms 2024-09-12T12:19:10.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.96 MB/sec execute 165 sec latency 6939.391 ms 2024-09-12T12:19:10.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:10 smithi189 ceph-mon[35475]: pgmap v182: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 359 op/s 2024-09-12T12:19:10.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:10 smithi171 ceph-mon[35098]: pgmap v182: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 359 op/s 2024-09-12T12:19:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:10 smithi137 ceph-mon[30950]: pgmap v182: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 359 op/s 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 -- 172.21.15.137:0/4236133564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81069e0 msgr2=0x7f96e8106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 --2- 172.21.15.137:0/4236133564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81069e0 0x7f96e8106dc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009930 tx=0x7f96d802a420 comp rx=0 tx=0).stop 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 -- 172.21.15.137:0/4236133564 shutdown_connections 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 --2- 172.21.15.137:0/4236133564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101370 0x7f96e810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 --2- 172.21.15.137:0/4236133564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96e81009d0 0x7f96e8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 --2- 172.21.15.137:0/4236133564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81069e0 0x7f96e8106dc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 -- 172.21.15.137:0/4236133564 >> 172.21.15.137:0/4236133564 conn(0x7f96e80fc6b0 msgr2=0x7f96e80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 -- 172.21.15.137:0/4236133564 shutdown_connections 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.364+0000 7f96ed88d640 1 -- 172.21.15.137:0/4236133564 wait complete. 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.365+0000 7f96ed88d640 1 Processor -- start 2024-09-12T12:19:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.365+0000 7f96ed88d640 1 -- start start 2024-09-12T12:19:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.365+0000 7f96ed88d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81009d0 0x7f96e80fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.367 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ed88d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101370 0x7f96e80fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ed88d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96e81069e0 0x7f96e80fd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ed88d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8110e20 con 0x7f96e81009d0 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ed88d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8110ca0 con 0x7f96e8101370 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ed88d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8110fa0 con 0x7f96e81069e0 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81009d0 0x7f96e80fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81009d0 0x7f96e80fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58176/0 (socket says 172.21.15.137:58176) 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 -- 172.21.15.137:0/2476767141 learned_addr learned my addr 172.21.15.137:0/2476767141 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e67fc640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101370 0x7f96e80fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e77fe640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96e81069e0 0x7f96e80fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 -- 172.21.15.137:0/2476767141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96e81069e0 msgr2=0x7f96e80fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96e81069e0 0x7f96e80fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 -- 172.21.15.137:0/2476767141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101370 msgr2=0x7f96e80fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101370 0x7f96e80fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 -- 172.21.15.137:0/2476767141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e81a6920 con 0x7f96e81009d0 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e77fe640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96e81069e0 0x7f96e80fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e67fc640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101370 0x7f96e80fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81009d0 0x7f96e80fcca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009230 tx=0x7f96d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ec88b640 1 -- 172.21.15.137:0/2476767141 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d803f030 con 0x7f96e81009d0 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ec88b640 1 -- 172.21.15.137:0/2476767141 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d8003ad0 con 0x7f96e81009d0 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e81a6bb0 con 0x7f96e81009d0 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ec88b640 1 -- 172.21.15.137:0/2476767141 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8035070 con 0x7f96e81009d0 2024-09-12T12:19:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.366+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a70a0 con 0x7f96e81009d0 2024-09-12T12:19:11.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96ec88b640 1 -- 172.21.15.137:0/2476767141 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f96d8042030 con 0x7f96e81009d0 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 Processor -- start 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 -- start start 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2facaf0 0x557ef2faced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557ef2f112f0 con 0x557ef2fb7cb0 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557ef2eb9270 con 0x557ef2fad410 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96f0628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557ef2f0d2b0 con 0x557ef2facaf0 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96e67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96e67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58184/0 (socket says 172.21.15.137:58184) 2024-09-12T12:19:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96e6ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96e67fc640 1 -- 172.21.15.137:0/213041423 learned_addr learned my addr 172.21.15.137:0/213041423 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.369+0000 7f96e77fe640 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2facaf0 0x557ef2faced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96ec88b640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f96d00a2da0 0x7f96d00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e67fc640 1 -- 172.21.15.137:0/213041423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2facaf0 msgr2=0x557ef2faced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e67fc640 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2facaf0 0x557ef2faced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e67fc640 1 -- 172.21.15.137:0/213041423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 msgr2=0x557ef2fb7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e67fc640 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e67fc640 1 -- 172.21.15.137:0/213041423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef2eb8340 con 0x557ef2fb7cb0 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e77fe640 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2facaf0 0x557ef2faced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e67fc640 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fba0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00bd30 tx=0x7f96dc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2911cb58fa340027 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96ec88b640 1 -- 172.21.15.137:0/2476767141 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f96d80ef9b0 con 0x7f96e81009d0 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96e77fe640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f96d00a2da0 0x7f96d00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96cdffb640 1 -- 172.21.15.137:0/213041423 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc015070 con 0x557ef2fb7cb0 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96cdffb640 1 -- 172.21.15.137:0/213041423 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96dc00be20 con 0x557ef2fb7cb0 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.370+0000 7f96cdffb640 1 -- 172.21.15.137:0/213041423 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0119b0 con 0x557ef2fb7cb0 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 -- 172.21.15.137:0/213041423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 msgr2=0x557ef2fba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96e77fe640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f96d00a2da0 0x7f96d00a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96e0005fd0 tx=0x7f96e0004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fba0a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00bd30 tx=0x7f96dc00f730 comp rx=0 tx=0).stop 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 -- 172.21.15.137:0/213041423 shutdown_connections 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fba0a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 --2- 172.21.15.137:0/213041423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2facaf0 0x557ef2faced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 -- 172.21.15.137:0/213041423 >> 172.21.15.137:0/213041423 conn(0x557ef2fa9060 msgr2=0x557ef2fa9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 -- 172.21.15.137:0/213041423 shutdown_connections 2024-09-12T12:19:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 -- 172.21.15.137:0/213041423 wait complete. 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 -- start start 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.371+0000 7f96f0628740 1 Processor -- start 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2facaf0 0x557ef2fb4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 0x557ef2fb5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557ef2fba880 con 0x557ef2facaf0 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557ef2e9a290 con 0x557ef2fad410 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557ef2fbaa00 con 0x557ef2fb7cb0 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 10 client.0 _ll_register_callbacks cb 0x557ef2ecb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2facaf0 0x557ef2fb4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2facaf0 0x557ef2fb4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58196/0 (socket says 172.21.15.137:58196) 2024-09-12T12:19:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 -- 172.21.15.137:0/1096827581 learned_addr learned my addr 172.21.15.137:0/1096827581 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e77fe640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 0x557ef2fb5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e67fc640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 -- 172.21.15.137:0/1096827581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 msgr2=0x557ef2fb5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 0x557ef2fb5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 -- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 msgr2=0x557ef2fb50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 -- 172.21.15.137:0/1096827581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef2c24210 con 0x557ef2facaf0 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e67fc640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2facaf0 0x557ef2fb4b90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009a50 tx=0x7f96d80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8155403602b81a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96c6ffd640 1 -- 172.21.15.137:0/1096827581 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d80efe60 con 0x557ef2facaf0 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96c6ffd640 1 -- 172.21.15.137:0/1096827581 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f96d80b8590 con 0x557ef2facaf0 2024-09-12T12:19:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.372+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96dc00be20 con 0x557ef2facaf0 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96c6ffd640 1 -- 172.21.15.137:0/1096827581 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96d8049070 con 0x557ef2facaf0 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 10 client.16107 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ef2cf0620 con 0x557ef2facaf0 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96c6ffd640 1 -- 172.21.15.137:0/1096827581 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96dc00be20 con 0x557ef2facaf0 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96c6ffd640 1 -- 172.21.15.137:0/1096827581 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f96d8003c80 con 0x557ef2facaf0 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 20 client.16107 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef2fc7b70 0x557ef2fc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 4 client.16107 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ef2f1ea20 con 0x557ef2fc7b70 2024-09-12T12:19:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.373+0000 7f96e67fc640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef2fc7b70 0x557ef2fc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.374+0000 7f96c6ffd640 10 client.16107 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.374+0000 7f96e67fc640 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef2fc7b70 0x557ef2fc9fa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41141eb19b86fd29 server_cookie=494dbca1f800be9e in_seq=0 out_seq=0 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.374+0000 7f96c6ffd640 1 -- 172.21.15.137:0/1096827581 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ef2f1ea20 con 0x557ef2fc7b70 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.374+0000 7f96c6ffd640 10 client.16107 handle_command_reply: tid=0 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.374+0000 7f96f0628740 1 client.16107 shutdown 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2facaf0 msgr2=0x557ef2fb4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2facaf0 0x557ef2fb4b90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009a50 tx=0x7f96d80058d0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 shutdown_connections 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 0x557ef2fb5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2fad410 0x557ef2fb50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef2fc7b70 0x557ef2fc9fa0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 --2- 172.21.15.137:0/1096827581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2facaf0 0x557ef2fb4b90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96c6ffd640 0 client.16107 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 >> 172.21.15.137:0/1096827581 conn(0x557ef2fa9060 msgr2=0x557ef2faed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 shutdown_connections 2024-09-12T12:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 1 -- 172.21.15.137:0/1096827581 wait complete. 2024-09-12T12:19:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.375+0000 7f96f0628740 20 client.16107 trim_cache size 0 max 16384 2024-09-12T12:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.389+0000 7f96f0628740 1 Processor -- start 2024-09-12T12:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.389+0000 7f96f0628740 1 -- start start 2024-09-12T12:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96f0628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fb5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96f0628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fad410 0x557ef2fcf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96f0628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 0x557ef2fcfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96f0628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81b21a0 con 0x557ef2fad410 2024-09-12T12:19:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96f0628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81afa70 con 0x557ef2facaf0 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96f0628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81ad8e0 con 0x557ef2fb7cb0 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fad410 0x557ef2fcf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fad410 0x557ef2fcf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58200/0 (socket says 172.21.15.137:58200) 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 -- 172.21.15.137:0/3258745458 learned_addr learned my addr 172.21.15.137:0/3258745458 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 -- 172.21.15.137:0/3258745458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 msgr2=0x557ef2fcfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 0x557ef2fcfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fb5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 -- 172.21.15.137:0/3258745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 msgr2=0x557ef2fb5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fb5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 -- 172.21.15.137:0/3258745458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef2fa8e90 con 0x557ef2fad410 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fb5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96e67fc640 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fad410 0x557ef2fcf8b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x557ef2faf690 tx=0x7f96dc0115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ca3d47d3d45780 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:11.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96cd7fa640 1 -- 172.21.15.137:0/3258745458 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc015050 con 0x557ef2fad410 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96cd7fa640 1 -- 172.21.15.137:0/3258745458 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96dc00be20 con 0x557ef2fad410 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.390+0000 7f96cd7fa640 1 -- 172.21.15.137:0/3258745458 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc004680 con 0x557ef2fad410 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 -- 172.21.15.137:0/3258745458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fad410 msgr2=0x557ef2fcf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fad410 0x557ef2fcf8b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x557ef2faf690 tx=0x7f96dc0115f0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 -- 172.21.15.137:0/3258745458 shutdown_connections 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fb7cb0 0x557ef2fcfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fad410 0x557ef2fcf8b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 --2- 172.21.15.137:0/3258745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fb5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 -- 172.21.15.137:0/3258745458 >> 172.21.15.137:0/3258745458 conn(0x557ef2fa9060 msgr2=0x557ef2fb6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 -- 172.21.15.137:0/3258745458 shutdown_connections 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 -- 172.21.15.137:0/3258745458 wait complete. 2024-09-12T12:19:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.391+0000 7f96f0628740 1 -- start start 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 1 Processor -- start 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fbba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fad410 0x557ef2fbbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fbc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81adf30 con 0x557ef2fb7cb0 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81af3a0 con 0x557ef2facaf0 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81ae600 con 0x557ef2fad410 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96f0628740 10 client.0 _ll_register_callbacks cb 0x7f96d8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fbc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fbc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58210/0 (socket says 172.21.15.137:58210) 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 -- 172.21.15.137:0/3171565247 learned_addr learned my addr 172.21.15.137:0/3171565247 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:11.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fbba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e67fc640 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fad410 0x557ef2fbbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 -- 172.21.15.137:0/3171565247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fad410 msgr2=0x557ef2fbbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fad410 0x557ef2fbbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 -- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 msgr2=0x557ef2fbba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fbba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 -- 172.21.15.137:0/3171565247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef2fa42d0 con 0x557ef2fb7cb0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96e77fe640 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fbc480 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f96e001d9d0 tx=0x7f96e0009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42a3496ed21f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96c6ffd640 1 -- 172.21.15.137:0/3171565247 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e001cb90 con 0x557ef2fb7cb0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.392+0000 7f96c6ffd640 1 -- 172.21.15.137:0/3171565247 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f96e001cd30 con 0x557ef2fb7cb0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96dc00be20 con 0x557ef2fb7cb0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96c6ffd640 1 -- 172.21.15.137:0/3171565247 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f96e001a460 con 0x557ef2fb7cb0 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 10 client.16113 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ef3032980 con 0x557ef2fb7cb0 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96c6ffd640 1 -- 172.21.15.137:0/3171565247 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96dc00be20 con 0x557ef2fb7cb0 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96c6ffd640 1 -- 172.21.15.137:0/3171565247 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f96e0027820 con 0x557ef2fb7cb0 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 10 client.16113 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 20 client.16113 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef3033150 0x557ef3035570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 4 client.16113 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ef2f18d70 con 0x557ef3033150 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.393+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef3033150 0x557ef3035570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.394+0000 7f96c6ffd640 10 client.16113 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.394+0000 7f96e6ffd640 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef3033150 0x557ef3035570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc205b50f07ebe server_cookie=ae42949a714cc419 in_seq=0 out_seq=0 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.440+0000 7f96c6ffd640 1 -- 172.21.15.137:0/3171565247 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ef2f18d70 con 0x557ef3033150 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.440+0000 7f96c6ffd640 10 client.16113 handle_command_reply: tid=0 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 client.16113 shutdown 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 msgr2=0x557ef2fbc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fbc480 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f96e001d9d0 tx=0x7f96e0009590 comp rx=0 tx=0).stop 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 shutdown_connections 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557ef3033150 0x557ef3035570 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef2fb7cb0 0x557ef2fbc480 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96c6ffd640 0 client.16113 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557ef2fad410 0x557ef2fbbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 --2- 172.21.15.137:0/3171565247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557ef2facaf0 0x557ef2fbba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 >> 172.21.15.137:0/3171565247 conn(0x557ef2fa9060 msgr2=0x557ef2fae550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 shutdown_connections 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 1 -- 172.21.15.137:0/3171565247 wait complete. 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.441+0000 7f96f0628740 20 client.16113 trim_cache size 0 max 16384 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:19:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f96d00a2da0 msgr2=0x7f96d00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f96d00a2da0 0x7f96d00a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96e0005fd0 tx=0x7f96e0004a40 comp rx=0 tx=0).stop 2024-09-12T12:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81009d0 msgr2=0x7f96e80fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81009d0 0x7f96e80fcca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009230 tx=0x7f96d8004270 comp rx=0 tx=0).stop 2024-09-12T12:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 shutdown_connections 2024-09-12T12:19:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f96d00a2da0 0x7f96d00a5260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96e81069e0 0x7f96e80fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e8101370 0x7f96e80fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 --2- 172.21.15.137:0/2476767141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e81009d0 0x7f96e80fcca0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 >> 172.21.15.137:0/2476767141 conn(0x7f96e80fc6b0 msgr2=0x7f96e806a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 shutdown_connections 2024-09-12T12:19:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:11.442+0000 7f96ed88d640 1 -- 172.21.15.137:0/2476767141 wait complete. 2024-09-12T12:19:11.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.90 MB/sec execute 166 sec latency 7939.464 ms 2024-09-12T12:19:12.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.85 MB/sec execute 167 sec latency 8939.554 ms 2024-09-12T12:19:12.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:12 smithi189 ceph-mon[35475]: pgmap v183: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-12T12:19:12.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:12 smithi171 ceph-mon[35098]: pgmap v183: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-12T12:19:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:12 smithi137 ceph-mon[30950]: pgmap v183: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-12T12:19:13.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.80 MB/sec execute 168 sec latency 9939.658 ms 2024-09-12T12:19:14.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.74 MB/sec execute 169 sec latency 10939.761 ms 2024-09-12T12:19:14.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:14 smithi189 ceph-mon[35475]: pgmap v184: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-12T12:19:14.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:14 smithi171 ceph-mon[35098]: pgmap v184: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-12T12:19:15.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:14 smithi137 ceph-mon[30950]: pgmap v184: 97 pgs: 97 active+clean; 135 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-09-12T12:19:15.732 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.69 MB/sec execute 170 sec latency 11939.854 ms 2024-09-12T12:19:16.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.64 MB/sec execute 171 sec latency 12939.958 ms 2024-09-12T12:19:16.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:16 smithi189 ceph-mon[35475]: pgmap v185: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-09-12T12:19:16.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:16 smithi137 ceph-mon[30950]: pgmap v185: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-09-12T12:19:16.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:16 smithi171 ceph-mon[35098]: pgmap v185: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 3.7 MiB/s wr, 237 op/s 2024-09-12T12:19:17.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.59 MB/sec execute 172 sec latency 13940.007 ms 2024-09-12T12:19:18.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.54 MB/sec execute 173 sec latency 14940.115 ms 2024-09-12T12:19:18.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:18 smithi189 ceph-mon[35475]: pgmap v186: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-12T12:19:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:18 smithi171 ceph-mon[35098]: pgmap v186: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-12T12:19:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:18 smithi137 ceph-mon[30950]: pgmap v186: 97 pgs: 97 active+clean; 132 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-12T12:19:19.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.49 MB/sec execute 174 sec latency 15940.200 ms 2024-09-12T12:19:20.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.44 MB/sec execute 175 sec latency 16940.297 ms 2024-09-12T12:19:20.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:20 smithi189 ceph-mon[35475]: pgmap v187: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 3.7 MiB/s wr, 120 op/s 2024-09-12T12:19:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:20 smithi171 ceph-mon[35098]: pgmap v187: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 3.7 MiB/s wr, 120 op/s 2024-09-12T12:19:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:20 smithi137 ceph-mon[30950]: pgmap v187: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 3.7 MiB/s wr, 120 op/s 2024-09-12T12:19:21.211 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:19:21.472 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:19:21.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.40 MB/sec execute 176 sec latency 17940.365 ms 2024-09-12T12:19:21.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 -- 172.21.15.137:0/3625132828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 msgr2=0x7fd21010eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 --2- 172.21.15.137:0/3625132828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21010eab0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd208009ca0 tx=0x7fd20802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 -- 172.21.15.137:0/3625132828 shutdown_connections 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 --2- 172.21.15.137:0/3625132828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21010eab0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 --2- 172.21.15.137:0/3625132828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd210100cc0 0x7fd210106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 --2- 172.21.15.137:0/3625132828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 0x7fd2101006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 -- 172.21.15.137:0/3625132828 >> 172.21.15.137:0/3625132828 conn(0x7fd210077210 msgr2=0x7fd2100fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 -- 172.21.15.137:0/3625132828 shutdown_connections 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.960+0000 7fd2180d6640 1 -- 172.21.15.137:0/3625132828 wait complete. 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 Processor -- start 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 -- start start 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 0x7fd2101984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:21.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd210100cc0 0x7fd210198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd210111f90 con 0x7fd210106f30 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd210111e10 con 0x7fd210100310 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd2180d6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd210112110 con 0x7fd210100cc0 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd216673640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd215e72640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 0x7fd2101984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd215671640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd210100cc0 0x7fd210198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd215e72640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 0x7fd2101984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37810/0 (socket says 172.21.15.137:37810) 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd216673640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53504/0 (socket says 172.21.15.137:53504) 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.961+0000 7fd215e72640 1 -- 172.21.15.137:0/4224315165 learned_addr learned my addr 172.21.15.137:0/4224315165 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd215671640 1 -- 172.21.15.137:0/4224315165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 msgr2=0x7fd2101984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:21.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd215671640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 0x7fd2101984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd215671640 1 -- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 msgr2=0x7fd21019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd215671640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd215671640 1 -- 172.21.15.137:0/4224315165 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21019d480 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd215e72640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 0x7fd2101984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd216673640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd215671640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd210100cc0 0x7fd210198a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d720 tx=0x7fd20000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd206ffd640 1 -- 172.21.15.137:0/4224315165 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200017070 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21019d770 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd206ffd640 1 -- 172.21.15.137:0/4224315165 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20000de30 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd206ffd640 1 -- 172.21.15.137:0/4224315165 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2000047a0 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.962+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2101a5050 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.963+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2101006f0 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.964+0000 7fd206ffd640 1 -- 172.21.15.137:0/4224315165 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd2000099d0 con 0x7fd210100cc0 2024-09-12T12:19:21.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.965+0000 7fd206ffd640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1fc0a2cf0 0x7fd1fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:21.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.965+0000 7fd215e72640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1fc0a2cf0 0x7fd1fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:21.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.965+0000 7fd206ffd640 1 -- 172.21.15.137:0/4224315165 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd2000cf3b0 con 0x7fd210100cc0 2024-09-12T12:19:21.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.966+0000 7fd215e72640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1fc0a2cf0 0x7fd1fc0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005fd0 tx=0x7fd20c00b4a0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:21.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:21.972+0000 7fd206ffd640 1 -- 172.21.15.137:0/4224315165 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd200020020 con 0x7fd210100cc0 2024-09-12T12:19:22.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.184+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd21010a110 con 0x7fd210100cc0 2024-09-12T12:19:22.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.185+0000 7fd206ffd640 1 -- 172.21.15.137:0/4224315165 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fd20009c020 con 0x7fd210100cc0 2024-09-12T12:19:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:19:22.186 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":29,"btime":"2024-09-12T12:17:07:265882+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:17:06.970365+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:19:22.186 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 29 2024-09-12T12:19:22.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1fc0a2cf0 msgr2=0x7fd1fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:22.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1fc0a2cf0 0x7fd1fc0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd20c005fd0 tx=0x7fd20c00b4a0 comp rx=0 tx=0).stop 2024-09-12T12:19:22.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd210100cc0 msgr2=0x7fd210198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:22.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd210100cc0 0x7fd210198a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd20000d720 tx=0x7fd20000dbe0 comp rx=0 tx=0).stop 2024-09-12T12:19:22.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 shutdown_connections 2024-09-12T12:19:22.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1fc0a2cf0 0x7fd1fc0a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:22.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd210106f30 0x7fd21019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:22.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd210100cc0 0x7fd210198a10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:22.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 --2- 172.21.15.137:0/4224315165 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd210100310 0x7fd2101984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:22.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 >> 172.21.15.137:0/4224315165 conn(0x7fd210077210 msgr2=0x7fd210105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:22.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 shutdown_connections 2024-09-12T12:19:22.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:22.187+0000 7fd2180d6640 1 -- 172.21.15.137:0/4224315165 wait complete. 2024-09-12T12:19:22.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.35 MB/sec execute 177 sec latency 18940.463 ms 2024-09-12T12:19:22.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:22 smithi189 ceph-mon[35475]: pgmap v188: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 85 op/s 2024-09-12T12:19:22.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:19:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:19:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:19:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:19:22.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:22 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4224315165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:19:22.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:22 smithi171 ceph-mon[35098]: pgmap v188: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 85 op/s 2024-09-12T12:19:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:19:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:19:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:19:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:19:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:22 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4224315165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:19:22.984 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:22 smithi137 ceph-mon[30950]: pgmap v188: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 85 op/s 2024-09-12T12:19:22.985 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:19:22.985 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:19:22.985 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:19:22.985 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:19:22.985 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:22 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4224315165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:19:23.016 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 7de7e21a --release --await' 2024-09-12T12:19:23.016 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 7de7e21a --release --await 2024-09-12T12:19:23.286 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:19:23.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 481317 8.30 MB/sec execute 178 sec latency 19940.906 ms 2024-09-12T12:19:23.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 -- 172.21.15.137:0/2399330450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 msgr2=0x7f16d4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 --2- 172.21.15.137:0/2399330450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d4100ee0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009930 tx=0x7f16c402a420 comp rx=0 tx=0).stop 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 -- 172.21.15.137:0/2399330450 shutdown_connections 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 --2- 172.21.15.137:0/2399330450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d4101420 0x7f16d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 --2- 172.21.15.137:0/2399330450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d4100ee0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 --2- 172.21.15.137:0/2399330450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d4106a80 0x7f16d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 -- 172.21.15.137:0/2399330450 >> 172.21.15.137:0/2399330450 conn(0x7f16d40fc7a0 msgr2=0x7f16d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 -- 172.21.15.137:0/2399330450 shutdown_connections 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.815+0000 7f16dc73f640 1 -- 172.21.15.137:0/2399330450 wait complete. 2024-09-12T12:19:23.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 Processor -- start 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 -- start start 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d4101420 0x7f16d4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d4106a80 0x7f16d419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16d4111fa0 con 0x7f16d4100a80 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16d4111e20 con 0x7f16d4106a80 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dc73f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16d4112120 con 0x7f16d4101420 2024-09-12T12:19:23.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53522/0 (socket says 172.21.15.137:53522) 2024-09-12T12:19:23.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 -- 172.21.15.137:0/886793750 learned_addr learned my addr 172.21.15.137:0/886793750 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:23.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16dacdc640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d4106a80 0x7f16d419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16d9cda640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d4101420 0x7f16d4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 -- 172.21.15.137:0/886793750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d4101420 msgr2=0x7f16d4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d4101420 0x7f16d4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 -- 172.21.15.137:0/886793750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d4106a80 msgr2=0x7f16d419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d4106a80 0x7f16d419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16da4db640 1 -- 172.21.15.137:0/886793750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d419d490 con 0x7f16d4100a80 2024-09-12T12:19:23.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.816+0000 7f16d9cda640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d4101420 0x7f16d4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:23.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.817+0000 7f16da4db640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d41984e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d770 tx=0x7f16d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.817+0000 7f16cb7fe640 1 -- 172.21.15.137:0/886793750 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0017070 con 0x7f16d4100a80 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.817+0000 7f16cb7fe640 1 -- 172.21.15.137:0/886793750 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d0005500 con 0x7f16d4100a80 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.817+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d419d780 con 0x7f16d4100a80 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.817+0000 7f16cb7fe640 1 -- 172.21.15.137:0/886793750 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004e00 con 0x7f16d4100a80 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.817+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d41a5060 con 0x7f16d4100a80 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.819+0000 7f16cb7fe640 1 -- 172.21.15.137:0/886793750 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f16d0004fa0 con 0x7f16d4100a80 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.819+0000 7f16deb18740 1 Processor -- start 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.819+0000 7f16deb18740 1 -- start start 2024-09-12T12:19:23.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16deb18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16deb18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16deb18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e44630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16deb18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eb71d9c620 con 0x55eb71e37060 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16deb18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eb71d43e50 con 0x55eb71e37980 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16deb18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb71d985e0 con 0x55eb71e42240 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16cb7fe640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f16c00a2d70 0x7f16c00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e37440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53526/0 (socket says 172.21.15.137:53526) 2024-09-12T12:19:23.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 -- 172.21.15.137:0/2319525133 learned_addr learned my addr 172.21.15.137:0/2319525133 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f16c00a2d70 0x7f16c00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16cb7fe640 1 -- 172.21.15.137:0/886793750 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f16d00ced90 con 0x7f16d4100a80 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16da4db640 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 -- 172.21.15.137:0/2319525133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 msgr2=0x55eb71e44630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16d9cda640 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e44630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e44630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 -- 172.21.15.137:0/2319525133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 msgr2=0x55eb71e41d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e41d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 -- 172.21.15.137:0/2319525133 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb71d34d40 con 0x55eb71e37060 2024-09-12T12:19:23.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16da4db640 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:23.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16d9cda640 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e44630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:19:23.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.820+0000 7f16dacdc640 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e37440 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f16cc005f40 tx=0x7f16cc014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b3da7144c5f2ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:23.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16c8ff9640 1 -- 172.21.15.137:0/2319525133 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc01ec50 con 0x55eb71e37060 2024-09-12T12:19:23.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16c8ff9640 1 -- 172.21.15.137:0/2319525133 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16cc023070 con 0x55eb71e37060 2024-09-12T12:19:23.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16c8ff9640 1 -- 172.21.15.137:0/2319525133 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16cc0057a0 con 0x55eb71e37060 2024-09-12T12:19:23.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16dacdc640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f16c00a2d70 0x7f16c00a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16cc022000 tx=0x7f16cc020000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:23.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16deb18740 1 -- 172.21.15.137:0/2319525133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 msgr2=0x55eb71e37440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16deb18740 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e37440 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f16cc005f40 tx=0x7f16cc014f70 comp rx=0 tx=0).stop 2024-09-12T12:19:23.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16deb18740 1 -- 172.21.15.137:0/2319525133 shutdown_connections 2024-09-12T12:19:23.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16deb18740 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e44630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16deb18740 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16deb18740 1 --2- 172.21.15.137:0/2319525133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e37440 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.821+0000 7f16deb18740 1 -- 172.21.15.137:0/2319525133 >> 172.21.15.137:0/2319525133 conn(0x55eb71e335f0 msgr2=0x55eb71e339d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:23.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 -- 172.21.15.137:0/2319525133 shutdown_connections 2024-09-12T12:19:23.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 -- 172.21.15.137:0/2319525133 wait complete. 2024-09-12T12:19:23.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 -- start start 2024-09-12T12:19:23.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 Processor -- start 2024-09-12T12:19:23.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37060 0x55eb71e3f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e42240 0x55eb71e3fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eb71e44e10 con 0x55eb71e42240 2024-09-12T12:19:23.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eb71d24f10 con 0x55eb71e37980 2024-09-12T12:19:23.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb71e44f90 con 0x55eb71e37060 2024-09-12T12:19:23.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16deb18740 10 client.0 _ll_register_callbacks cb 0x55eb71d48db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:23.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16dacdc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e42240 0x55eb71e3fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16d9cda640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16dacdc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e42240 0x55eb71e3fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53532/0 (socket says 172.21.15.137:53532) 2024-09-12T12:19:23.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16dacdc640 1 -- 172.21.15.137:0/2465920599 learned_addr learned my addr 172.21.15.137:0/2465920599 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:23.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.822+0000 7f16da4db640 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37060 0x55eb71e3f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16d9cda640 1 -- 172.21.15.137:0/2465920599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37060 msgr2=0x55eb71e3f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16d9cda640 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37060 0x55eb71e3f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16d9cda640 1 -- 172.21.15.137:0/2465920599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e42240 msgr2=0x55eb71e3fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16d9cda640 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e42240 0x55eb71e3fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16d9cda640 1 -- 172.21.15.137:0/2465920599 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb71aab580 con 0x55eb71e37980 2024-09-12T12:19:23.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16dacdc640 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e42240 0x55eb71e3fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:23.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16d9cda640 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e3f620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009900 tx=0x7f16c40318e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ecb5085f0fc4729 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:23.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:19:23.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16bdffb640 1 -- 172.21.15.137:0/2465920599 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c4035030 con 0x55eb71e37980 2024-09-12T12:19:23.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16bdffb640 1 -- 172.21.15.137:0/2465920599 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f16c4004460 con 0x55eb71e37980 2024-09-12T12:19:23.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16cc023070 con 0x55eb71e37980 2024-09-12T12:19:23.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16bdffb640 1 -- 172.21.15.137:0/2465920599 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16c4047020 con 0x55eb71e37980 2024-09-12T12:19:23.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16deb18740 10 client.25562 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:23.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb71b75960 con 0x55eb71e37980 2024-09-12T12:19:23.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.823+0000 7f16bdffb640 1 -- 172.21.15.137:0/2465920599 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16cc023070 con 0x55eb71e37980 2024-09-12T12:19:23.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16bdffb640 1 -- 172.21.15.137:0/2465920599 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f16c4004b80 con 0x55eb71e37980 2024-09-12T12:19:23.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16deb18740 10 client.25562 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:23.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16deb18740 10 client.25562 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:19:23.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16deb18740 20 client.25562 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16deb18740 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71e51cc0 0x55eb71e54120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16deb18740 4 client.25562 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb71da9b90 con 0x55eb71e51cc0 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16da4db640 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71e51cc0 0x55eb71e54120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16bdffb640 10 client.25562 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.824+0000 7f16da4db640 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71e51cc0 0x55eb71e54120 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56f68e934e73b6b server_cookie=a343f64e78b212da in_seq=0 out_seq=0 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16bdffb640 1 -- 172.21.15.137:0/2465920599 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb71da9b90 con 0x55eb71e51cc0 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16bdffb640 10 client.25562 handle_command_reply: tid=0 2024-09-12T12:19:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 client.25562 shutdown 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 msgr2=0x55eb71e3f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e3f620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f16c4009900 tx=0x7f16c40318e0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 shutdown_connections 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71e51cc0 0x55eb71e54120 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e42240 0x55eb71e3fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e3f620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 --2- 172.21.15.137:0/2465920599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37060 0x55eb71e3f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16bdffb640 0 client.25562 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 >> 172.21.15.137:0/2465920599 conn(0x55eb71e335f0 msgr2=0x55eb71e32750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 shutdown_connections 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 1 -- 172.21.15.137:0/2465920599 wait complete. 2024-09-12T12:19:23.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.825+0000 7f16deb18740 20 client.25562 trim_cache size 0 max 16384 2024-09-12T12:19:23.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 Processor -- start 2024-09-12T12:19:23.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 -- start start 2024-09-12T12:19:23.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e39b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37980 0x55eb71e32640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e42240 0x55eb71e599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41afff0 con 0x55eb71e37060 2024-09-12T12:19:23.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41ad8c0 con 0x55eb71e42240 2024-09-12T12:19:23.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.846+0000 7f16deb18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41ab730 con 0x55eb71e37980 2024-09-12T12:19:23.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e39b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e39b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53534/0 (socket says 172.21.15.137:53534) 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 -- 172.21.15.137:0/817513747 learned_addr learned my addr 172.21.15.137:0/817513747 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16d9cda640 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37980 0x55eb71e32640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16dacdc640 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e42240 0x55eb71e599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 -- 172.21.15.137:0/817513747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37980 msgr2=0x55eb71e32640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37980 0x55eb71e32640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 -- 172.21.15.137:0/817513747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e42240 msgr2=0x55eb71e599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e42240 0x55eb71e599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 -- 172.21.15.137:0/817513747 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb71e36e70 con 0x55eb71e37060 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16da4db640 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e39b10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f16d00cc5e0 tx=0x7f16d00ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b7e049bcf0222e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16bffff640 1 -- 172.21.15.137:0/817513747 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0020040 con 0x55eb71e37060 2024-09-12T12:19:23.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16bffff640 1 -- 172.21.15.137:0/817513747 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d009ba00 con 0x55eb71e37060 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.847+0000 7f16bffff640 1 -- 172.21.15.137:0/817513747 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0009cc0 con 0x55eb71e37060 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 -- 172.21.15.137:0/817513747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 msgr2=0x55eb71e39b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e39b10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f16d00cc5e0 tx=0x7f16d00ce040 comp rx=0 tx=0).stop 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 -- 172.21.15.137:0/817513747 shutdown_connections 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e42240 0x55eb71e599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e37980 0x55eb71e32640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 --2- 172.21.15.137:0/817513747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e39b10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 -- 172.21.15.137:0/817513747 >> 172.21.15.137:0/817513747 conn(0x55eb71e335f0 msgr2=0x55eb71e31f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 -- 172.21.15.137:0/817513747 shutdown_connections 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 -- 172.21.15.137:0/817513747 wait complete. 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 -- start start 2024-09-12T12:19:23.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.848+0000 7f16deb18740 1 Processor -- start 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16deb18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16deb18740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16deb18740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e46ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16deb18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41abd80 con 0x55eb71e37060 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16deb18740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41ad1f0 con 0x55eb71e37980 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16deb18740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16d41ac450 con 0x55eb71e42240 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16deb18740 10 client.0 _ll_register_callbacks cb 0x7f16c4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e46060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53540/0 (socket says 172.21.15.137:53540) 2024-09-12T12:19:23.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 -- 172.21.15.137:0/165183853 learned_addr learned my addr 172.21.15.137:0/165183853 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16d9cda640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16dacdc640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e46ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 -- 172.21.15.137:0/165183853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 msgr2=0x55eb71e46ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e46ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 -- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 msgr2=0x55eb71e465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 -- 172.21.15.137:0/165183853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb71ebbb20 con 0x55eb71e37060 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16dacdc640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e46ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16da4db640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e46060 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f16d009b000 tx=0x7f16d00976d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811da9f6008e5318 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:23.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.849+0000 7f16bdffb640 1 -- 172.21.15.137:0/165183853 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0020070 con 0x55eb71e37060 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16bdffb640 1 -- 172.21.15.137:0/165183853 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f16d0099600 con 0x55eb71e37060 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16d009ba00 con 0x55eb71e37060 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16bdffb640 1 -- 172.21.15.137:0/165183853 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16d009f6b0 con 0x55eb71e37060 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 10 client.16137 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb71ebce00 con 0x55eb71e37060 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16bdffb640 1 -- 172.21.15.137:0/165183853 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16d009ba00 con 0x55eb71e37060 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16bdffb640 1 -- 172.21.15.137:0/165183853 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f16d009a750 con 0x55eb71e37060 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 10 client.16137 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 10 client.16137 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 20 client.16137 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:23.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71ebd4b0 0x55eb71ebf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 4 client.16137 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7de7e21a", "release": true, "await": true}] 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7de7e21a", "release": true, "await": true}) -- 0x55eb71da40a0 con 0x55eb71ebd4b0 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.850+0000 7f16d9cda640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71ebd4b0 0x55eb71ebf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.851+0000 7f16d9cda640 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71ebd4b0 0x55eb71ebf8a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac052b765ec8f1b server_cookie=a45e1451a79a49d9 in_seq=0 out_seq=0 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.851+0000 7f16bdffb640 10 client.16137 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.855+0000 7f16bdffb640 1 -- 172.21.15.137:0/165183853 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eb71da40a0 con 0x55eb71ebd4b0 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.855+0000 7f16bdffb640 10 client.16137 handle_command_reply: tid=0 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.855+0000 7f16deb18740 1 client.16137 shutdown 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 msgr2=0x55eb71e46060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e46060 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f16d009b000 tx=0x7f16d00976d0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 shutdown_connections 2024-09-12T12:19:23.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb71e42240 0x55eb71e46ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb71e37980 0x55eb71e465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb71ebd4b0 0x55eb71ebf8a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 --2- 172.21.15.137:0/165183853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb71e37060 0x55eb71e46060 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16bdffb640 0 client.16137 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:23.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 >> 172.21.15.137:0/165183853 conn(0x55eb71e335f0 msgr2=0x55eb71e32750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:23.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 shutdown_connections 2024-09-12T12:19:23.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 1 -- 172.21.15.137:0/165183853 wait complete. 2024-09-12T12:19:23.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16deb18740 20 client.16137 trim_cache size 0 max 16384 2024-09-12T12:19:23.873 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:19:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 29, 2024-09-12T12:19:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:19:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 24, 2024-09-12T12:19:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "7de7e21a": { 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 24, 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 140.1, 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:19:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:19:23.875 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:19:23.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f16c00a2d70 msgr2=0x7f16c00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f16c00a2d70 0x7f16c00a5230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16cc022000 tx=0x7f16cc020000 comp rx=0 tx=0).stop 2024-09-12T12:19:23.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 msgr2=0x7f16d41984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:23.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d41984e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f16d000d770 tx=0x7f16d000dc30 comp rx=0 tx=0).stop 2024-09-12T12:19:23.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 shutdown_connections 2024-09-12T12:19:23.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f16c00a2d70 0x7f16c00a5230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d4106a80 0x7f16d419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16d4101420 0x7f16d4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 --2- 172.21.15.137:0/886793750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16d4100a80 0x7f16d41984e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:23.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.856+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 >> 172.21.15.137:0/886793750 conn(0x7f16d40fc7a0 msgr2=0x7f16d40fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:23.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.857+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 shutdown_connections 2024-09-12T12:19:23.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:19:23.857+0000 7f16dc73f640 1 -- 172.21.15.137:0/886793750 wait complete. 2024-09-12T12:19:24.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 -- 172.21.15.137:0/855290830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101b10 msgr2=0x7f8ee010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 --2- 172.21.15.137:0/855290830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101b10 0x7f8ee010d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009ca0 tx=0x7f8ed802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 -- 172.21.15.137:0/855290830 shutdown_connections 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 --2- 172.21.15.137:0/855290830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101b10 0x7f8ee010d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 --2- 172.21.15.137:0/855290830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0101190 0x7f8ee01015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 --2- 172.21.15.137:0/855290830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0102840 0x7f8ee0102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 -- 172.21.15.137:0/855290830 >> 172.21.15.137:0/855290830 conn(0x7f8ee00770f0 msgr2=0x7f8ee00fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 -- 172.21.15.137:0/855290830 shutdown_connections 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 -- 172.21.15.137:0/855290830 wait complete. 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 Processor -- start 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.535+0000 7f8ee859f640 1 -- start start 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee859f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101190 0x7f8ee0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee859f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0101b10 0x7f8ee01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee859f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0102840 0x7f8ee01a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee859f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0110eb0 con 0x7f8ee0101190 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee859f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0110d30 con 0x7f8ee0101b10 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee859f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0111030 con 0x7f8ee0102840 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101190 0x7f8ee0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee5b3a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0101b10 0x7f8ee01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee6b3c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0102840 0x7f8ee01a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101190 0x7f8ee0199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53564/0 (socket says 172.21.15.137:53564) 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 -- 172.21.15.137:0/1848873579 learned_addr learned my addr 172.21.15.137:0/1848873579 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 -- 172.21.15.137:0/1848873579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0102840 msgr2=0x7f8ee01a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0102840 0x7f8ee01a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 -- 172.21.15.137:0/1848873579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0101b10 msgr2=0x7f8ee01996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0101b10 0x7f8ee01996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 -- 172.21.15.137:0/1848873579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee01a4740 con 0x7f8ee0101190 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee6b3c640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0102840 0x7f8ee01a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0101b10 0x7f8ee01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.536+0000 7f8ee633b640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101190 0x7f8ee0199180 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00e9f0 tx=0x7f8edc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.537+0000 7f8ed77fe640 1 -- 172.21.15.137:0/1848873579 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00cd80 con 0x7f8ee0101190 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.537+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee01a4910 con 0x7f8ee0101190 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.537+0000 7f8ed77fe640 1 -- 172.21.15.137:0/1848873579 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc010910 con 0x7f8ee0101190 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.537+0000 7f8ed77fe640 1 -- 172.21.15.137:0/1848873579 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0188f0 con 0x7f8ee0101190 2024-09-12T12:19:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.537+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee01a4e80 con 0x7f8ee0101190 2024-09-12T12:19:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.539+0000 7f8ed77fe640 1 -- 172.21.15.137:0/1848873579 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8edc010ac0 con 0x7f8ee0101190 2024-09-12T12:19:24.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.540+0000 7f8ed77fe640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8ecc0a2da0 0x7f8ecc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.540+0000 7f8eeb416740 1 Processor -- start 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.540+0000 7f8ed77fe640 1 -- 172.21.15.137:0/1848873579 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f8edc0cf740 con 0x7f8ee0101190 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.540+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8ecc0a2da0 0x7f8ecc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.540+0000 7f8eeb416740 1 -- start start 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa61357ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61362770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa613650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8eeb416740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fa612bc2f0 con 0x55fa61362cb0 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8eeb416740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fa61264270 con 0x55fa61357af0 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8eeb416740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa612b82b0 con 0x55fa61358410 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee6b3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa613650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa61357ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa61357ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37876/0 (socket says 172.21.15.137:37876) 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee6b3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa613650a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53568/0 (socket says 172.21.15.137:53568) 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 -- 172.21.15.137:0/3704028657 learned_addr learned my addr 172.21.15.137:0/3704028657 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8ecc0a2da0 0x7f8ecc0a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0100bd0 tx=0x7f8ed0005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61362770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 -- 172.21.15.137:0/3704028657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 msgr2=0x55fa61362770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61362770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 -- 172.21.15.137:0/3704028657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 msgr2=0x55fa613650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa613650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 -- 172.21.15.137:0/3704028657 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa61263340 con 0x55fa61357af0 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee6b3c640 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa613650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ee633b640 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa61357ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8edc098730 tx=0x7f8edc09bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f08af400a3222a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.541+0000 7f8ed4bfe640 1 -- 172.21.15.137:0/3704028657 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0cd160 con 0x55fa61357af0 2024-09-12T12:19:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8ed4bfe640 1 -- 172.21.15.137:0/3704028657 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc0983d0 con 0x55fa61357af0 2024-09-12T12:19:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8ed4bfe640 1 -- 172.21.15.137:0/3704028657 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc097d10 con 0x55fa61357af0 2024-09-12T12:19:24.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8eeb416740 1 -- 172.21.15.137:0/3704028657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 msgr2=0x55fa61357ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa61357ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8edc098730 tx=0x7f8edc09bea0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8eeb416740 1 -- 172.21.15.137:0/3704028657 shutdown_connections 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa613650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61362770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3704028657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa61357ed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.542+0000 7f8eeb416740 1 -- 172.21.15.137:0/3704028657 >> 172.21.15.137:0/3704028657 conn(0x55fa61354060 msgr2=0x55fa61354440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 -- 172.21.15.137:0/3704028657 shutdown_connections 2024-09-12T12:19:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 -- 172.21.15.137:0/3704028657 wait complete. 2024-09-12T12:19:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 -- start start 2024-09-12T12:19:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 Processor -- start 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61357af0 0x55fa6135fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61360110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61362cb0 0x55fa61360650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fa61365880 con 0x55fa61357af0 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fa61245290 con 0x55fa61362cb0 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa61365a00 con 0x55fa61358410 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8eeb416740 10 client.0 _ll_register_callbacks cb 0x55fa61276140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8ee633b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61357af0 0x55fa6135fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8ee633b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61357af0 0x55fa6135fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53582/0 (socket says 172.21.15.137:53582) 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8ee633b640 1 -- 172.21.15.137:0/3752252308 learned_addr learned my addr 172.21.15.137:0/3752252308 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.543+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61360110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ee6b3c640 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61362cb0 0x55fa61360650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ee633b640 1 -- 172.21.15.137:0/3752252308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 msgr2=0x55fa61360110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ee633b640 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61360110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ee633b640 1 -- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61362cb0 msgr2=0x55fa61360650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ee633b640 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61362cb0 0x55fa61360650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ee633b640 1 -- 172.21.15.137:0/3752252308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa60fcf210 con 0x55fa61357af0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ee633b640 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61357af0 0x55fa6135fbd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c6a0 tx=0x7f8edc00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a43e6fc56dc089 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/3752252308 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0b8070 con 0x55fa61357af0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8edc0983d0 con 0x55fa61357af0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/3752252308 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f8edc010640 con 0x55fa61357af0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/3752252308 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8edc09b4e0 con 0x55fa61357af0 2024-09-12T12:19:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8eeb416740 10 client.16149 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa6109b620 con 0x55fa61357af0 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/3752252308 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8edc0983d0 con 0x55fa61357af0 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.544+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/3752252308 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f8edc018a90 con 0x55fa61357af0 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8eeb416740 10 client.16149 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8eeb416740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8eeb416740 20 client.16149 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa61372ad0 0x55fa61374ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8eeb416740 4 client.16149 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa612c9a20 con 0x55fa61372ad0 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa61372ad0 0x55fa61374ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8ec9ffb640 10 client.16149 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.545+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa61372ad0 0x55fa61374ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50187b22ae17983a server_cookie=ea3ec45b54a315b7 in_seq=0 out_seq=0 2024-09-12T12:19:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/3752252308 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa612c9a20 con 0x55fa61372ad0 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8ec9ffb640 10 client.16149 handle_command_reply: tid=0 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 client.16149 shutdown 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61357af0 msgr2=0x55fa6135fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61357af0 0x55fa6135fbd0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c6a0 tx=0x7f8edc00c840 comp rx=0 tx=0).stop 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 shutdown_connections 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61362cb0 0x55fa61360650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61358410 0x55fa61360110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa61372ad0 0x55fa61374ec0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 --2- 172.21.15.137:0/3752252308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61357af0 0x55fa6135fbd0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 >> 172.21.15.137:0/3752252308 conn(0x55fa61354060 msgr2=0x55fa61359d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8ec9ffb640 0 client.16149 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 shutdown_connections 2024-09-12T12:19:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 1 -- 172.21.15.137:0/3752252308 wait complete. 2024-09-12T12:19:24.555 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.546+0000 7f8eeb416740 20 client.16149 trim_cache size 0 max 16384 2024-09-12T12:19:24.555 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7de7e21a', total seconds elapsed: 23.5 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.561+0000 7f8eeb416740 1 Processor -- start 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.561+0000 7f8eeb416740 1 -- start start 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.561+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61357af0 0x55fa61360480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.561+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61358410 0x55fa6137a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa6137a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8eeb416740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01b0030 con 0x55fa61362cb0 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8eeb416740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01ad900 con 0x55fa61358410 2024-09-12T12:19:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8eeb416740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01ab750 con 0x55fa61357af0 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa6137a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa6137a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53596/0 (socket says 172.21.15.137:53596) 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 -- 172.21.15.137:0/162192908 learned_addr learned my addr 172.21.15.137:0/162192908 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61358410 0x55fa6137a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee633b640 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61357af0 0x55fa61360480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 -- 172.21.15.137:0/162192908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61357af0 msgr2=0x55fa61360480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61357af0 0x55fa61360480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 -- 172.21.15.137:0/162192908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61358410 msgr2=0x55fa6137a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61358410 0x55fa6137a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 -- 172.21.15.137:0/162192908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa6135aa80 con 0x55fa61362cb0 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ee6b3c640 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa6137a9f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009ca0 tx=0x7f8ed8009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b6cc709d76fcab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ecbfff640 1 -- 172.21.15.137:0/162192908 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8003cb0 con 0x55fa61362cb0 2024-09-12T12:19:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ecbfff640 1 -- 172.21.15.137:0/162192908 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed804d070 con 0x55fa61362cb0 2024-09-12T12:19:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.562+0000 7f8ecbfff640 1 -- 172.21.15.137:0/162192908 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80305c0 con 0x55fa61362cb0 2024-09-12T12:19:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 -- 172.21.15.137:0/162192908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 msgr2=0x55fa6137a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa6137a9f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009ca0 tx=0x7f8ed8009940 comp rx=0 tx=0).stop 2024-09-12T12:19:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 -- 172.21.15.137:0/162192908 shutdown_connections 2024-09-12T12:19:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61362cb0 0x55fa6137a9f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61358410 0x55fa6137a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 --2- 172.21.15.137:0/162192908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61357af0 0x55fa61360480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 -- 172.21.15.137:0/162192908 >> 172.21.15.137:0/162192908 conn(0x55fa61354060 msgr2=0x55fa61361c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 -- 172.21.15.137:0/162192908 shutdown_connections 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 -- 172.21.15.137:0/162192908 wait complete. 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.563+0000 7f8eeb416740 1 -- start start 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 1 Processor -- start 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa6135cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61358410 0x55fa6135d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61362cb0 0x55fa6135d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01abdc0 con 0x55fa61358410 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01ad230 con 0x55fa61357af0 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01ac490 con 0x55fa61362cb0 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8eeb416740 10 client.0 _ll_register_callbacks cb 0x7f8ed0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61358410 0x55fa6135d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee633b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa6135cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61358410 0x55fa6135d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53608/0 (socket says 172.21.15.137:53608) 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 -- 172.21.15.137:0/2090483956 learned_addr learned my addr 172.21.15.137:0/2090483956 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee6b3c640 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61362cb0 0x55fa6135d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 -- 172.21.15.137:0/2090483956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61362cb0 msgr2=0x55fa6135d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61362cb0 0x55fa6135d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 -- 172.21.15.137:0/2090483956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 msgr2=0x55fa6135cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa6135cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 -- 172.21.15.137:0/2090483956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa613dc480 con 0x55fa61358410 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ee5b3a640 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61358410 0x55fa6135d0c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0004650 tx=0x7f8ed0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88014c7cfa866a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ec9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.564+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/2090483956 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0037060 con 0x55fa61358410 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/2090483956 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7f8ed000f540 con 0x55fa61358410 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ed804d070 con 0x55fa61358410 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/2090483956 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ed000f740 con 0x55fa61358410 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 10 client.16155 fetch_fsmap learned FSMap version 29 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa613dd760 con 0x55fa61358410 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/2090483956 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ed804d070 con 0x55fa61358410 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/2090483956 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f8ed001a040 con 0x55fa61358410 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 10 client.16155 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 20 client.16155 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa613ddec0 0x55fa613e02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 4 client.16155 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:19:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa612c3d70 con 0x55fa613ddec0 2024-09-12T12:19:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.565+0000 7f8ee633b640 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa613ddec0 0x55fa613e02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.566+0000 7f8ee633b640 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa613ddec0 0x55fa613e02e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17d4fba6e083ea9 server_cookie=7d2001208bd5ff22 in_seq=0 out_seq=0 2024-09-12T12:19:24.572 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.566+0000 7f8ec9ffb640 10 client.16155 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.595+0000 7f8ec9ffb640 1 -- 172.21.15.137:0/2090483956 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa612c3d70 con 0x55fa613ddec0 2024-09-12T12:19:24.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.595+0000 7f8ec9ffb640 10 client.16155 handle_command_reply: tid=0 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.595+0000 7f8eeb416740 1 client.16155 shutdown 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.595+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61358410 msgr2=0x55fa6135d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.595+0000 7f8eeb416740 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61358410 0x55fa6135d0c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0004650 tx=0x7f8ed0004730 comp rx=0 tx=0).stop 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 shutdown_connections 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa61362cb0 0x55fa6135d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fa613ddec0 0x55fa613e02e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa61358410 0x55fa6135d0c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 --2- 172.21.15.137:0/2090483956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fa61357af0 0x55fa6135cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ec9ffb640 0 client.16155 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 >> 172.21.15.137:0/2090483956 conn(0x55fa61354060 msgr2=0x55fa613529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 shutdown_connections 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 1 -- 172.21.15.137:0/2090483956 wait complete. 2024-09-12T12:19:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8eeb416740 20 client.16155 trim_cache size 0 max 16384 2024-09-12T12:19:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:19:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:19:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:19:24.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8ecc0a2da0 msgr2=0x7f8ecc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8ecc0a2da0 0x7f8ecc0a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0100bd0 tx=0x7f8ed0005f90 comp rx=0 tx=0).stop 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101190 msgr2=0x7f8ee0199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101190 0x7f8ee0199180 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00e9f0 tx=0x7f8edc00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 shutdown_connections 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8ecc0a2da0 0x7f8ecc0a5260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ee0102840 0x7f8ee01a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee0101b10 0x7f8ee01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.596+0000 7f8ee859f640 1 --2- 172.21.15.137:0/1848873579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ee0101190 0x7f8ee0199180 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:24.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.597+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 >> 172.21.15.137:0/1848873579 conn(0x7f8ee00770f0 msgr2=0x7f8ee01046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.597+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 shutdown_connections 2024-09-12T12:19:24.600 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:24.597+0000 7f8ee859f640 1 -- 172.21.15.137:0/1848873579 wait complete. 2024-09-12T12:19:24.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 483579 8.33 MB/sec execute 179 sec latency 20065.406 ms 2024-09-12T12:19:24.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:24 smithi137 ceph-mon[30950]: pgmap v189: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 85 op/s 2024-09-12T12:19:24.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:24 smithi189 ceph-mon[35475]: pgmap v189: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 85 op/s 2024-09-12T12:19:24.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:24 smithi171 ceph-mon[35098]: pgmap v189: 97 pgs: 97 active+clean; 153 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 2.9 MiB/s wr, 85 op/s 2024-09-12T12:19:25.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 485673 8.33 MB/sec execute 180 sec latency 15.142 ms 2024-09-12T12:19:26.733 INFO:tasks.workunit.client.0.smithi137.stdout: 1 487710 8.35 MB/sec execute 181 sec latency 17.988 ms 2024-09-12T12:19:26.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:26 smithi189 ceph-mon[35475]: pgmap v190: 97 pgs: 97 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 3.8 MiB/s wr, 132 op/s 2024-09-12T12:19:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:26 smithi171 ceph-mon[35098]: pgmap v190: 97 pgs: 97 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 3.8 MiB/s wr, 132 op/s 2024-09-12T12:19:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:26 smithi137 ceph-mon[30950]: pgmap v190: 97 pgs: 97 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 3.8 MiB/s wr, 132 op/s 2024-09-12T12:19:27.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 490569 8.40 MB/sec execute 182 sec latency 14.157 ms 2024-09-12T12:19:28.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 492119 8.39 MB/sec execute 183 sec latency 48.459 ms 2024-09-12T12:19:28.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:28 smithi189 ceph-mon[35475]: pgmap v191: 97 pgs: 97 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.9 MiB/s wr, 76 op/s 2024-09-12T12:19:28.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:28 smithi171 ceph-mon[35098]: pgmap v191: 97 pgs: 97 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.9 MiB/s wr, 76 op/s 2024-09-12T12:19:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:28 smithi137 ceph-mon[30950]: pgmap v191: 97 pgs: 97 active+clean; 146 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.9 MiB/s wr, 76 op/s 2024-09-12T12:19:29.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 494422 8.42 MB/sec execute 184 sec latency 14.965 ms 2024-09-12T12:19:30.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 496342 8.41 MB/sec execute 185 sec latency 17.227 ms 2024-09-12T12:19:30.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:30 smithi189 ceph-mon[35475]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2024-09-12T12:19:30.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:30 smithi171 ceph-mon[35098]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2024-09-12T12:19:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:30 smithi137 ceph-mon[30950]: pgmap v192: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2024-09-12T12:19:31.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 498343 8.44 MB/sec execute 186 sec latency 18.660 ms 2024-09-12T12:19:32.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 501059 8.48 MB/sec execute 187 sec latency 17.836 ms 2024-09-12T12:19:32.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:32 smithi171 ceph-mon[35098]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2024-09-12T12:19:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:32 smithi137 ceph-mon[30950]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2024-09-12T12:19:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:32 smithi189 ceph-mon[35475]: pgmap v193: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2024-09-12T12:19:33.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 502460 8.47 MB/sec execute 188 sec latency 63.998 ms 2024-09-12T12:19:34.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 504884 8.50 MB/sec execute 189 sec latency 27.440 ms 2024-09-12T12:19:34.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:34 smithi171 ceph-mon[35098]: pgmap v194: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2024-09-12T12:19:34.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:34 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:19:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:34 smithi137 ceph-mon[30950]: pgmap v194: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2024-09-12T12:19:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:34 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:19:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:34 smithi189 ceph-mon[35475]: pgmap v194: 97 pgs: 97 active+clean; 147 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 200 op/s 2024-09-12T12:19:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:34 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:19:35.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 506839 8.49 MB/sec execute 190 sec latency 46.894 ms 2024-09-12T12:19:36.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 508634 8.51 MB/sec execute 191 sec latency 21.563 ms 2024-09-12T12:19:36.850 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:36 smithi137 ceph-mon[30950]: pgmap v195: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 324 op/s 2024-09-12T12:19:36.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:36 smithi171 ceph-mon[35098]: pgmap v195: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 324 op/s 2024-09-12T12:19:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:36 smithi189 ceph-mon[35475]: pgmap v195: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 324 op/s 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 -- 172.21.15.137:0/2003952724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 msgr2=0x7fd78010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 --2- 172.21.15.137:0/2003952724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd78010bae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009900 tx=0x7fd76c02a480 comp rx=0 tx=0).stop 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 -- 172.21.15.137:0/2003952724 shutdown_connections 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 --2- 172.21.15.137:0/2003952724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78010c0b0 0x7fd78010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 --2- 172.21.15.137:0/2003952724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd78010bae0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 --2- 172.21.15.137:0/2003952724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd780102ce0 0x7fd7801030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 -- 172.21.15.137:0/2003952724 >> 172.21.15.137:0/2003952724 conn(0x7fd780077990 msgr2=0x7fd7800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 -- 172.21.15.137:0/2003952724 shutdown_connections 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.687+0000 7fd786c4c640 1 -- 172.21.15.137:0/2003952724 wait complete. 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 Processor -- start 2024-09-12T12:19:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 -- start start 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd780102ce0 0x7fd780196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd780196670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78010c0b0 0x7fd78019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010fca0 con 0x7fd780103600 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010fb20 con 0x7fd780102ce0 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd786c4c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010fe20 con 0x7fd78010c0b0 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd77ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd780196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd7849e8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd780102ce0 0x7fd780196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd77ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd780196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36992/0 (socket says 172.21.15.137:36992) 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd7851e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78010c0b0 0x7fd78019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.688+0000 7fd77ffff640 1 -- 172.21.15.137:0/3038299669 learned_addr learned my addr 172.21.15.137:0/3038299669 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77ffff640 1 -- 172.21.15.137:0/3038299669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78010c0b0 msgr2=0x7fd78019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77ffff640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78010c0b0 0x7fd78019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77ffff640 1 -- 172.21.15.137:0/3038299669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd780102ce0 msgr2=0x7fd780196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77ffff640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd780102ce0 0x7fd780196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77ffff640 1 -- 172.21.15.137:0/3038299669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78019b120 con 0x7fd780103600 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd7849e8640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd780102ce0 0x7fd780196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd7851e9640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78010c0b0 0x7fd78019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77ffff640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd780196670 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77dffb640 1 -- 172.21.15.137:0/3038299669 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c03f030 con 0x7fd780103600 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78019b3b0 con 0x7fd780103600 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77dffb640 1 -- 172.21.15.137:0/3038299669 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76c031990 con 0x7fd780103600 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd77dffb640 1 -- 172.21.15.137:0/3038299669 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c035070 con 0x7fd780103600 2024-09-12T12:19:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.689+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7801a2ca0 con 0x7fd780103600 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd77dffb640 1 -- 172.21.15.137:0/3038299669 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd76c042030 con 0x7fd780103600 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 Processor -- start 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 -- start start 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c04eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c059770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c05c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558b7bfb32f0 con 0x558b7c059cb0 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558b7bf5b270 con 0x558b7c04eaf0 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7899e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b7bfaf2b0 con 0x558b7c04f410 2024-09-12T12:19:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7851e9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c04eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd77dffb640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7700a2da0 0x7fd7700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd77ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c05c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7849e8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c059770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7851e9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c04eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:35936/0 (socket says 172.21.15.137:35936) 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.692+0000 7fd7851e9640 1 -- 172.21.15.137:0/2354804815 learned_addr learned my addr 172.21.15.137:0/2354804815 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7851e9640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7700a2da0 0x7fd7700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd77dffb640 1 -- 172.21.15.137:0/3038299669 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd76c0ef7e0 con 0x7fd780103600 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7849e8640 1 -- 172.21.15.137:0/2354804815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 msgr2=0x558b7c04eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7849e8640 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c04eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7849e8640 1 -- 172.21.15.137:0/2354804815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 msgr2=0x558b7c05c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7849e8640 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c05c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7849e8640 1 -- 172.21.15.137:0/2354804815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7bf5a340 con 0x558b7c04f410 2024-09-12T12:19:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd77ffff640 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c05c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7849e8640 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c059770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd774009fd0 tx=0x7fd7740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ba3d87cd547047 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd7851e9640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7700a2da0 0x7fd7700a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd77800dfd0 tx=0x7fd778004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd76affd640 1 -- 172.21.15.137:0/2354804815 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774004020 con 0x558b7c04f410 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd76affd640 1 -- 172.21.15.137:0/2354804815 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd774005500 con 0x558b7c04f410 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.693+0000 7fd76affd640 1 -- 172.21.15.137:0/2354804815 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774004db0 con 0x558b7c04f410 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 -- 172.21.15.137:0/2354804815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 msgr2=0x558b7c059770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c059770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd774009fd0 tx=0x7fd7740118d0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 -- 172.21.15.137:0/2354804815 shutdown_connections 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c05c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c059770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 --2- 172.21.15.137:0/2354804815 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c04eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 -- 172.21.15.137:0/2354804815 >> 172.21.15.137:0/2354804815 conn(0x558b7c04b060 msgr2=0x558b7c04b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 -- 172.21.15.137:0/2354804815 shutdown_connections 2024-09-12T12:19:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 -- 172.21.15.137:0/2354804815 wait complete. 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 -- start start 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.694+0000 7fd7899e6740 1 Processor -- start 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c056b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c057080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c0575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558b7c05c880 con 0x558b7c059cb0 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558b7bf3c290 con 0x558b7c04eaf0 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558b7c05ca00 con 0x558b7c04f410 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 10 client.0 _ll_register_callbacks cb 0x558b7bf6d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c0575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7849e8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c056b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c0575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37008/0 (socket says 172.21.15.137:37008) 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 -- 172.21.15.137:0/3077945296 learned_addr learned my addr 172.21.15.137:0/3077945296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd77ffff640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c057080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 -- 172.21.15.137:0/3077945296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 msgr2=0x558b7c057080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c057080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 -- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 msgr2=0x558b7c056b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c056b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 -- 172.21.15.137:0/3077945296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7bcc6210 con 0x558b7c059cb0 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7849e8640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c056b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd77ffff640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c057080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7851e9640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c0575c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd778012ff0 tx=0x7fd778004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b6c095d5622f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd75ffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd75ffff640 1 -- 172.21.15.137:0/3077945296 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77801e650 con 0x558b7c059cb0 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd75ffff640 1 -- 172.21.15.137:0/3077945296 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd77801e7f0 con 0x558b7c059cb0 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.695+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd774005500 con 0x558b7c059cb0 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd75ffff640 1 -- 172.21.15.137:0/3077945296 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd77801ea30 con 0x558b7c059cb0 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 10 client.16167 fetch_fsmap learned FSMap version 30 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b7bd92620 con 0x558b7c059cb0 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd75ffff640 1 -- 172.21.15.137:0/3077945296 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd774005500 con 0x558b7c059cb0 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd75ffff640 1 -- 172.21.15.137:0/3077945296 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fd778033c30 con 0x558b7c059cb0 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 10 client.16167 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 20 client.16167 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c069c00 0x558b7c06c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 4 client.16167 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b7bfc0a20 con 0x558b7c069c00 2024-09-12T12:19:37.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.696+0000 7fd7849e8640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c069c00 0x558b7c06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7849e8640 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c069c00 0x558b7c06c030 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1480efc9e824d86 server_cookie=fd0deaaf452304e in_seq=0 out_seq=0 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd75ffff640 10 client.16167 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd75ffff640 1 -- 172.21.15.137:0/3077945296 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b7bfc0a20 con 0x558b7c069c00 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd75ffff640 10 client.16167 handle_command_reply: tid=0 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 client.16167 shutdown 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 msgr2=0x558b7c0575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c0575c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd778012ff0 tx=0x7fd778004a40 comp rx=0 tx=0).stop 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 shutdown_connections 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c069c00 0x558b7c06c030 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c059cb0 0x558b7c0575c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd75ffff640 0 client.16167 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04f410 0x558b7c057080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 --2- 172.21.15.137:0/3077945296 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04eaf0 0x558b7c056b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.697+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 >> 172.21.15.137:0/3077945296 conn(0x558b7c04b060 msgr2=0x558b7c050d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.698+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 shutdown_connections 2024-09-12T12:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.698+0000 7fd7899e6740 1 -- 172.21.15.137:0/3077945296 wait complete. 2024-09-12T12:19:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.698+0000 7fd7899e6740 20 client.16167 trim_cache size 0 max 16384 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 Processor -- start 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 -- start start 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04eaf0 0x558b7c051560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04f410 0x558b7c055a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c059cb0 0x558b7c0573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801addb0 con 0x558b7c04f410 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801ab650 con 0x558b7c059cb0 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7899e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801a94c0 con 0x558b7c04eaf0 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd77ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04f410 0x558b7c055a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd77ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04f410 0x558b7c055a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37012/0 (socket says 172.21.15.137:37012) 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd77ffff640 1 -- 172.21.15.137:0/1752241335 learned_addr learned my addr 172.21.15.137:0/1752241335 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.725+0000 7fd7851e9640 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c059cb0 0x558b7c0573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd7849e8640 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04eaf0 0x558b7c051560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd77ffff640 1 -- 172.21.15.137:0/1752241335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04eaf0 msgr2=0x558b7c051560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd77ffff640 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04eaf0 0x558b7c051560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd77ffff640 1 -- 172.21.15.137:0/1752241335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c059cb0 msgr2=0x558b7c0573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd77ffff640 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c059cb0 0x558b7c0573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd77ffff640 1 -- 172.21.15.137:0/1752241335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7c04ae90 con 0x558b7c04f410 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd77ffff640 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04f410 0x558b7c055a60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0ed580 tx=0x7fd76c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0a3f860379445b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd76a7fc640 1 -- 172.21.15.137:0/1752241335 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c049070 con 0x558b7c04f410 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd76a7fc640 1 -- 172.21.15.137:0/1752241335 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76c003e00 con 0x558b7c04f410 2024-09-12T12:19:37.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.726+0000 7fd76a7fc640 1 -- 172.21.15.137:0/1752241335 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0b8530 con 0x558b7c04f410 2024-09-12T12:19:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 -- 172.21.15.137:0/1752241335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04f410 msgr2=0x558b7c055a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04f410 0x558b7c055a60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0ed580 tx=0x7fd76c009680 comp rx=0 tx=0).stop 2024-09-12T12:19:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 -- 172.21.15.137:0/1752241335 shutdown_connections 2024-09-12T12:19:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c059cb0 0x558b7c0573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04f410 0x558b7c055a60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1752241335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c04eaf0 0x558b7c051560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.732 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 -- 172.21.15.137:0/1752241335 >> 172.21.15.137:0/1752241335 conn(0x558b7c04b060 msgr2=0x558b7c058bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 -- 172.21.15.137:0/1752241335 shutdown_connections 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 -- 172.21.15.137:0/1752241335 wait complete. 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 -- start start 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.727+0000 7fd7899e6740 1 Processor -- start 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04eaf0 0x558b7c061ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04f410 0x558b7c062430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7899e6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c059cb0 0x558b7c053390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7899e6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801a9b10 con 0x558b7c04eaf0 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7899e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801aaf80 con 0x558b7c04f410 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7899e6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd7801aa1e0 con 0x558b7c059cb0 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7899e6740 10 client.0 _ll_register_callbacks cb 0x7fd7780048f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04eaf0 0x558b7c061ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04eaf0 0x558b7c061ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37024/0 (socket says 172.21.15.137:37024) 2024-09-12T12:19:37.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 -- 172.21.15.137:0/1970650495 learned_addr learned my addr 172.21.15.137:0/1970650495 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7851e9640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c059cb0 0x558b7c053390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd77ffff640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04f410 0x558b7c062430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 -- 172.21.15.137:0/1970650495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c059cb0 msgr2=0x558b7c053390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c059cb0 0x558b7c053390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 -- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04f410 msgr2=0x558b7c062430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04f410 0x558b7c062430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7849e8640 1 -- 172.21.15.137:0/1970650495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7c04bcf0 con 0x558b7c04eaf0 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd77ffff640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04f410 0x558b7c062430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.728+0000 7fd7851e9640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c059cb0 0x558b7c053390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7849e8640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04eaf0 0x558b7c061ef0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd774004940 tx=0x7fd774004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe667099757695b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd75ffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd75ffff640 1 -- 172.21.15.137:0/1970650495 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774032070 con 0x558b7c04eaf0 2024-09-12T12:19:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd75ffff640 1 -- 172.21.15.137:0/1970650495 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fd774004020 con 0x558b7c04eaf0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b7bfbad70 con 0x558b7c04eaf0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd75ffff640 1 -- 172.21.15.137:0/1970650495 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd77403b050 con 0x558b7c04eaf0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 10 client.16173 fetch_fsmap learned FSMap version 30 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b7c0d42d0 con 0x558b7c04eaf0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd75ffff640 1 -- 172.21.15.137:0/1970650495 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558b7bfbad70 con 0x558b7c04eaf0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd75ffff640 1 -- 172.21.15.137:0/1970650495 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fd774035020 con 0x558b7c04eaf0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 10 client.16173 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 20 client.16173 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c0d48b0 0x558b7c0d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 4 client.16173 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.729+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b7bf37900 con 0x558b7c0d48b0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.730+0000 7fd77ffff640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c0d48b0 0x558b7c0d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.730+0000 7fd75ffff640 10 client.16173 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.730+0000 7fd77ffff640 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c0d48b0 0x558b7c0d6ca0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612ade17060ba390 server_cookie=4633d257cf9ee266 in_seq=0 out_seq=0 2024-09-12T12:19:37.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 511387 8.54 MB/sec execute 192 sec latency 22.838 ms 2024-09-12T12:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.811+0000 7fd75ffff640 1 -- 172.21.15.137:0/1970650495 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b7bf37900 con 0x558b7c0d48b0 2024-09-12T12:19:37.811 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.811+0000 7fd75ffff640 10 client.16173 handle_command_reply: tid=0 2024-09-12T12:19:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.812+0000 7fd7899e6740 1 client.16173 shutdown 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.813+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04eaf0 msgr2=0x558b7c061ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.813+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04eaf0 0x558b7c061ef0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd774004940 tx=0x7fd774004970 comp rx=0 tx=0).stop 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 shutdown_connections 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7c059cb0 0x558b7c053390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558b7c04f410 0x558b7c062430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558b7c0d48b0 0x558b7c0d6ca0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 --2- 172.21.15.137:0/1970650495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558b7c04eaf0 0x558b7c061ef0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd75ffff640 0 client.16173 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 >> 172.21.15.137:0/1970650495 conn(0x558b7c04b060 msgr2=0x558b7c0499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 shutdown_connections 2024-09-12T12:19:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 1 -- 172.21.15.137:0/1970650495 wait complete. 2024-09-12T12:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.814+0000 7fd7899e6740 20 client.16173 trim_cache size 0 max 16384 2024-09-12T12:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:19:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7700a2da0 msgr2=0x7fd7700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7700a2da0 0x7fd7700a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd77800dfd0 tx=0x7fd778004ab0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 msgr2=0x7fd780196670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd780196670 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c0042f0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 shutdown_connections 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7700a2da0 0x7fd7700a5260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd78010c0b0 0x7fd78019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd780103600 0x7fd780196670 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 --2- 172.21.15.137:0/3038299669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd780102ce0 0x7fd780196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 >> 172.21.15.137:0/3038299669 conn(0x7fd780077990 msgr2=0x7fd7800fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 shutdown_connections 2024-09-12T12:19:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:37.815+0000 7fd786c4c640 1 -- 172.21.15.137:0/3038299669 wait complete. 2024-09-12T12:19:38.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 512761 8.53 MB/sec execute 193 sec latency 48.276 ms 2024-09-12T12:19:38.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:38 smithi171 ceph-mon[35098]: pgmap v196: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-12T12:19:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:38 smithi137 ceph-mon[30950]: pgmap v196: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-12T12:19:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:38 smithi189 ceph-mon[35475]: pgmap v196: 97 pgs: 97 active+clean; 154 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 277 op/s 2024-09-12T12:19:39.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 515373 8.57 MB/sec execute 194 sec latency 18.397 ms 2024-09-12T12:19:40.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 516828 8.56 MB/sec execute 195 sec latency 17.338 ms 2024-09-12T12:19:40.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:40 smithi171 ceph-mon[35098]: pgmap v197: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-09-12T12:19:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:40 smithi137 ceph-mon[30950]: pgmap v197: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-09-12T12:19:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:40 smithi189 ceph-mon[35475]: pgmap v197: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 437 op/s 2024-09-12T12:19:41.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 519244 8.59 MB/sec execute 196 sec latency 13.759 ms 2024-09-12T12:19:42.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 521940 8.62 MB/sec execute 197 sec latency 18.727 ms 2024-09-12T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:42 smithi171 ceph-mon[35098]: pgmap v198: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-12T12:19:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:42 smithi137 ceph-mon[30950]: pgmap v198: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-12T12:19:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:42 smithi189 ceph-mon[35475]: pgmap v198: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-12T12:19:43.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 523598 8.62 MB/sec execute 198 sec latency 15.343 ms 2024-09-12T12:19:44.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 526147 8.65 MB/sec execute 199 sec latency 16.714 ms 2024-09-12T12:19:44.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:44 smithi171 ceph-mon[35098]: pgmap v199: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-12T12:19:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:44 smithi137 ceph-mon[30950]: pgmap v199: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-12T12:19:45.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:44 smithi189 ceph-mon[35475]: pgmap v199: 97 pgs: 97 active+clean; 150 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-12T12:19:45.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 527636 8.63 MB/sec execute 200 sec latency 15.112 ms 2024-09-12T12:19:46.734 INFO:tasks.workunit.client.0.smithi137.stdout: 1 529859 8.66 MB/sec execute 201 sec latency 10.859 ms 2024-09-12T12:19:46.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:46 smithi171 ceph-mon[35098]: pgmap v200: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-09-12T12:19:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:46 smithi137 ceph-mon[30950]: pgmap v200: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-09-12T12:19:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:46 smithi189 ceph-mon[35475]: pgmap v200: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-09-12T12:19:47.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 532352 8.68 MB/sec execute 202 sec latency 19.627 ms 2024-09-12T12:19:48.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 534066 8.68 MB/sec execute 203 sec latency 15.316 ms 2024-09-12T12:19:48.954 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:48 smithi171 ceph-mon[35098]: pgmap v201: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-09-12T12:19:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:48 smithi137 ceph-mon[30950]: pgmap v201: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-09-12T12:19:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:48 smithi189 ceph-mon[35475]: pgmap v201: 97 pgs: 97 active+clean; 159 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 263 op/s 2024-09-12T12:19:49.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 536675 8.72 MB/sec execute 204 sec latency 17.887 ms 2024-09-12T12:19:50.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 538260 8.70 MB/sec execute 205 sec latency 18.438 ms 2024-09-12T12:19:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 -- 172.21.15.137:0/4088936253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d010c0d0 msgr2=0x7fe9d010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 --2- 172.21.15.137:0/4088936253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d010c0d0 0x7fe9d010e550 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009ca0 tx=0x7fe9c802a950 comp rx=0 tx=0).stop 2024-09-12T12:19:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 -- 172.21.15.137:0/4088936253 shutdown_connections 2024-09-12T12:19:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 --2- 172.21.15.137:0/4088936253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d010c0d0 0x7fe9d010e550 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 --2- 172.21.15.137:0/4088936253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9d01035a0 0x7fe9d010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 --2- 172.21.15.137:0/4088936253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9d0102c80 0x7fe9d0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 -- 172.21.15.137:0/4088936253 >> 172.21.15.137:0/4088936253 conn(0x7fe9d0077990 msgr2=0x7fe9d00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 -- 172.21.15.137:0/4088936253 shutdown_connections 2024-09-12T12:19:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.909+0000 7fe9d8186640 1 -- 172.21.15.137:0/4088936253 wait complete. 2024-09-12T12:19:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 Processor -- start 2024-09-12T12:19:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 -- start start 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9d0102c80 0x7fe9d0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d01035a0 0x7fe9d01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9d010c0d0 0x7fe9d019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0111eb0 con 0x7fe9d01035a0 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0111d30 con 0x7fe9d0102c80 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d8186640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0112030 con 0x7fe9d010c0d0 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d5721640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d01035a0 0x7fe9d01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d5f22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9d0102c80 0x7fe9d0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d5721640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d01035a0 0x7fe9d01988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44316/0 (socket says 172.21.15.137:44316) 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d6723640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9d010c0d0 0x7fe9d019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.910+0000 7fe9d5721640 1 -- 172.21.15.137:0/436237107 learned_addr learned my addr 172.21.15.137:0/436237107 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d5721640 1 -- 172.21.15.137:0/436237107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9d010c0d0 msgr2=0x7fe9d019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d5721640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9d010c0d0 0x7fe9d019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d5721640 1 -- 172.21.15.137:0/436237107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9d0102c80 msgr2=0x7fe9d0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d5721640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9d0102c80 0x7fe9d0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d5721640 1 -- 172.21.15.137:0/436237107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d019d350 con 0x7fe9d01035a0 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d6723640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9d010c0d0 0x7fe9d019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9d0102c80 0x7fe9d0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d5721640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d01035a0 0x7fe9d01988a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000e9f0 tx=0x7fe9c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9c6ffd640 1 -- 172.21.15.137:0/436237107 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000ccc0 con 0x7fe9d01035a0 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d019d640 con 0x7fe9d01035a0 2024-09-12T12:19:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9c6ffd640 1 -- 172.21.15.137:0/436237107 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c0010910 con 0x7fe9d01035a0 2024-09-12T12:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9c6ffd640 1 -- 172.21.15.137:0/436237107 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0018920 con 0x7fe9d01035a0 2024-09-12T12:19:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.911+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d01a4ed0 con 0x7fe9d01035a0 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.913+0000 7fe9c6ffd640 1 -- 172.21.15.137:0/436237107 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe9c0010430 con 0x7fe9d01035a0 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.913+0000 7fe9daffa740 1 Processor -- start 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9daffa740 1 -- start start 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75c9d0 0x55c0df75cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75d2f0 0x55c0df76bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df76c0c0 0x55c0df76e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9daffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0df6c12f0 con 0x55c0df75c9d0 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9daffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c0df669270 con 0x55c0df75d2f0 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9daffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0df6bd2b0 con 0x55c0df76c0c0 2024-09-12T12:19:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9c6ffd640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe9bc0a2da0 0x7fe9bc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d6723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75c9d0 0x55c0df75cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5721640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df76c0c0 0x55c0df76e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5f22640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75d2f0 0x55c0df76bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d6723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75c9d0 0x55c0df75cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44328/0 (socket says 172.21.15.137:44328) 2024-09-12T12:19:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d6723640 1 -- 172.21.15.137:0/2989235988 learned_addr learned my addr 172.21.15.137:0/2989235988 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75d2f0 0x55c0df76bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37434/0 (socket says 172.21.15.137:37434) 2024-09-12T12:19:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d6723640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe9bc0a2da0 0x7fe9bc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5f22640 1 -- 172.21.15.137:0/2989235988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df76c0c0 msgr2=0x55c0df76e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df76c0c0 0x55c0df76e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5f22640 1 -- 172.21.15.137:0/2989235988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75c9d0 msgr2=0x55c0df75cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75c9d0 0x55c0df75cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9c6ffd640 1 -- 172.21.15.137:0/436237107 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fe9c00cf5b0 con 0x7fe9d01035a0 2024-09-12T12:19:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5f22640 1 -- 172.21.15.137:0/2989235988 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0df668340 con 0x55c0df75d2f0 2024-09-12T12:19:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.914+0000 7fe9d5721640 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df76c0c0 0x55c0df76e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.915+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75d2f0 0x55c0df76bb80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc011440 tx=0x7fe9cc011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df026b37b3941b6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.915+0000 7fe9d6723640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe9bc0a2da0 0x7fe9bc0a5260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8030040 tx=0x7fe9c802a950 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.915+0000 7fe9bbfff640 1 -- 172.21.15.137:0/2989235988 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc004310 con 0x55c0df75d2f0 2024-09-12T12:19:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.915+0000 7fe9bbfff640 1 -- 172.21.15.137:0/2989235988 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9cc0044b0 con 0x55c0df75d2f0 2024-09-12T12:19:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.915+0000 7fe9bbfff640 1 -- 172.21.15.137:0/2989235988 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00fb40 con 0x55c0df75d2f0 2024-09-12T12:19:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 -- 172.21.15.137:0/2989235988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75d2f0 msgr2=0x55c0df76bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75d2f0 0x55c0df76bb80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc011440 tx=0x7fe9cc011900 comp rx=0 tx=0).stop 2024-09-12T12:19:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 -- 172.21.15.137:0/2989235988 shutdown_connections 2024-09-12T12:19:50.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df76c0c0 0x55c0df76e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75d2f0 0x55c0df76bb80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2989235988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75c9d0 0x55c0df75cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 -- 172.21.15.137:0/2989235988 >> 172.21.15.137:0/2989235988 conn(0x55c0df759060 msgr2=0x55c0df759440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 -- 172.21.15.137:0/2989235988 shutdown_connections 2024-09-12T12:19:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 -- 172.21.15.137:0/2989235988 wait complete. 2024-09-12T12:19:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 -- start start 2024-09-12T12:19:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.916+0000 7fe9daffa740 1 Processor -- start 2024-09-12T12:19:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75c9d0 0x55c0df769030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df769570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df76c0c0 0x55c0df769ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9daffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c0df76ecf0 con 0x55c0df75d2f0 2024-09-12T12:19:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9daffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c0df64a290 con 0x55c0df76c0c0 2024-09-12T12:19:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9daffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c0df76ee70 con 0x55c0df75c9d0 2024-09-12T12:19:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9daffa740 10 client.0 _ll_register_callbacks cb 0x55c0df67b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d5721640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df769570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d5721640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df769570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44332/0 (socket says 172.21.15.137:44332) 2024-09-12T12:19:50.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d5721640 1 -- 172.21.15.137:0/1208845426 learned_addr learned my addr 172.21.15.137:0/1208845426 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d6723640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df76c0c0 0x55c0df769ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75c9d0 0x55c0df769030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d6723640 1 -- 172.21.15.137:0/1208845426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75c9d0 msgr2=0x55c0df769030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d6723640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75c9d0 0x55c0df769030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d6723640 1 -- 172.21.15.137:0/1208845426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 msgr2=0x55c0df769570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d6723640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df769570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d6723640 1 -- 172.21.15.137:0/1208845426 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0df3d4210 con 0x55c0df76c0c0 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d5721640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df769570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9d6723640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df76c0c0 0x55c0df769ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005fe0 tx=0x7fe9c80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d1eff4047bf3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.917+0000 7fe9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1208845426 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8043050 con 0x55c0df76c0c0 2024-09-12T12:19:50.930 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9cc0044b0 con 0x55c0df76c0c0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1208845426 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fe9c804b440 con 0x55c0df76c0c0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1208845426 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9c8034040 con 0x55c0df76c0c0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9daffa740 10 client.25616 fetch_fsmap learned FSMap version 30 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c0df4a0620 con 0x55c0df76c0c0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1208845426 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9cc0044b0 con 0x55c0df76c0c0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.918+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1208845426 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe9c805d5a0 con 0x55c0df76c0c0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9daffa740 10 client.25616 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9daffa740 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9daffa740 20 client.25616 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df77c0e0 0x55c0df77e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9daffa740 4 client.25616 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0df6cea20 con 0x55c0df77c0e0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df77c0e0 0x55c0df77e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df77c0e0 0x55c0df77e4d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e978325ffe89715e server_cookie=36478a82cc8fcdd8 in_seq=0 out_seq=0 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.919+0000 7fe9b97fa640 10 client.25616 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1208845426 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0df6cea20 con 0x55c0df77c0e0 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9b97fa640 10 client.25616 handle_command_reply: tid=0 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 client.25616 shutdown 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df76c0c0 msgr2=0x55c0df769ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df76c0c0 0x55c0df769ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005fe0 tx=0x7fe9c80095c0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 shutdown_connections 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df76c0c0 0x55c0df769ab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df77c0e0 0x55c0df77e4d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df769570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1208845426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75c9d0 0x55c0df769030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9b97fa640 0 client.25616 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 >> 172.21.15.137:0/1208845426 conn(0x55c0df759060 msgr2=0x55c0df75edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 shutdown_connections 2024-09-12T12:19:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 1 -- 172.21.15.137:0/1208845426 wait complete. 2024-09-12T12:19:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.920+0000 7fe9daffa740 20 client.25616 trim_cache size 0 max 16384 2024-09-12T12:19:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 Processor -- start 2024-09-12T12:19:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 -- start start 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df759060 0x55c0df76afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df783dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df784300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01add50 con 0x55c0df75d2f0 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01ab5f0 con 0x55c0df75c9d0 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9daffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01a9460 con 0x55c0df759060 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9d6723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df784300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9d6723640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df784300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44346/0 (socket says 172.21.15.137:44346) 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9d6723640 1 -- 172.21.15.137:0/2186670257 learned_addr learned my addr 172.21.15.137:0/2186670257 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df759060 0x55c0df76afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9d5721640 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df783dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.936+0000 7fe9d6723640 1 -- 172.21.15.137:0/2186670257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df759060 msgr2=0x55c0df76afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9d6723640 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df759060 0x55c0df76afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9d6723640 1 -- 172.21.15.137:0/2186670257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 msgr2=0x55c0df783dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9d6723640 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df783dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9d6723640 1 -- 172.21.15.137:0/2186670257 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0df767560 con 0x55c0df75d2f0 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df759060 0x55c0df76afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9d5721640 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df783dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9d6723640 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df784300 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009bf0 tx=0x7fe9c8004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdcbc23e2151c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9bb7fe640 1 -- 172.21.15.137:0/2186670257 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8043070 con 0x55c0df75d2f0 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9bb7fe640 1 -- 172.21.15.137:0/2186670257 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8062020 con 0x55c0df75d2f0 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.937+0000 7fe9bb7fe640 1 -- 172.21.15.137:0/2186670257 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c805d480 con 0x55c0df75d2f0 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- 172.21.15.137:0/2186670257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 msgr2=0x55c0df784300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df784300 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009bf0 tx=0x7fe9c8004410 comp rx=0 tx=0).stop 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- 172.21.15.137:0/2186670257 shutdown_connections 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df75d2f0 0x55c0df784300 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df783dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 --2- 172.21.15.137:0/2186670257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df759060 0x55c0df76afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- 172.21.15.137:0/2186670257 >> 172.21.15.137:0/2186670257 conn(0x55c0df7649e0 msgr2=0x55c0df76a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- 172.21.15.137:0/2186670257 shutdown_connections 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- 172.21.15.137:0/2186670257 wait complete. 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- start start 2024-09-12T12:19:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 Processor -- start 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df759060 0x55c0df76fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df7702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75d2f0 0x55c0df770820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01a9ab0 con 0x55c0df759060 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01aaf20 con 0x55c0df75c9d0 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d01aa180 con 0x55c0df75d2f0 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.938+0000 7fe9daffa740 10 client.0 _ll_register_callbacks cb 0x7fe9c8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df759060 0x55c0df76fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df759060 0x55c0df76fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44356/0 (socket says 172.21.15.137:44356) 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 -- 172.21.15.137:0/1735091949 learned_addr learned my addr 172.21.15.137:0/1735091949 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5721640 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df7702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d6723640 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75d2f0 0x55c0df770820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 -- 172.21.15.137:0/1735091949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75d2f0 msgr2=0x55c0df770820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75d2f0 0x55c0df770820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 -- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 msgr2=0x55c0df7702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df7702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 -- 172.21.15.137:0/1735091949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0df7e5d90 con 0x55c0df759060 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9d5f22640 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df759060 0x55c0df76fda0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55c0df75f6c0 tx=0x7fe9cc009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a6a54a70c4b940 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1735091949 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc01d040 con 0x55c0df759060 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1735091949 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8440+0+0 (secure 0 0 0) 0x7fe9cc005100 con 0x55c0df759060 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c8062020 con 0x55c0df759060 2024-09-12T12:19:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1735091949 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe9cc005300 con 0x55c0df759060 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9daffa740 10 client.16191 fetch_fsmap learned FSMap version 30 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c0df7e7020 con 0x55c0df759060 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.939+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1735091949 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8062020 con 0x55c0df759060 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1735091949 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fe9cc003710 con 0x55c0df759060 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9daffa740 10 client.16191 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9daffa740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9daffa740 20 client.16191 populate_metadata read hostname 'smithi137' 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df76c0c0 0x55c0df7e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9daffa740 4 client.16191 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0df6c8d70 con 0x55c0df76c0c0 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9d5721640 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df76c0c0 0x55c0df7e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9b97fa640 10 client.16191 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:50.940+0000 7fe9d5721640 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df76c0c0 0x55c0df7e9550 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c53559bd4f8aa3 server_cookie=8114a5576095a018 in_seq=0 out_seq=0 2024-09-12T12:19:50.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:50 smithi171 ceph-mon[35098]: pgmap v202: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-09-12T12:19:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9b97fa640 1 -- 172.21.15.137:0/1735091949 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0df6c8d70 con 0x55c0df76c0c0 2024-09-12T12:19:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9b97fa640 10 client.16191 handle_command_reply: tid=0 2024-09-12T12:19:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 client.16191 shutdown 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df759060 msgr2=0x55c0df76fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df759060 0x55c0df76fda0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x55c0df75f6c0 tx=0x7fe9cc009c90 comp rx=0 tx=0).stop 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 shutdown_connections 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0df75d2f0 0x55c0df770820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c0df75c9d0 0x55c0df7702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c0df76c0c0 0x55c0df7e9550 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 --2- 172.21.15.137:0/1735091949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0df759060 0x55c0df76fda0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9b97fa640 0 client.16191 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 >> 172.21.15.137:0/1735091949 conn(0x55c0df7649e0 msgr2=0x55c0df7581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 shutdown_connections 2024-09-12T12:19:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 1 -- 172.21.15.137:0/1735091949 wait complete. 2024-09-12T12:19:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.006+0000 7fe9daffa740 20 client.16191 trim_cache size 0 max 16384 2024-09-12T12:19:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:19:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:19:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:19:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:19:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe9bc0a2da0 msgr2=0x7fe9bc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe9bc0a2da0 0x7fe9bc0a5260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8030040 tx=0x7fe9c802a950 comp rx=0 tx=0).stop 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d01035a0 msgr2=0x7fe9d01988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d01035a0 0x7fe9d01988a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000e9f0 tx=0x7fe9c000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 shutdown_connections 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe9bc0a2da0 0x7fe9bc0a5260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9d010c0d0 0x7fe9d019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe9d01035a0 0x7fe9d01988a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 --2- 172.21.15.137:0/436237107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe9d0102c80 0x7fe9d0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 >> 172.21.15.137:0/436237107 conn(0x7fe9d0077990 msgr2=0x7fe9d00fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 shutdown_connections 2024-09-12T12:19:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:19:51.007+0000 7fe9d8186640 1 -- 172.21.15.137:0/436237107 wait complete. 2024-09-12T12:19:51.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:50 smithi137 ceph-mon[30950]: pgmap v202: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-09-12T12:19:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:50 smithi189 ceph-mon[35475]: pgmap v202: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 359 op/s 2024-09-12T12:19:51.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 540417 8.73 MB/sec execute 206 sec latency 29.237 ms 2024-09-12T12:19:52.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 542428 8.72 MB/sec execute 207 sec latency 18.966 ms 2024-09-12T12:19:52.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:52 smithi171 ceph-mon[35098]: pgmap v203: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-12T12:19:53.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:52 smithi137 ceph-mon[30950]: pgmap v203: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-12T12:19:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:52 smithi189 ceph-mon[35475]: pgmap v203: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-12T12:19:53.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 544243 8.74 MB/sec execute 208 sec latency 14.905 ms 2024-09-12T12:19:54.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 547009 8.78 MB/sec execute 209 sec latency 21.468 ms 2024-09-12T12:19:54.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:54 smithi171 ceph-mon[35098]: pgmap v204: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-12T12:19:55.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:54 smithi137 ceph-mon[30950]: pgmap v204: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-12T12:19:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:54 smithi189 ceph-mon[35475]: pgmap v204: 97 pgs: 97 active+clean; 168 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-12T12:19:55.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 548470 8.77 MB/sec execute 210 sec latency 29.341 ms 2024-09-12T12:19:56.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 550745 8.79 MB/sec execute 211 sec latency 13.143 ms 2024-09-12T12:19:56.892 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:56 smithi189 ceph-mon[35475]: pgmap v205: 97 pgs: 97 active+clean; 164 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 343 op/s 2024-09-12T12:19:56.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:56 smithi171 ceph-mon[35098]: pgmap v205: 97 pgs: 97 active+clean; 164 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 343 op/s 2024-09-12T12:19:57.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:56 smithi137 ceph-mon[30950]: pgmap v205: 97 pgs: 97 active+clean; 164 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.5 MiB/s wr, 343 op/s 2024-09-12T12:19:57.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 552083 8.77 MB/sec execute 212 sec latency 65.143 ms 2024-09-12T12:19:58.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 554530 8.80 MB/sec execute 213 sec latency 11.060 ms 2024-09-12T12:19:58.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:19:58 smithi171 ceph-mon[35098]: pgmap v206: 97 pgs: 97 active+clean; 164 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-09-12T12:19:59.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:19:58 smithi137 ceph-mon[30950]: pgmap v206: 97 pgs: 97 active+clean; 164 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-09-12T12:19:59.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:19:58 smithi189 ceph-mon[35475]: pgmap v206: 97 pgs: 97 active+clean; 164 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 240 op/s 2024-09-12T12:19:59.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 555828 8.78 MB/sec execute 214 sec latency 44.873 ms 2024-09-12T12:20:00.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 557779 8.80 MB/sec execute 215 sec latency 20.758 ms 2024-09-12T12:20:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:00 smithi171 ceph-mon[35098]: pgmap v207: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-09-12T12:20:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:00 smithi171 ceph-mon[35098]: overall HEALTH_OK 2024-09-12T12:20:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:00 smithi137 ceph-mon[30950]: pgmap v207: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-09-12T12:20:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:00 smithi137 ceph-mon[30950]: overall HEALTH_OK 2024-09-12T12:20:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:00 smithi189 ceph-mon[35475]: pgmap v207: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 352 op/s 2024-09-12T12:20:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:00 smithi189 ceph-mon[35475]: overall HEALTH_OK 2024-09-12T12:20:01.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 559090 8.79 MB/sec execute 216 sec latency 13.916 ms 2024-09-12T12:20:01.884 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-12T12:20:01.884 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:20:02.154 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:20:02.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.705+0000 7f55bf941640 1 -- 172.21.15.137:0/3366572126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8106a80 msgr2=0x7f55b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.705+0000 7f55bf941640 1 --2- 172.21.15.137:0/3366572126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8106a80 0x7f55b8106e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f55b401a8a0 tx=0x7f55b403b3a0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.706+0000 7f55bf941640 1 -- 172.21.15.137:0/3366572126 shutdown_connections 2024-09-12T12:20:02.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.706+0000 7f55bf941640 1 --2- 172.21.15.137:0/3366572126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b8101420 0x7f55b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.706+0000 7f55bf941640 1 --2- 172.21.15.137:0/3366572126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8100a80 0x7f55b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.706+0000 7f55bf941640 1 --2- 172.21.15.137:0/3366572126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8106a80 0x7f55b8106e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.706+0000 7f55bf941640 1 -- 172.21.15.137:0/3366572126 >> 172.21.15.137:0/3366572126 conn(0x7f55b80fc7a0 msgr2=0x7f55b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:02.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.707+0000 7f55bf941640 1 -- 172.21.15.137:0/3366572126 shutdown_connections 2024-09-12T12:20:02.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.707+0000 7f55bf941640 1 -- 172.21.15.137:0/3366572126 wait complete. 2024-09-12T12:20:02.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.708+0000 7f55bf941640 1 Processor -- start 2024-09-12T12:20:02.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.708+0000 7f55bf941640 1 -- start start 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.708+0000 7f55bf941640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b8100a80 0x7f55b8073a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.708+0000 7f55bf941640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8101420 0x7f55b806eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.708+0000 7f55bf941640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8106a80 0x7f55b806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.708+0000 7f55bf941640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55b806d450 con 0x7f55b8100a80 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bf941640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55b806d2d0 con 0x7f55b8106a80 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bf941640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55b806d5d0 con 0x7f55b8101420 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8106a80 0x7f55b806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8106a80 0x7f55b806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37020/0 (socket says 172.21.15.137:37020) 2024-09-12T12:20:02.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 -- 172.21.15.137:0/271280254 learned_addr learned my addr 172.21.15.137:0/271280254 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bcedc640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8101420 0x7f55b806eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 -- 172.21.15.137:0/271280254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8101420 msgr2=0x7f55b806eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8101420 0x7f55b806eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 -- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b8100a80 msgr2=0x7f55b8073a60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b8100a80 0x7f55b8073a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.709+0000 7f55bdede640 1 -- 172.21.15.137:0/271280254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b806f730 con 0x7f55b8106a80 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.710+0000 7f55bdede640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8106a80 0x7f55b806f030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f55b000efc0 tx=0x7f55b000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.710+0000 7f55ae7fc640 1 -- 172.21.15.137:0/271280254 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0003970 con 0x7f55b8106a80 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.710+0000 7f55ae7fc640 1 -- 172.21.15.137:0/271280254 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b0004020 con 0x7f55b8106a80 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.710+0000 7f55ae7fc640 1 -- 172.21.15.137:0/271280254 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b000b9b0 con 0x7f55b8106a80 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.710+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b8115580 con 0x7f55b8106a80 2024-09-12T12:20:02.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.710+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b8115a70 con 0x7f55b8106a80 2024-09-12T12:20:02.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.712+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b81021c0 con 0x7f55b8106a80 2024-09-12T12:20:02.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.714+0000 7f55ae7fc640 1 -- 172.21.15.137:0/271280254 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f55b001d020 con 0x7f55b8106a80 2024-09-12T12:20:02.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.715+0000 7f55ae7fc640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55a40a2e30 0x7f55a40a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:02.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.715+0000 7f55ae7fc640 1 -- 172.21.15.137:0/271280254 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f55b00cf3d0 con 0x7f55b8106a80 2024-09-12T12:20:02.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.716+0000 7f55bd6dd640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55a40a2e30 0x7f55a40a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:02.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.717+0000 7f55bd6dd640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55a40a2e30 0x7f55a40a52f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55b40057c0 tx=0x7f55b4005c90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:02.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:02.723+0000 7f55ae7fc640 1 -- 172.21.15.137:0/271280254 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55b000a2f0 con 0x7f55b8106a80 2024-09-12T12:20:02.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 561321 8.81 MB/sec execute 217 sec latency 54.461 ms 2024-09-12T12:20:02.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:02 smithi171 ceph-mon[35098]: pgmap v208: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-09-12T12:20:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:02 smithi137 ceph-mon[30950]: pgmap v208: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-09-12T12:20:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:02 smithi189 ceph-mon[35475]: pgmap v208: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-09-12T12:20:03.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.140+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55b8108a60 con 0x7f55b8106a80 2024-09-12T12:20:03.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.142+0000 7f55ae7fc640 1 -- 172.21.15.137:0/271280254 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11937 (secure 0 0 0) 0x7f55b009bfb0 con 0x7f55b8106a80 2024-09-12T12:20:03.143 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:20:03.143 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":30,"btime":"2024-09-12T12:19:33:591767+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:19:33.155386+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25087":{"gid":25087,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3680297976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3680297976},{"type":"v1","addr":"172.21.15.189:6835","nonce":3680297976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:20:03.143 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 30 2024-09-12T12:20:03.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.145+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55a40a2e30 msgr2=0x7f55a40a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.145+0000 7f55bf941640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55a40a2e30 0x7f55a40a52f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55b40057c0 tx=0x7f55b4005c90 comp rx=0 tx=0).stop 2024-09-12T12:20:03.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.145+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8106a80 msgr2=0x7f55b806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:03.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.145+0000 7f55bf941640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8106a80 0x7f55b806f030 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f55b000efc0 tx=0x7f55b000c800 comp rx=0 tx=0).stop 2024-09-12T12:20:03.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.146+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 shutdown_connections 2024-09-12T12:20:03.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.146+0000 7f55bf941640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f55a40a2e30 0x7f55a40a52f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.146+0000 7f55bf941640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55b8106a80 0x7f55b806f030 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.146+0000 7f55bf941640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55b8101420 0x7f55b806eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.146+0000 7f55bf941640 1 --2- 172.21.15.137:0/271280254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55b8100a80 0x7f55b8073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:03.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.146+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 >> 172.21.15.137:0/271280254 conn(0x7f55b80fc7a0 msgr2=0x7f55b80fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:03.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.146+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 shutdown_connections 2024-09-12T12:20:03.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:03.147+0000 7f55bf941640 1 -- 172.21.15.137:0/271280254 wait complete. 2024-09-12T12:20:03.704 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:03 smithi137 ceph-mon[30950]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T12:20:03.705 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:03 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/271280254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:03.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 562043 8.77 MB/sec execute 218 sec latency 28.002 ms 2024-09-12T12:20:03.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:03 smithi171 ceph-mon[35098]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T12:20:03.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:03 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/271280254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:03 smithi189 ceph-mon[35475]: osdmap e87: 12 total, 12 up, 12 in 2024-09-12T12:20:04.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:03 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/271280254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.126+0000 7f55081b7640 1 -- 172.21.15.137:0/4165051276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 msgr2=0x7f5500100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.126+0000 7f55081b7640 1 --2- 172.21.15.137:0/4165051276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500100e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009860 tx=0x7f54f402a340 comp rx=0 tx=0).stop 2024-09-12T12:20:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.126+0000 7f55081b7640 1 -- 172.21.15.137:0/4165051276 shutdown_connections 2024-09-12T12:20:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.126+0000 7f55081b7640 1 --2- 172.21.15.137:0/4165051276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5500101350 0x7f550010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.126+0000 7f55081b7640 1 --2- 172.21.15.137:0/4165051276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500100e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.126+0000 7f55081b7640 1 --2- 172.21.15.137:0/4165051276 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55001069c0 0x7f5500106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.126+0000 7f55081b7640 1 -- 172.21.15.137:0/4165051276 >> 172.21.15.137:0/4165051276 conn(0x7f5500077990 msgr2=0x7f55000fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.127+0000 7f55081b7640 1 -- 172.21.15.137:0/4165051276 shutdown_connections 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.127+0000 7f55081b7640 1 -- 172.21.15.137:0/4165051276 wait complete. 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.127+0000 7f55081b7640 1 Processor -- start 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.127+0000 7f55081b7640 1 -- start start 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.127+0000 7f55081b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f55081b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5500101350 0x7f55001988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f55081b7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55001069c0 0x7f550019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f55081b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5500111eb0 con 0x7f5500101350 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f55081b7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5500111d30 con 0x7f55001009b0 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f55081b7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5500112030 con 0x7f55001069c0 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5500101350 0x7f55001988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37032/0 (socket says 172.21.15.137:37032) 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5500101350 0x7f55001988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33036/0 (socket says 172.21.15.137:33036) 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 -- 172.21.15.137:0/726693386 learned_addr learned my addr 172.21.15.137:0/726693386 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5506754640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55001069c0 0x7f550019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 -- 172.21.15.137:0/726693386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55001069c0 msgr2=0x7f550019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55001069c0 0x7f550019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 -- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5500101350 msgr2=0x7f55001988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5500101350 0x7f55001988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 -- 172.21.15.137:0/726693386 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550019d350 con 0x7f55001009b0 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505752640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5500101350 0x7f55001988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f5505f53640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500198360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e9f0 tx=0x7f54f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f54feffd640 1 -- 172.21.15.137:0/726693386 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f800cd80 con 0x7f55001009b0 2024-09-12T12:20:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550019d640 con 0x7f55001009b0 2024-09-12T12:20:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f54feffd640 1 -- 172.21.15.137:0/726693386 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f8010910 con 0x7f55001009b0 2024-09-12T12:20:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f54feffd640 1 -- 172.21.15.137:0/726693386 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8018990 con 0x7f55001009b0 2024-09-12T12:20:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.128+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55001a4ed0 con 0x7f55001009b0 2024-09-12T12:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 Processor -- start 2024-09-12T12:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f54feffd640 1 -- 172.21.15.137:0/726693386 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f54f8010ac0 con 0x7f55001009b0 2024-09-12T12:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 -- start start 2024-09-12T12:20:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa46ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa51770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556d9a9ab2f0 con 0x556d9aa47410 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556d9a953270 con 0x556d9aa51cb0 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f550b02d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556d9a9a72b0 con 0x556d9aa46af0 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f5505f53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa51770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f5505752640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f5505752640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37048/0 (socket says 172.21.15.137:37048) 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f5505f53640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa51770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33052/0 (socket says 172.21.15.137:33052) 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.131+0000 7f5505752640 1 -- 172.21.15.137:0/2039375372 learned_addr learned my addr 172.21.15.137:0/2039375372 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5506754640 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa46ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5505752640 1 -- 172.21.15.137:0/2039375372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 msgr2=0x556d9aa46ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5505752640 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa46ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5505752640 1 -- 172.21.15.137:0/2039375372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 msgr2=0x556d9aa51770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5505752640 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa51770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5505752640 1 -- 172.21.15.137:0/2039375372 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9a952340 con 0x556d9aa51cb0 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5505752640 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa540a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54f4005e00 tx=0x7f54f4009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c63dc2ab687b63e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f54feffd640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54e80a2da0 0x7f54e80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f54e7fff640 1 -- 172.21.15.137:0/2039375372 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4037040 con 0x556d9aa51cb0 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f5506754640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54e80a2da0 0x7f54e80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f54feffd640 1 -- 172.21.15.137:0/726693386 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f54f80cf720 con 0x7f55001009b0 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f54e7fff640 1 -- 172.21.15.137:0/2039375372 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f4030780 con 0x556d9aa51cb0 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.132+0000 7f54e7fff640 1 -- 172.21.15.137:0/2039375372 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4003d50 con 0x556d9aa51cb0 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 -- 172.21.15.137:0/2039375372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 msgr2=0x556d9aa540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f5506754640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54e80a2da0 0x7f54e80a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005fd0 tx=0x7f54f0010040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa540a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f54f4005e00 tx=0x7f54f4009510 comp rx=0 tx=0).stop 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 -- 172.21.15.137:0/2039375372 shutdown_connections 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa540a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa51770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 --2- 172.21.15.137:0/2039375372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa46ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 -- 172.21.15.137:0/2039375372 >> 172.21.15.137:0/2039375372 conn(0x556d9aa43060 msgr2=0x556d9aa43440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 -- 172.21.15.137:0/2039375372 shutdown_connections 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.133+0000 7f550b02d740 1 -- 172.21.15.137:0/2039375372 wait complete. 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 -- start start 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 Processor -- start 2024-09-12T12:20:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa4f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa4f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556d9aa58c90 con 0x556d9aa51cb0 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556d9a934290 con 0x556d9aa46af0 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556d9aa58e10 con 0x556d9aa47410 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f5505f53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f5505f53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa4eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37054/0 (socket says 172.21.15.137:37054) 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f5505f53640 1 -- 172.21.15.137:0/3227762012 learned_addr learned my addr 172.21.15.137:0/3227762012 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f550b02d740 10 client.0 _ll_register_callbacks cb 0x556d9a965140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.134+0000 7f5505752640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa4f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5506754640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa4f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5505f53640 1 -- 172.21.15.137:0/3227762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 msgr2=0x556d9aa4f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5505f53640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa4f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5505f53640 1 -- 172.21.15.137:0/3227762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 msgr2=0x556d9aa4f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5505f53640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa4f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5505f53640 1 -- 172.21.15.137:0/3227762012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9a6be210 con 0x556d9aa46af0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5506754640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa4f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f5505f53640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa4eb00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c6a0 tx=0x7f54f8098040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc645a8b4f5a5d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f54e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f54e57fa640 1 -- 172.21.15.137:0/3227762012 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80cd2c0 con 0x556d9aa46af0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4030780 con 0x556d9aa46af0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f54e57fa640 1 -- 172.21.15.137:0/3227762012 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f54f809b100 con 0x556d9aa46af0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f54e57fa640 1 -- 172.21.15.137:0/3227762012 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54f809bcb0 con 0x556d9aa46af0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f550b02d740 10 client.25640 fetch_fsmap learned FSMap version 30 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d9a78a620 con 0x556d9aa46af0 2024-09-12T12:20:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.135+0000 7f54e57fa640 1 -- 172.21.15.137:0/3227762012 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f4030780 con 0x556d9aa46af0 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f54e57fa640 1 -- 172.21.15.137:0/3227762012 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f54f8097a00 con 0x556d9aa46af0 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f550b02d740 10 client.25640 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f550b02d740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f550b02d740 20 client.25640 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f550b02d740 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa68420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f550b02d740 4 client.25640 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d9a9b8a20 con 0x556d9aa65fd0 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f5505752640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa68420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f5505752640 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa68420 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557bc1e009fd0908 server_cookie=8ed32d89595ae44b in_seq=0 out_seq=0 2024-09-12T12:20:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.136+0000 7f54e57fa640 10 client.25640 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.137+0000 7f54e57fa640 1 -- 172.21.15.137:0/3227762012 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d9a9b8a20 con 0x556d9aa65fd0 2024-09-12T12:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.137+0000 7f54e57fa640 10 client.25640 handle_command_reply: tid=0 2024-09-12T12:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.137+0000 7f550b02d740 1 client.25640 shutdown 2024-09-12T12:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.137+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 msgr2=0x556d9aa4eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.137+0000 7f550b02d740 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa4eb00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c6a0 tx=0x7f54f8098040 comp rx=0 tx=0).stop 2024-09-12T12:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 shutdown_connections 2024-09-12T12:20:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa68420 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa4f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f54e57fa640 0 client.25640 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa4f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 --2- 172.21.15.137:0/3227762012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa4eb00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 >> 172.21.15.137:0/3227762012 conn(0x556d9aa43060 msgr2=0x556d9aa48dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 shutdown_connections 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 1 -- 172.21.15.137:0/3227762012 wait complete. 2024-09-12T12:20:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.138+0000 7f550b02d740 20 client.25640 trim_cache size 0 max 16384 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 Processor -- start 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 -- start start 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa50a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa6dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa6e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55001adf60 con 0x556d9aa51cb0 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55001ab800 con 0x556d9aa46af0 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.153+0000 7f550b02d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55001a9670 con 0x556d9aa47410 2024-09-12T12:20:04.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505f53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa50a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa6dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5506754640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa6e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa6dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:51246/0 (socket says 172.21.15.137:51246) 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505f53640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa50a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37058/0 (socket says 172.21.15.137:37058) 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 -- 172.21.15.137:0/3772938624 learned_addr learned my addr 172.21.15.137:0/3772938624 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 -- 172.21.15.137:0/3772938624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 msgr2=0x556d9aa50a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 --2- 172.21.15.137:0/3772938624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa50a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 -- 172.21.15.137:0/3772938624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 msgr2=0x556d9aa6e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 --2- 172.21.15.137:0/3772938624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa6e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 -- 172.21.15.137:0/3772938624 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9aa42e90 con 0x556d9aa47410 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f5505752640 1 --2- 172.21.15.137:0/3772938624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa6dd30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009860 tx=0x7f54f402c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1cd92dcb796a085 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f54e77fe640 1 -- 172.21.15.137:0/3772938624 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4037040 con 0x556d9aa47410 2024-09-12T12:20:04.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f54e77fe640 1 -- 172.21.15.137:0/3772938624 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f4052070 con 0x556d9aa47410 2024-09-12T12:20:04.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.154+0000 7f54e77fe640 1 -- 172.21.15.137:0/3772938624 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f403d440 con 0x556d9aa47410 2024-09-12T12:20:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 -- 172.21.15.137:0/3772938624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 msgr2=0x556d9aa6dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 --2- 172.21.15.137:0/3772938624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa6dd30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009860 tx=0x7f54f402c040 comp rx=0 tx=0).stop 2024-09-12T12:20:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 -- 172.21.15.137:0/3772938624 shutdown_connections 2024-09-12T12:20:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 --2- 172.21.15.137:0/3772938624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa51cb0 0x556d9aa6e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 --2- 172.21.15.137:0/3772938624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa47410 0x556d9aa6dd30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 --2- 172.21.15.137:0/3772938624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa46af0 0x556d9aa50a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 -- 172.21.15.137:0/3772938624 >> 172.21.15.137:0/3772938624 conn(0x556d9aa43060 msgr2=0x556d9aa50360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 -- 172.21.15.137:0/3772938624 shutdown_connections 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 -- 172.21.15.137:0/3772938624 wait complete. 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.155+0000 7f550b02d740 1 -- start start 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 1 Processor -- start 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa5e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55001a9cc0 con 0x556d9aa47410 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55001ab130 con 0x556d9aa51cb0 2024-09-12T12:20:04.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55001aa390 con 0x556d9aa46af0 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f5505f53640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa5e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f5505752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f550b02d740 10 client.0 _ll_register_callbacks cb 0x7f54f800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f5505f53640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa5e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:51260/0 (socket says 172.21.15.137:51260) 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f5505752640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33072/0 (socket says 172.21.15.137:33072) 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f5505f53640 1 -- 172.21.15.137:0/591686495 learned_addr learned my addr 172.21.15.137:0/591686495 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.156+0000 7f5506754640 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa59f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f5505752640 1 -- 172.21.15.137:0/591686495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 msgr2=0x556d9aa5e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f5505752640 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa5e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f5505752640 1 -- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 msgr2=0x556d9aa59f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f5505752640 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa59f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f5505752640 1 -- 172.21.15.137:0/591686495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d9aa5a610 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f5505f53640 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa5e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f5505752640 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa599e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f54f402c010 tx=0x7f54f402aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f7d2887e5f71aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f54e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f54e57fa640 1 -- 172.21.15.137:0/591686495 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4060070 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f54e57fa640 1 -- 172.21.15.137:0/591686495 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8528+0+0 (secure 0 0 0) 0x7f54f405cb20 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f4052070 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f54e57fa640 1 -- 172.21.15.137:0/591686495 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54f4037070 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f550b02d740 10 client.16212 fetch_fsmap learned FSMap version 30 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d9aa54640 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f54e57fa640 1 -- 172.21.15.137:0/591686495 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f4052070 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f54e57fa640 1 -- 172.21.15.137:0/591686495 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f54f403d440 con 0x556d9aa47410 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f550b02d740 10 client.16212 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T12:20:04.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.157+0000 7f550b02d740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.158+0000 7f550b02d740 20 client.16212 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.158+0000 7f550b02d740 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.158+0000 7f550b02d740 4 client.16212 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:20:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.158+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d9a9b2d70 con 0x556d9aa65fd0 2024-09-12T12:20:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.158+0000 7f5505f53640 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.158+0000 7f54e57fa640 10 client.16212 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:04.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.158+0000 7f5505f53640 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa569b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96fa5b03c1201ec server_cookie=f52ad783caade0e in_seq=0 out_seq=0 2024-09-12T12:20:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f54e57fa640 1 -- 172.21.15.137:0/591686495 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d9a9b2d70 con 0x556d9aa65fd0 2024-09-12T12:20:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f54e57fa640 10 client.16212 handle_command_reply: tid=0 2024-09-12T12:20:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 client.16212 shutdown 2024-09-12T12:20:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 msgr2=0x556d9aa599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa599e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f54f402c010 tx=0x7f54f402aac0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 shutdown_connections 2024-09-12T12:20:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556d9aa51cb0 0x556d9aa59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556d9aa65fd0 0x556d9aa569b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556d9aa47410 0x556d9aa599e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 --2- 172.21.15.137:0/591686495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d9aa46af0 0x556d9aa5e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.226+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 >> 172.21.15.137:0/591686495 conn(0x556d9aa43060 msgr2=0x556d9aa48d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.227+0000 7f54e57fa640 0 client.16212 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.227+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 shutdown_connections 2024-09-12T12:20:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.227+0000 7f550b02d740 1 -- 172.21.15.137:0/591686495 wait complete. 2024-09-12T12:20:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.227+0000 7f550b02d740 20 client.16212 trim_cache size 0 max 16384 2024-09-12T12:20:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:20:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:20:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:20:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:20:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54e80a2da0 msgr2=0x7f54e80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54e80a2da0 0x7f54e80a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005fd0 tx=0x7f54f0010040 comp rx=0 tx=0).stop 2024-09-12T12:20:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 msgr2=0x7f5500198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500198360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54f800e9f0 tx=0x7f54f800eeb0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 shutdown_connections 2024-09-12T12:20:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54e80a2da0 0x7f54e80a5260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55001069c0 0x7f550019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5500101350 0x7f55001988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 --2- 172.21.15.137:0/726693386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f55001009b0 0x7f5500198360 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.228+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 >> 172.21.15.137:0/726693386 conn(0x7f5500077990 msgr2=0x7f5500103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.229+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 shutdown_connections 2024-09-12T12:20:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:04.229+0000 7f55081b7640 1 -- 172.21.15.137:0/726693386 wait complete. 2024-09-12T12:20:04.310 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-12T12:20:04.311 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-12T12:20:04.586 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:20:04.590 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:20:04.590 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:20:04.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:04 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/1977221445 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be07ae9860 con 0x55be07ac7400 2024-09-12T12:20:04.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:20:04.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 564420 8.79 MB/sec execute 219 sec latency 40.667 ms 2024-09-12T12:20:04.858 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:04 smithi137 ceph-mon[30950]: pgmap v210: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-09-12T12:20:04.858 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:04 smithi137 ceph-mon[30950]: daemon mds.f restarted 2024-09-12T12:20:04.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:04 smithi171 ceph-mon[35098]: pgmap v210: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-09-12T12:20:04.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:04 smithi171 ceph-mon[35098]: daemon mds.f restarted 2024-09-12T12:20:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:04 smithi189 ceph-mon[35475]: pgmap v210: 97 pgs: 97 active+clean; 181 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 307 op/s 2024-09-12T12:20:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:04 smithi189 ceph-mon[35475]: daemon mds.f restarted 2024-09-12T12:20:05.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 -- 172.21.15.137:0/914993596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 msgr2=0x7f8808100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 --2- 172.21.15.137:0/914993596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808100ed0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009930 tx=0x7f87f402a420 comp rx=0 tx=0).stop 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 -- 172.21.15.137:0/914993596 shutdown_connections 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 --2- 172.21.15.137:0/914993596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8808101410 0x7f880810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 --2- 172.21.15.137:0/914993596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808100ed0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 --2- 172.21.15.137:0/914993596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8808106a70 0x7f8808106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 -- 172.21.15.137:0/914993596 >> 172.21.15.137:0/914993596 conn(0x7f88080fc7d0 msgr2=0x7f88080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 -- 172.21.15.137:0/914993596 shutdown_connections 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 -- 172.21.15.137:0/914993596 wait complete. 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.130+0000 7f880eba2640 1 Processor -- start 2024-09-12T12:20:05.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880eba2640 1 -- start start 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880eba2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880eba2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8808101410 0x7f8808198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880eba2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8808106a70 0x7f880819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880eba2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110e70 con 0x7f8808100a70 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880eba2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110cf0 con 0x7f8808101410 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880eba2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8808110ff0 con 0x7f8808106a70 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33094/0 (socket says 172.21.15.137:33094) 2024-09-12T12:20:05.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 -- 172.21.15.137:0/191346986 learned_addr learned my addr 172.21.15.137:0/191346986 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f8807fff640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8808101410 0x7f8808198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880d13f640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8808106a70 0x7f880819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 -- 172.21.15.137:0/191346986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8808106a70 msgr2=0x7f880819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8808106a70 0x7f880819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 -- 172.21.15.137:0/191346986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8808101410 msgr2=0x7f8808198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8808101410 0x7f8808198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 -- 172.21.15.137:0/191346986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880819d3b0 con 0x7f8808100a70 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.131+0000 7f880c93e640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808198400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c920 tx=0x7f87fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.132+0000 7f8805ffb640 1 -- 172.21.15.137:0/191346986 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc007c70 con 0x7f8808100a70 2024-09-12T12:20:05.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.132+0000 7f8805ffb640 1 -- 172.21.15.137:0/191346986 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc007e10 con 0x7f8808100a70 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.132+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880819d6a0 con 0x7f8808100a70 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.132+0000 7f8805ffb640 1 -- 172.21.15.137:0/191346986 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc018890 con 0x7f8808100a70 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.132+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081a4f80 con 0x7f8808100a70 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8805ffb640 1 -- 172.21.15.137:0/191346986 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f87fc018a70 con 0x7f8808100a70 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 Processor -- start 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 -- start start 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21639b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21644400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b21646d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b2159f2a0 con 0x564b21639760 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564b21546b10 con 0x564b21644940 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f8810f7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b2159b260 con 0x564b2163a080 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21639b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f880c93e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21644400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21639b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33106/0 (socket says 172.21.15.137:33106) 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.135+0000 7f880d13f640 1 -- 172.21.15.137:0/2921611720 learned_addr learned my addr 172.21.15.137:0/2921611720 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f8807fff640 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b21646d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f880d13f640 1 -- 172.21.15.137:0/2921611720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 msgr2=0x564b21644400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f880d13f640 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21644400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f880d13f640 1 -- 172.21.15.137:0/2921611720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 msgr2=0x564b21646d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f880d13f640 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b21646d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f880d13f640 1 -- 172.21.15.137:0/2921611720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b21310b90 con 0x564b21639760 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f880d13f640 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21639b40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fd0 tx=0x7f88000118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e62091393240c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f8805ffb640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f87f80a2cf0 0x7f87f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f880c93e640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f87f80a2cf0 0x7f87f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f87f2ffd640 1 -- 172.21.15.137:0/2921611720 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800004d20 con 0x564b21639760 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f87f2ffd640 1 -- 172.21.15.137:0/2921611720 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800004050 con 0x564b21639760 2024-09-12T12:20:05.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f87f2ffd640 1 -- 172.21.15.137:0/2921611720 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800005280 con 0x564b21639760 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.136+0000 7f8805ffb640 1 -- 172.21.15.137:0/191346986 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f87fc0cf480 con 0x7f8808100a70 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 -- 172.21.15.137:0/2921611720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 msgr2=0x564b21639b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f880c93e640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f87f80a2cf0 0x7f87f80a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ccca0 tx=0x7f87fc0d5060 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21639b40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8800009fd0 tx=0x7f88000118d0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 -- 172.21.15.137:0/2921611720 shutdown_connections 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b21646d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21644400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 --2- 172.21.15.137:0/2921611720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21639b40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 -- 172.21.15.137:0/2921611720 >> 172.21.15.137:0/2921611720 conn(0x564b21635cf0 msgr2=0x564b216360d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:05.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 -- 172.21.15.137:0/2921611720 shutdown_connections 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 -- 172.21.15.137:0/2921611720 wait complete. 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 -- start start 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.137+0000 7f8810f7b740 1 Processor -- start 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21641750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b21641c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21644940 0x564b216421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8810f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564b21647510 con 0x564b21639760 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8810f7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564b215276e0 con 0x564b2163a080 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8810f7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564b21647690 con 0x564b21644940 2024-09-12T12:20:05.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8810f7b740 10 client.0 _ll_register_callbacks cb 0x564b2154ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21641750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21641750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33120/0 (socket says 172.21.15.137:33120) 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 -- 172.21.15.137:0/1091296563 learned_addr learned my addr 172.21.15.137:0/1091296563 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f8807fff640 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b21641c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21644940 0x564b216421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 -- 172.21.15.137:0/1091296563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21644940 msgr2=0x564b216421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21644940 0x564b216421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 -- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 msgr2=0x564b21641c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b21641c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 -- 172.21.15.137:0/1091296563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b21650980 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f880d13f640 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21641750 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8800009de0 tx=0x7f8800004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace1a855f2616962 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.138+0000 7f87e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f87e7fff640 1 -- 172.21.15.137:0/1091296563 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800004610 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f87e7fff640 1 -- 172.21.15.137:0/1091296563 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f880000f4e0 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8800004050 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f87e7fff640 1 -- 172.21.15.137:0/1091296563 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8800037050 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 10 client.16230 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564b216532b0 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f87e7fff640 1 -- 172.21.15.137:0/1091296563 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800004050 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f87e7fff640 1 -- 172.21.15.137:0/1091296563 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f8800028780 con 0x564b21639760 2024-09-12T12:20:05.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 10 client.16230 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 10 client.16230 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 20 client.16230 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b21654770 0x564b21656b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 4 client.16230 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b215ac9d0 con 0x564b21654770 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.139+0000 7f8807fff640 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b21654770 0x564b21656b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f87e7fff640 10 client.16230 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8807fff640 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b21654770 0x564b21656b60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bee6f3235d63d0 server_cookie=a0379a72441dac7c in_seq=0 out_seq=0 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f87e7fff640 1 -- 172.21.15.137:0/1091296563 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b215ac9d0 con 0x564b21654770 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f87e7fff640 10 client.16230 handle_command_reply: tid=0 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 client.16230 shutdown 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 msgr2=0x564b21641750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21641750 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8800009de0 tx=0x7f8800004810 comp rx=0 tx=0).stop 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 shutdown_connections 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21644940 0x564b216421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b21641c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b21654770 0x564b21656b60 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f87e7fff640 0 client.16230 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1091296563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21641750 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.140+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 >> 172.21.15.137:0/1091296563 conn(0x564b21635cf0 msgr2=0x564b21634e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.141+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 shutdown_connections 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.141+0000 7f8810f7b740 1 -- 172.21.15.137:0/1091296563 wait complete. 2024-09-12T12:20:05.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.141+0000 7f8810f7b740 20 client.16230 trim_cache size 0 max 16384 2024-09-12T12:20:05.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 Processor -- start 2024-09-12T12:20:05.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 -- start start 2024-09-12T12:20:05.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21639760 0x564b21643620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b2165c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21644940 0x564b2165c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b0010 con 0x564b21644940 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ad8b0 con 0x564b2163a080 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8810f7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ab700 con 0x564b21639760 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21644940 0x564b2165c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f8807fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b2165c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21644940 0x564b2165c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33136/0 (socket says 172.21.15.137:33136) 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f880d13f640 1 -- 172.21.15.137:0/1099676170 learned_addr learned my addr 172.21.15.137:0/1099676170 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.158+0000 7f880d13f640 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21639760 0x564b21643620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f880d13f640 1 -- 172.21.15.137:0/1099676170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21639760 msgr2=0x564b21643620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f880d13f640 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21639760 0x564b21643620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f880d13f640 1 -- 172.21.15.137:0/1099676170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 msgr2=0x564b2165c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f880d13f640 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b2165c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f880d13f640 1 -- 172.21.15.137:0/1099676170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b21635b20 con 0x564b21644940 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f8807fff640 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b2165c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f880d13f640 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21644940 0x564b2165c7c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8800005d70 tx=0x7f88000118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd5a6e5002a1d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f87f27fc640 1 -- 172.21.15.137:0/1099676170 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800004410 con 0x564b21644940 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f87f27fc640 1 -- 172.21.15.137:0/1099676170 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800028930 con 0x564b21644940 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.159+0000 7f87f27fc640 1 -- 172.21.15.137:0/1099676170 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880000f970 con 0x564b21644940 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 -- 172.21.15.137:0/1099676170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21644940 msgr2=0x564b2165c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21644940 0x564b2165c7c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8800005d70 tx=0x7f88000118d0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 -- 172.21.15.137:0/1099676170 shutdown_connections 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21644940 0x564b2165c7c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b2163a080 0x564b2165c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 --2- 172.21.15.137:0/1099676170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b21639760 0x564b21643620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 -- 172.21.15.137:0/1099676170 >> 172.21.15.137:0/1099676170 conn(0x564b21635cf0 msgr2=0x564b2163b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 -- 172.21.15.137:0/1099676170 shutdown_connections 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 -- 172.21.15.137:0/1099676170 wait complete. 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 -- start start 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 Processor -- start 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.160+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21648a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21648f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8810f7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b216be250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8810f7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88081abd70 con 0x564b21639760 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8810f7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ad1e0 con 0x564b21644940 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8810f7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ac440 con 0x564b2163a080 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8810f7b740 10 client.0 _ll_register_callbacks cb 0x7f8800003bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21648a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21648a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33138/0 (socket says 172.21.15.137:33138) 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 -- 172.21.15.137:0/974555746 learned_addr learned my addr 172.21.15.137:0/974555746 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8807fff640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21648f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b216be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 -- 172.21.15.137:0/974555746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 msgr2=0x564b21648f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21648f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 -- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 msgr2=0x564b216be250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b216be250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 -- 172.21.15.137:0/974555746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b21631260 con 0x564b21639760 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8807fff640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21648f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f880d13f640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21648a10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8800003af0 tx=0x7f880000f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a7bc1885c03eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f87e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f87e7fff640 1 -- 172.21.15.137:0/974555746 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800004620 con 0x564b21639760 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f87e7fff640 1 -- 172.21.15.137:0/974555746 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8484+0+0 (secure 0 0 0) 0x7f8800028450 con 0x564b21639760 2024-09-12T12:20:05.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.161+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8800028930 con 0x564b21639760 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f87e7fff640 1 -- 172.21.15.137:0/974555746 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8800028650 con 0x564b21639760 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 10 client.16236 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564b216bf6b0 con 0x564b21639760 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f87e7fff640 1 -- 172.21.15.137:0/974555746 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800028930 con 0x564b21639760 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f87e7fff640 1 -- 172.21.15.137:0/974555746 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f8800047560 con 0x564b21639760 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 10 client.16236 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 10 client.16236 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 20 client.16236 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b216bfc80 0x564b216c2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 4 client.16236 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x564b215a6d20 con 0x564b216bfc80 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.162+0000 7f8807fff640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b216bfc80 0x564b216c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:05.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.163+0000 7f87e7fff640 10 client.16236 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:05.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.163+0000 7f8807fff640 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b216bfc80 0x564b216c2070 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b14ba4b6f53e059 server_cookie=e52d1b805421788 in_seq=0 out_seq=0 2024-09-12T12:20:05.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f87e7fff640 1 -- 172.21.15.137:0/974555746 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564b215a6d20 con 0x564b216bfc80 2024-09-12T12:20:05.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f87e7fff640 10 client.16236 handle_command_reply: tid=0 2024-09-12T12:20:05.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 client.16236 shutdown 2024-09-12T12:20:05.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 msgr2=0x564b21648a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21648a10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8800003af0 tx=0x7f880000f5f0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 shutdown_connections 2024-09-12T12:20:05.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564b21644940 0x564b216be250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564b2163a080 0x564b21648f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564b216bfc80 0x564b216c2070 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 --2- 172.21.15.137:0/974555746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b21639760 0x564b21648a10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f87e7fff640 0 client.16236 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 >> 172.21.15.137:0/974555746 conn(0x564b21635cf0 msgr2=0x564b2164f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 shutdown_connections 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 1 -- 172.21.15.137:0/974555746 wait complete. 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.240+0000 7f8810f7b740 20 client.16236 trim_cache size 0 max 16384 2024-09-12T12:20:05.241 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 32, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 26, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "8063b8e7": { 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 26, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 136.6, 2024-09-12T12:20:05.242 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:05.243 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f87f80a2cf0 msgr2=0x7f87f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f87f80a2cf0 0x7f87f80a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ccca0 tx=0x7f87fc0d5060 comp rx=0 tx=0).stop 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 msgr2=0x7f8808198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808198400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00c920 tx=0x7f87fc00cde0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 shutdown_connections 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f87f80a2cf0 0x7f87f80a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8808106a70 0x7f880819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8808101410 0x7f8808198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 --2- 172.21.15.137:0/191346986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8808100a70 0x7f8808198400 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:05.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 >> 172.21.15.137:0/191346986 conn(0x7f88080fc7d0 msgr2=0x7f880806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 shutdown_connections 2024-09-12T12:20:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:05.241+0000 7f880eba2640 1 -- 172.21.15.137:0/191346986 wait complete. 2024-09-12T12:20:05.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.78 MB/sec execute 220 sec latency 573.114 ms 2024-09-12T12:20:05.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:05 smithi171 ceph-mon[35098]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T12:20:05.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:05 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/905550172,v1:172.21.15.189:6835/905550172] up:boot 2024-09-12T12:20:05.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:05 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:20:05.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:05 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:20:05.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:05 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:20:06.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:05 smithi137 ceph-mon[30950]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T12:20:06.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:05 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/905550172,v1:172.21.15.189:6835/905550172] up:boot 2024-09-12T12:20:06.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:05 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:20:06.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:05 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:20:06.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:05 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:20:06.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:05 smithi189 ceph-mon[35475]: osdmap e88: 12 total, 12 up, 12 in 2024-09-12T12:20:06.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:05 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/905550172,v1:172.21.15.189:6835/905550172] up:boot 2024-09-12T12:20:06.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:05 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:20:06.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:05 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:20:06.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:05 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:20:06.142 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8063b8e7', quiesce took 4.3 seconds. Will release after: 12.3 2024-09-12T12:20:06.735 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.74 MB/sec execute 221 sec latency 1573.164 ms 2024-09-12T12:20:06.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:06 smithi171 ceph-mon[35098]: pgmap v212: 97 pgs: 97 active+clean; 173 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 445 op/s 2024-09-12T12:20:07.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:06 smithi137 ceph-mon[30950]: pgmap v212: 97 pgs: 97 active+clean; 173 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 445 op/s 2024-09-12T12:20:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:06 smithi189 ceph-mon[35475]: pgmap v212: 97 pgs: 97 active+clean; 173 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.7 MiB/s wr, 445 op/s 2024-09-12T12:20:07.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.70 MB/sec execute 222 sec latency 2573.244 ms 2024-09-12T12:20:08.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.66 MB/sec execute 223 sec latency 3573.301 ms 2024-09-12T12:20:08.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:08 smithi189 ceph-mon[35475]: pgmap v213: 97 pgs: 97 active+clean; 173 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-09-12T12:20:08.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:08 smithi189 ceph-mon[35475]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T12:20:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:08 smithi171 ceph-mon[35098]: pgmap v213: 97 pgs: 97 active+clean; 173 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-09-12T12:20:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:08 smithi171 ceph-mon[35098]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T12:20:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:08 smithi137 ceph-mon[30950]: pgmap v213: 97 pgs: 97 active+clean; 173 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 275 op/s 2024-09-12T12:20:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:08 smithi137 ceph-mon[30950]: osdmap e89: 12 total, 12 up, 12 in 2024-09-12T12:20:09.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.62 MB/sec execute 224 sec latency 4573.381 ms 2024-09-12T12:20:10.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.58 MB/sec execute 225 sec latency 5573.498 ms 2024-09-12T12:20:10.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:10 smithi189 ceph-mon[35475]: pgmap v215: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-09-12T12:20:10.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:10 smithi171 ceph-mon[35098]: pgmap v215: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-09-12T12:20:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:10 smithi137 ceph-mon[30950]: pgmap v215: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.1 MiB/s wr, 416 op/s 2024-09-12T12:20:11.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.54 MB/sec execute 226 sec latency 6573.579 ms 2024-09-12T12:20:12.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.51 MB/sec execute 227 sec latency 7573.675 ms 2024-09-12T12:20:12.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:12 smithi189 ceph-mon[35475]: pgmap v216: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-09-12T12:20:12.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:12 smithi171 ceph-mon[35098]: pgmap v216: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-09-12T12:20:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:12 smithi137 ceph-mon[30950]: pgmap v216: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-09-12T12:20:13.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.47 MB/sec execute 228 sec latency 8573.773 ms 2024-09-12T12:20:14.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.43 MB/sec execute 229 sec latency 9573.878 ms 2024-09-12T12:20:14.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:14 smithi189 ceph-mon[35475]: pgmap v217: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-09-12T12:20:14.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:14 smithi171 ceph-mon[35098]: pgmap v217: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-09-12T12:20:15.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:14 smithi137 ceph-mon[30950]: pgmap v217: 97 pgs: 97 active+clean; 174 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-09-12T12:20:15.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.40 MB/sec execute 230 sec latency 10573.976 ms 2024-09-12T12:20:16.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.36 MB/sec execute 231 sec latency 11574.039 ms 2024-09-12T12:20:16.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:16 smithi189 ceph-mon[35475]: pgmap v218: 97 pgs: 97 active+clean; 177 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-09-12T12:20:16.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:16 smithi171 ceph-mon[35098]: pgmap v218: 97 pgs: 97 active+clean; 177 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-09-12T12:20:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:16 smithi137 ceph-mon[30950]: pgmap v218: 97 pgs: 97 active+clean; 177 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-09-12T12:20:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 -- 172.21.15.137:0/103760374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc103660 msgr2=0x7f48fc10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 --2- 172.21.15.137:0/103760374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc103660 0x7f48fc10bb40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009900 tx=0x7f48f002a480 comp rx=0 tx=0).stop 2024-09-12T12:20:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 -- 172.21.15.137:0/103760374 shutdown_connections 2024-09-12T12:20:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 --2- 172.21.15.137:0/103760374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48fc10c0d0 0x7f48fc10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 --2- 172.21.15.137:0/103760374 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc103660 0x7f48fc10bb40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 --2- 172.21.15.137:0/103760374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48fc102d40 0x7f48fc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 -- 172.21.15.137:0/103760374 >> 172.21.15.137:0/103760374 conn(0x7f48fc0fc6f0 msgr2=0x7f48fc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 -- 172.21.15.137:0/103760374 shutdown_connections 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 -- 172.21.15.137:0/103760374 wait complete. 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.327+0000 7f4903023640 1 Processor -- start 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4903023640 1 -- start start 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4903023640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc102d40 0x7f48fc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4903023640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48fc103660 0x7f48fc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4903023640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48fc10c0d0 0x7f48fc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4903023640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc111ed0 con 0x7f48fc102d40 2024-09-12T12:20:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4903023640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc111d50 con 0x7f48fc103660 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4903023640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc112050 con 0x7f48fc10c0d0 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc102d40 0x7f48fc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc102d40 0x7f48fc1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40166/0 (socket says 172.21.15.137:40166) 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 -- 172.21.15.137:0/3180769247 learned_addr learned my addr 172.21.15.137:0/3180769247 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f49015c0640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48fc10c0d0 0x7f48fc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f48fbfff640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48fc103660 0x7f48fc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 -- 172.21.15.137:0/3180769247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48fc10c0d0 msgr2=0x7f48fc19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48fc10c0d0 0x7f48fc19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 -- 172.21.15.137:0/3180769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48fc103660 msgr2=0x7f48fc1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48fc103660 0x7f48fc1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 -- 172.21.15.137:0/3180769247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48fc19d2a0 con 0x7f48fc102d40 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f48fbfff640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48fc103660 0x7f48fc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.328+0000 7f4900dbf640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc102d40 0x7f48fc1982b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c920 tx=0x7f48ec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.329+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3180769247 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec007c70 con 0x7f48fc102d40 2024-09-12T12:20:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.329+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3180769247 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ec007e10 con 0x7f48fc102d40 2024-09-12T12:20:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.329+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3180769247 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec018890 con 0x7f48fc102d40 2024-09-12T12:20:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.329+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48fc19d590 con 0x7f48fc102d40 2024-09-12T12:20:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.329+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48fc1a4e20 con 0x7f48fc102d40 2024-09-12T12:20:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.331+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3180769247 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f48ec018a70 con 0x7f48fc102d40 2024-09-12T12:20:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 Processor -- start 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 -- start start 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f48f9ffb640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f48e40a2e20 0x7f48e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f49015c0640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f48e40a2e20 0x7f48e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563b277092f0 con 0x563b277b40c0 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563b276b1270 con 0x563b277a49b0 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4905dbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b277052b0 con 0x563b277a52d0 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f48f9ffb640 1 -- 172.21.15.137:0/3180769247 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7f48ec0cf4f0 con 0x7f48fc102d40 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f49015c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f49015c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40178/0 (socket says 172.21.15.137:40178) 2024-09-12T12:20:17.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f49015c0640 1 -- 172.21.15.137:0/2588718282 learned_addr learned my addr 172.21.15.137:0/2588718282 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f4900dbf640 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f48fbfff640 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.332+0000 7f49015c0640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f48e40a2e20 0x7f48e40a52e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f48f4014820 tx=0x7f48f4014c20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f49015c0640 1 -- 172.21.15.137:0/2588718282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 msgr2=0x563b277b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f49015c0640 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f49015c0640 1 -- 172.21.15.137:0/2588718282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 msgr2=0x563b277a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f49015c0640 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f49015c0640 1 -- 172.21.15.137:0/2588718282 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b276b0340 con 0x563b277b40c0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f49015c0640 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277b64b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f48f4012ac0 tx=0x7f48f401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb993eca45923c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f48e2ffd640 1 -- 172.21.15.137:0/2588718282 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4003b70 con 0x563b277b40c0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f48e2ffd640 1 -- 172.21.15.137:0/2588718282 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f401ed90 con 0x563b277b40c0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.333+0000 7f48e2ffd640 1 -- 172.21.15.137:0/2588718282 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4020440 con 0x563b277b40c0 2024-09-12T12:20:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 -- 172.21.15.137:0/2588718282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 msgr2=0x563b277b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277b64b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f48f4012ac0 tx=0x7f48f401e000 comp rx=0 tx=0).stop 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 -- 172.21.15.137:0/2588718282 shutdown_connections 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277b64b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 --2- 172.21.15.137:0/2588718282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 -- 172.21.15.137:0/2588718282 >> 172.21.15.137:0/2588718282 conn(0x563b277a1060 msgr2=0x563b277a1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 -- 172.21.15.137:0/2588718282 shutdown_connections 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 -- 172.21.15.137:0/2588718282 wait complete. 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 -- start start 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.334+0000 7f4905dbd740 1 Processor -- start 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a49b0 0x563b277b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a52d0 0x563b277b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277b40c0 0x563b277b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563b277b6c30 con 0x563b277a52d0 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563b27692290 con 0x563b277b40c0 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b277b6db0 con 0x563b277a49b0 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 10 client.0 _ll_register_callbacks cb 0x563b276c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a52d0 0x563b277b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a52d0 0x563b277b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40186/0 (socket says 172.21.15.137:40186) 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 -- 172.21.15.137:0/4168727730 learned_addr learned my addr 172.21.15.137:0/4168727730 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f49015c0640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277b40c0 0x563b277b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4900dbf640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a49b0 0x563b277b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 -- 172.21.15.137:0/4168727730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a49b0 msgr2=0x563b277b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a49b0 0x563b277b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 -- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277b40c0 msgr2=0x563b277b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277b40c0 0x563b277b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 -- 172.21.15.137:0/4168727730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b2741c210 con 0x563b277a52d0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f49015c0640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277b40c0 0x563b277b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48fbfff640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a52d0 0x563b277b1500 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f48f0031550 tx=0x7f48f00318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f73682625a93723 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48dbfff640 1 -- 172.21.15.137:0/4168727730 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0035070 con 0x563b277a52d0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48dbfff640 1 -- 172.21.15.137:0/4168727730 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7f48f0004240 con 0x563b277a52d0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f401ed90 con 0x563b277a52d0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.335+0000 7f48dbfff640 1 -- 172.21.15.137:0/4168727730 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f48f0004440 con 0x563b277a52d0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 10 client.16248 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b274e8620 con 0x563b277a52d0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f48dbfff640 1 -- 172.21.15.137:0/4168727730 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f401ed90 con 0x563b277a52d0 2024-09-12T12:20:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f48dbfff640 1 -- 172.21.15.137:0/4168727730 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f48f004b020 con 0x563b277a52d0 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 10 client.16248 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 20 client.16248 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b277c3fb0 0x563b277c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 4 client.16248 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b27716a20 con 0x563b277c3fb0 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.336+0000 7f4900dbf640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b277c3fb0 0x563b277c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f48dbfff640 10 client.16248 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4900dbf640 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b277c3fb0 0x563b277c63a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59e2d26c0d3f7a04 server_cookie=6ed1e66af19715d9 in_seq=0 out_seq=0 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f48dbfff640 1 -- 172.21.15.137:0/4168727730 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b27716a20 con 0x563b277c3fb0 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f48dbfff640 10 client.16248 handle_command_reply: tid=0 2024-09-12T12:20:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 client.16248 shutdown 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a52d0 msgr2=0x563b277b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a52d0 0x563b277b1500 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f48f0031550 tx=0x7f48f00318d0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 shutdown_connections 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277b40c0 0x563b277b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b277c3fb0 0x563b277c63a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a52d0 0x563b277b1500 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 --2- 172.21.15.137:0/4168727730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a49b0 0x563b277b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f48dbfff640 0 client.16248 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.337+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 >> 172.21.15.137:0/4168727730 conn(0x563b277a1060 msgr2=0x563b277a01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.338+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 shutdown_connections 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.338+0000 7f4905dbd740 1 -- 172.21.15.137:0/4168727730 wait complete. 2024-09-12T12:20:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.338+0000 7f4905dbd740 20 client.16248 trim_cache size 0 max 16384 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 Processor -- start 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 -- start start 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a49b0 0x563b277b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a52d0 0x563b277cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277b40c0 0x563b277cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1add80 con 0x563b277a49b0 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1ab620 con 0x563b277a52d0 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4905dbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1a9490 con 0x563b277b40c0 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a49b0 0x563b277b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a49b0 0x563b277b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40196/0 (socket says 172.21.15.137:40196) 2024-09-12T12:20:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 -- 172.21.15.137:0/3275292474 learned_addr learned my addr 172.21.15.137:0/3275292474 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f49015c0640 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277b40c0 0x563b277cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 -- 172.21.15.137:0/3275292474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277b40c0 msgr2=0x563b277cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277b40c0 0x563b277cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 -- 172.21.15.137:0/3275292474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a52d0 msgr2=0x563b277cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a52d0 0x563b277cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.352+0000 7f4900dbf640 1 -- 172.21.15.137:0/3275292474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b277af4f0 con 0x563b277a49b0 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4900dbf640 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a49b0 0x563b277b1870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f48ec098400 tx=0x7f48ec00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dde3525d79529e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f48e27fc640 1 -- 172.21.15.137:0/3275292474 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0a46a0 con 0x563b277a49b0 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f48e27fc640 1 -- 172.21.15.137:0/3275292474 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ec099d30 con 0x563b277a49b0 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f48e27fc640 1 -- 172.21.15.137:0/3275292474 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ec060340 con 0x563b277a49b0 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 -- 172.21.15.137:0/3275292474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a49b0 msgr2=0x563b277b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a49b0 0x563b277b1870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f48ec098400 tx=0x7f48ec00ff30 comp rx=0 tx=0).stop 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 -- 172.21.15.137:0/3275292474 shutdown_connections 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277b40c0 0x563b277cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a52d0 0x563b277cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 --2- 172.21.15.137:0/3275292474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277a49b0 0x563b277b1870 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 -- 172.21.15.137:0/3275292474 >> 172.21.15.137:0/3275292474 conn(0x563b277a1060 msgr2=0x563b277b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 -- 172.21.15.137:0/3275292474 shutdown_connections 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.353+0000 7f4905dbd740 1 -- 172.21.15.137:0/3275292474 wait complete. 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 -- start start 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 Processor -- start 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277bc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277bc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1a9ae0 con 0x563b277b40c0 2024-09-12T12:20:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1aaf50 con 0x563b277a49b0 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48fc1aa1b0 con 0x563b277a52d0 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f4905dbd740 10 client.0 _ll_register_callbacks cb 0x7f48f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f49015c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277bc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f49015c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277bc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40212/0 (socket says 172.21.15.137:40212) 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f49015c0640 1 -- 172.21.15.137:0/69736587 learned_addr learned my addr 172.21.15.137:0/69736587 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.354+0000 7f48fbfff640 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f4900dbf640 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f49015c0640 1 -- 172.21.15.137:0/69736587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 msgr2=0x563b277bc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f49015c0640 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277bc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f49015c0640 1 -- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 msgr2=0x563b277ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f49015c0640 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f49015c0640 1 -- 172.21.15.137:0/69736587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b277bcc70 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f49015c0640 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277bc5b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x563b277a2150 tx=0x7f48f4004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536a193767940cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f48dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f48dbfff640 1 -- 172.21.15.137:0/69736587 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4003970 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f48dbfff640 1 -- 172.21.15.137:0/69736587 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7f48f4003b10 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48ec099d30 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f48dbfff640 1 -- 172.21.15.137:0/69736587 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f48f4003d30 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f4905dbd740 10 client.16254 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b27833510 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f48dbfff640 1 -- 172.21.15.137:0/69736587 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48ec099d30 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.355+0000 7f48dbfff640 1 -- 172.21.15.137:0/69736587 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f48f4038070 con 0x563b277b40c0 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4905dbd740 10 client.16254 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4905dbd740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4905dbd740 20 client.16254 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4905dbd740 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b27833900 0x563b27835cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4905dbd740 4 client.16254 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:20:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b27710d70 con 0x563b27833900 2024-09-12T12:20:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4900dbf640 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b27833900 0x563b27835cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f48dbfff640 10 client.16254 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.356+0000 7f4900dbf640 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b27833900 0x563b27835cf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=518af4002d07e23d server_cookie=d2c0d75b19f539f9 in_seq=0 out_seq=0 2024-09-12T12:20:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f48dbfff640 1 -- 172.21.15.137:0/69736587 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b27710d70 con 0x563b27833900 2024-09-12T12:20:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f48dbfff640 10 client.16254 handle_command_reply: tid=0 2024-09-12T12:20:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 client.16254 shutdown 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 msgr2=0x563b277bc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277bc5b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x563b277a2150 tx=0x7f48f4004f00 comp rx=0 tx=0).stop 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 shutdown_connections 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b27833900 0x563b27835cf0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b277b40c0 0x563b277bc5b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f48dbfff640 0 client.16254 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b277a52d0 0x563b277bc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 --2- 172.21.15.137:0/69736587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b277a49b0 0x563b277ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 >> 172.21.15.137:0/69736587 conn(0x563b277a1060 msgr2=0x563b2779f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 shutdown_connections 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 1 -- 172.21.15.137:0/69736587 wait complete. 2024-09-12T12:20:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.399+0000 7f4905dbd740 20 client.16254 trim_cache size 0 max 16384 2024-09-12T12:20:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:20:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:20:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:20:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f48e40a2e20 msgr2=0x7f48e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f48e40a2e20 0x7f48e40a52e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f48f4014820 tx=0x7f48f4014c20 comp rx=0 tx=0).stop 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc102d40 msgr2=0x7f48fc1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc102d40 0x7f48fc1982b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c920 tx=0x7f48ec00cde0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 shutdown_connections 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f48e40a2e20 0x7f48e40a52e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48fc10c0d0 0x7f48fc19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48fc103660 0x7f48fc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 --2- 172.21.15.137:0/3180769247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f48fc102d40 0x7f48fc1982b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 >> 172.21.15.137:0/3180769247 conn(0x7f48fc0fc6f0 msgr2=0x7f48fc0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 shutdown_connections 2024-09-12T12:20:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:17.400+0000 7f4903023640 1 -- 172.21.15.137:0/3180769247 wait complete. 2024-09-12T12:20:17.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.32 MB/sec execute 232 sec latency 12574.154 ms 2024-09-12T12:20:18.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:20:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:20:18.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.29 MB/sec execute 233 sec latency 13574.191 ms 2024-09-12T12:20:18.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:18 smithi137 ceph-mon[30950]: pgmap v219: 97 pgs: 97 active+clean; 177 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-09-12T12:20:18.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:18 smithi189 ceph-mon[35475]: pgmap v219: 97 pgs: 97 active+clean; 177 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-09-12T12:20:18.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:18 smithi171 ceph-mon[35098]: pgmap v219: 97 pgs: 97 active+clean; 177 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 2.4 MiB/s wr, 104 op/s 2024-09-12T12:20:19.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 -- 172.21.15.137:0/1186395183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 msgr2=0x7f0ef4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 --2- 172.21.15.137:0/1186395183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef4106e60 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009930 tx=0x7f0ee402a420 comp rx=0 tx=0).stop 2024-09-12T12:20:19.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 -- 172.21.15.137:0/1186395183 shutdown_connections 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 --2- 172.21.15.137:0/1186395183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ef4101420 0x7f0ef410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 --2- 172.21.15.137:0/1186395183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef4100a80 0x7f0ef4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 --2- 172.21.15.137:0/1186395183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef4106e60 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 -- 172.21.15.137:0/1186395183 >> 172.21.15.137:0/1186395183 conn(0x7f0ef40fc7a0 msgr2=0x7f0ef40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 -- 172.21.15.137:0/1186395183 shutdown_connections 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.191+0000 7f0efaf31640 1 -- 172.21.15.137:0/1186395183 wait complete. 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 Processor -- start 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 -- start start 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ef4100a80 0x7f0ef4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef4101420 0x7f0ef4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4110ea0 con 0x7f0ef4106a80 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4110d20 con 0x7f0ef4101420 2024-09-12T12:20:19.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0efaf31640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111020 con 0x7f0ef4100a80 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0ef94ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0ef94ce640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53666/0 (socket says 172.21.15.137:53666) 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0ef94ce640 1 -- 172.21.15.137:0/2554843835 learned_addr learned my addr 172.21.15.137:0/2554843835 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0ef8ccd640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ef4100a80 0x7f0ef4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.192+0000 7f0ef3fff640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef4101420 0x7f0ef4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef94ce640 1 -- 172.21.15.137:0/2554843835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ef4100a80 msgr2=0x7f0ef4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef94ce640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ef4100a80 0x7f0ef4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef94ce640 1 -- 172.21.15.137:0/2554843835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef4101420 msgr2=0x7f0ef4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef94ce640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef4101420 0x7f0ef4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef94ce640 1 -- 172.21.15.137:0/2554843835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef419d3e0 con 0x7f0ef4106a80 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef94ce640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef419cd00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00b970 tx=0x7f0eec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef1ffb640 1 -- 172.21.15.137:0/2554843835 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec019070 con 0x7f0ef4106a80 2024-09-12T12:20:19.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef419d6d0 con 0x7f0ef4106a80 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef1ffb640 1 -- 172.21.15.137:0/2554843835 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eec00cb40 con 0x7f0ef4106a80 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0ef1ffb640 1 -- 172.21.15.137:0/2554843835 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec007610 con 0x7f0ef4106a80 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.193+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a4fb0 con 0x7f0ef4106a80 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.194+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef4100ee0 con 0x7f0ef4106a80 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.195+0000 7f0ef1ffb640 1 -- 172.21.15.137:0/2554843835 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0eec01c070 con 0x7f0ef4106a80 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.196+0000 7f0ef1ffb640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0edc0a2cf0 0x7f0edc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.196+0000 7f0ef1ffb640 1 -- 172.21.15.137:0/2554843835 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7f0eec0ce8d0 con 0x7f0ef4106a80 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.196+0000 7f0ef8ccd640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0edc0a2cf0 0x7f0edc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.196+0000 7f0ef8ccd640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0edc0a2cf0 0x7f0edc0a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009900 tx=0x7f0ee40058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:19.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.200+0000 7f0ef1ffb640 1 -- 172.21.15.137:0/2554843835 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0eec09afd0 con 0x7f0ef4106a80 2024-09-12T12:20:19.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.402+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ef410a060 con 0x7f0ef4106a80 2024-09-12T12:20:19.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.403+0000 7f0ef1ffb640 1 -- 172.21.15.137:0/2554843835 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11934 (secure 0 0 0) 0x7f0eec09f2d0 con 0x7f0ef4106a80 2024-09-12T12:20:19.404 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:20:19.404 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":32,"btime":"2024-09-12T12:20:04:645234+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:20:04.645208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25714":{"gid":25714,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/905550172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":905550172},{"type":"v1","addr":"172.21.15.189:6835","nonce":905550172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:20:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 32 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0edc0a2cf0 msgr2=0x7f0edc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0edc0a2cf0 0x7f0edc0a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009900 tx=0x7f0ee40058d0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 msgr2=0x7f0ef419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef419cd00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00b970 tx=0x7f0eec00be30 comp rx=0 tx=0).stop 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 shutdown_connections 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0edc0a2cf0 0x7f0edc0a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ef4106a80 0x7f0ef419cd00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ef4101420 0x7f0ef4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 --2- 172.21.15.137:0/2554843835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ef4100a80 0x7f0ef4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 >> 172.21.15.137:0/2554843835 conn(0x7f0ef40fc7a0 msgr2=0x7f0ef406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 shutdown_connections 2024-09-12T12:20:19.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:19.407+0000 7f0efaf31640 1 -- 172.21.15.137:0/2554843835 wait complete. 2024-09-12T12:20:19.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.25 MB/sec execute 234 sec latency 14574.285 ms 2024-09-12T12:20:19.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:19 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2554843835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:19.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:19 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2554843835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:19.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:19 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2554843835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:20.218 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 8063b8e7 --release --await' 2024-09-12T12:20:20.218 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 8063b8e7 --release --await 2024-09-12T12:20:20.466 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:20:20.736 INFO:tasks.workunit.client.0.smithi137.stdout: 1 565122 8.22 MB/sec execute 235 sec latency 15574.311 ms 2024-09-12T12:20:20.874 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:20 smithi137 ceph-mon[30950]: pgmap v220: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 2.2 MiB/s wr, 121 op/s 2024-09-12T12:20:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:20 smithi171 ceph-mon[35098]: pgmap v220: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 2.2 MiB/s wr, 121 op/s 2024-09-12T12:20:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 -- 172.21.15.137:0/1195778690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac101de0 msgr2=0x7fa8ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 --2- 172.21.15.137:0/1195778690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac101de0 0x7fa8ac1021c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009900 tx=0x7fa89c02a410 comp rx=0 tx=0).stop 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 -- 172.21.15.137:0/1195778690 shutdown_connections 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 --2- 172.21.15.137:0/1195778690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8ac106f10 0x7fa8ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 --2- 172.21.15.137:0/1195778690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8ac102790 0x7fa8ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 --2- 172.21.15.137:0/1195778690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac101de0 0x7fa8ac1021c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 -- 172.21.15.137:0/1195778690 >> 172.21.15.137:0/1195778690 conn(0x7fa8ac0fd8c0 msgr2=0x7fa8ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 -- 172.21.15.137:0/1195778690 shutdown_connections 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.960+0000 7fa8b258a640 1 -- 172.21.15.137:0/1195778690 wait complete. 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 Processor -- start 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 -- start start 2024-09-12T12:20:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8ac101de0 0x7fa8ac198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8ac102790 0x7fa8ac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac106f10 0x7fa8ac19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac111f90 con 0x7fa8ac106f10 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac111e10 con 0x7fa8ac102790 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b258a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac112110 con 0x7fa8ac101de0 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b0b27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac106f10 0x7fa8ac19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b0b27640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac106f10 0x7fa8ac19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53690/0 (socket says 172.21.15.137:53690) 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8b0b27640 1 -- 172.21.15.137:0/3384003521 learned_addr learned my addr 172.21.15.137:0/3384003521 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8ac102790 0x7fa8ac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.961+0000 7fa8abfff640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8ac101de0 0x7fa8ac198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b0b27640 1 -- 172.21.15.137:0/3384003521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8ac101de0 msgr2=0x7fa8ac198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b0b27640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8ac101de0 0x7fa8ac198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b0b27640 1 -- 172.21.15.137:0/3384003521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8ac102790 msgr2=0x7fa8ac198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b0b27640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8ac102790 0x7fa8ac198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b0b27640 1 -- 172.21.15.137:0/3384003521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ac19d400 con 0x7fa8ac106f10 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b0b27640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac106f10 0x7fa8ac19cd20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400cce0 tx=0x7fa8a400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8a97fa640 1 -- 172.21.15.137:0/3384003521 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a400f450 con 0x7fa8ac106f10 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ac19d6f0 con 0x7fa8ac106f10 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8a97fa640 1 -- 172.21.15.137:0/3384003521 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a400fad0 con 0x7fa8ac106f10 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8a97fa640 1 -- 172.21.15.137:0/3384003521 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4018b60 con 0x7fa8ac106f10 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.962+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ac1a4fd0 con 0x7fa8ac106f10 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8a97fa640 1 -- 172.21.15.137:0/3384003521 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa8a400fc80 con 0x7fa8ac106f10 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 Processor -- start 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 -- start start 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b060 0x56143cd9b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56143cd00620 con 0x56143cd9b980 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56143cca7e50 con 0x56143cda6240 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8b4963740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56143ccfc5e0 con 0x56143cd9b060 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53698/0 (socket says 172.21.15.137:53698) 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8abfff640 1 -- 172.21.15.137:0/247367272 learned_addr learned my addr 172.21.15.137:0/247367272 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8a97fa640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8940a2d70 0x7fa8940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b060 0x56143cd9b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.965+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8b0b27640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8940a2d70 0x7fa8940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8abfff640 1 -- 172.21.15.137:0/247367272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b060 msgr2=0x56143cd9b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8abfff640 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b060 0x56143cd9b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8abfff640 1 -- 172.21.15.137:0/247367272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 msgr2=0x56143cda8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8abfff640 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8abfff640 1 -- 172.21.15.137:0/247367272 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143cc98d40 con 0x56143cd9b980 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8a97fa640 1 -- 172.21.15.137:0/3384003521 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7fa8a40cf880 con 0x7fa8ac106f10 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8abfff640 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda5d00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009900 tx=0x7fa89c0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fc719b5d1e335d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8927fc640 1 -- 172.21.15.137:0/247367272 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c040070 con 0x56143cd9b980 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8b0b27640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8940a2d70 0x7fa8940a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa8a409ba60 tx=0x7fa8a406bbc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8927fc640 1 -- 172.21.15.137:0/247367272 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa89c005020 con 0x56143cd9b980 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.966+0000 7fa8927fc640 1 -- 172.21.15.137:0/247367272 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c035040 con 0x56143cd9b980 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 -- 172.21.15.137:0/247367272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 msgr2=0x56143cda5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda5d00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009900 tx=0x7fa89c0316e0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 -- 172.21.15.137:0/247367272 shutdown_connections 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda5d00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 --2- 172.21.15.137:0/247367272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b060 0x56143cd9b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 -- 172.21.15.137:0/247367272 >> 172.21.15.137:0/247367272 conn(0x56143cd975f0 msgr2=0x56143cd979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 -- 172.21.15.137:0/247367272 shutdown_connections 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 -- 172.21.15.137:0/247367272 wait complete. 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 -- start start 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.967+0000 7fa8b4963740 1 Processor -- start 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cd9b060 0x56143cda3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cda6240 0x56143cda3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56143cda8e10 con 0x56143cd9b980 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56143cc88f10 con 0x56143cd9b060 2024-09-12T12:20:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56143cda8f90 con 0x56143cda6240 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 10 client.0 _ll_register_callbacks cb 0x56143ccacdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53712/0 (socket says 172.21.15.137:53712) 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 -- 172.21.15.137:0/987023265 learned_addr learned my addr 172.21.15.137:0/987023265 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cda6240 0x56143cda3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 -- 172.21.15.137:0/987023265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cda6240 msgr2=0x56143cda3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cda6240 0x56143cda3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 -- 172.21.15.137:0/987023265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cd9b060 msgr2=0x56143cda3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cd9b060 0x56143cda3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 -- 172.21.15.137:0/987023265 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143ca0f580 con 0x56143cd9b980 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cda6240 0x56143cda3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8abfff640 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda3660 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0057b0 tx=0x7fa89c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba00aac01b94a921 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa88b7fe640 1 -- 172.21.15.137:0/987023265 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c04c020 con 0x56143cd9b980 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa88b7fe640 1 -- 172.21.15.137:0/987023265 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7fa89c0049e0 con 0x56143cd9b980 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.968+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa89c005020 con 0x56143cd9b980 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa88b7fe640 1 -- 172.21.15.137:0/987023265 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa89c057050 con 0x56143cd9b980 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 10 client.16272 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56143cad9960 con 0x56143cd9b980 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa88b7fe640 1 -- 172.21.15.137:0/987023265 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa89c005020 con 0x56143cd9b980 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa88b7fe640 1 -- 172.21.15.137:0/987023265 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fa89c044400 con 0x56143cd9b980 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 10 client.16272 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 10 client.16272 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 20 client.16272 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143cdb5c40 0x56143cdb8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 4 client.16272 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56143cd0db90 con 0x56143cdb5c40 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.969+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143cdb5c40 0x56143cdb8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.970+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143cdb5c40 0x56143cdb8030 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab3699a46e755ac server_cookie=98b4d238abdc234 in_seq=0 out_seq=0 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.970+0000 7fa88b7fe640 10 client.16272 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.970+0000 7fa88b7fe640 1 -- 172.21.15.137:0/987023265 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56143cd0db90 con 0x56143cdb5c40 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.970+0000 7fa88b7fe640 10 client.16272 handle_command_reply: tid=0 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.970+0000 7fa8b4963740 1 client.16272 shutdown 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 msgr2=0x56143cda3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda3660 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa89c0057b0 tx=0x7fa89c02a410 comp rx=0 tx=0).stop 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 shutdown_connections 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cda6240 0x56143cda3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143cdb5c40 0x56143cdb8030 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b980 0x56143cda3660 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 --2- 172.21.15.137:0/987023265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cd9b060 0x56143cda3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa88b7fe640 0 client.16272 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 >> 172.21.15.137:0/987023265 conn(0x56143cd975f0 msgr2=0x56143cd9d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 shutdown_connections 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 1 -- 172.21.15.137:0/987023265 wait complete. 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.971+0000 7fa8b4963740 20 client.16272 trim_cache size 0 max 16384 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.987+0000 7fa8b4963740 1 Processor -- start 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.987+0000 7fa8b4963740 1 -- start start 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdbd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdbdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8b4963740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1ade10 con 0x56143cd9b060 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8b4963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1ab6b0 con 0x56143cda6240 2024-09-12T12:20:21.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8b4963740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1a9520 con 0x56143cd9b980 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdbd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdbd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53718/0 (socket says 172.21.15.137:53718) 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/889973789 learned_addr learned my addr 172.21.15.137:0/889973789 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/889973789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 msgr2=0x56143cdbdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/889973789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdbdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/889973789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 msgr2=0x56143cda0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/889973789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/889973789 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143cd9b7b0 con 0x56143cd9b060 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/889973789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdbd630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009330 tx=0x7fa8a00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ac544cbaf431a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa891ffb640 1 -- 172.21.15.137:0/889973789 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0015810 con 0x56143cd9b060 2024-09-12T12:20:21.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa891ffb640 1 -- 172.21.15.137:0/889973789 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a00233b0 con 0x56143cd9b060 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.988+0000 7fa891ffb640 1 -- 172.21.15.137:0/889973789 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0022570 con 0x56143cd9b060 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 -- 172.21.15.137:0/889973789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 msgr2=0x56143cdbd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 --2- 172.21.15.137:0/889973789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdbd630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0009330 tx=0x7fa8a00047d0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 -- 172.21.15.137:0/889973789 shutdown_connections 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 --2- 172.21.15.137:0/889973789 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cda0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 --2- 172.21.15.137:0/889973789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdbdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 --2- 172.21.15.137:0/889973789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdbd630 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 -- 172.21.15.137:0/889973789 >> 172.21.15.137:0/889973789 conn(0x56143cd975f0 msgr2=0x56143cda19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 -- 172.21.15.137:0/889973789 shutdown_connections 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 -- 172.21.15.137:0/889973789 wait complete. 2024-09-12T12:20:21.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.989+0000 7fa8b4963740 1 -- start start 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 1 Processor -- start 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdaaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cdaaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1a9b70 con 0x56143cd9b060 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1aafe0 con 0x56143cda6240 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac1aa240 con 0x56143cd9b980 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8b4963740 10 client.0 _ll_register_callbacks cb 0x7fa89c009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53732/0 (socket says 172.21.15.137:53732) 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/2825365972 learned_addr learned my addr 172.21.15.137:0/2825365972 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8abfff640 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdaaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/2825365972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 msgr2=0x56143cdaaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdaaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/2825365972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 msgr2=0x56143cdaaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cdaaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8ab7fe640 1 -- 172.21.15.137:0/2825365972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143cdab500 con 0x56143cd9b060 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.990+0000 7fa8abfff640 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdaaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8ab7fe640 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdae6c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0023d80 tx=0x7fa8a0023560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5cc94543d3de08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa88b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa88b7fe640 1 -- 172.21.15.137:0/2825365972 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0015810 con 0x56143cd9b060 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa88b7fe640 1 -- 172.21.15.137:0/2825365972 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7fa8a00159b0 con 0x56143cd9b060 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a00233b0 con 0x56143cd9b060 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa88b7fe640 1 -- 172.21.15.137:0/2825365972 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8a0015bb0 con 0x56143cd9b060 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 10 client.16278 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:21.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56143ce213e0 con 0x56143cd9b060 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa88b7fe640 1 -- 172.21.15.137:0/2825365972 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a00233b0 con 0x56143cd9b060 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa88b7fe640 1 -- 172.21.15.137:0/2825365972 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fa8a0036410 con 0x56143cd9b060 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 10 client.16278 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 10 client.16278 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 20 client.16278 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143ce219f0 0x56143ce23e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 4 client.16278 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8063b8e7", "release": true, "await": true}] 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.991+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8063b8e7", "release": true, "await": true}) -- 0x56143cd080a0 con 0x56143ce219f0 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.992+0000 7fa8abfff640 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143ce219f0 0x56143ce23e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.992+0000 7fa8abfff640 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143ce219f0 0x56143ce23e10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4faec015778170 server_cookie=1adabf69b32eefd7 in_seq=0 out_seq=0 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:20.992+0000 7fa88b7fe640 10 client.16278 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa88b7fe640 1 -- 172.21.15.137:0/2825365972 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56143cd080a0 con 0x56143ce219f0 2024-09-12T12:20:21.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa88b7fe640 10 client.16278 handle_command_reply: tid=0 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 client.16278 shutdown 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 msgr2=0x56143cdae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdae6c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0023d80 tx=0x7fa8a0023560 comp rx=0 tx=0).stop 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 shutdown_connections 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56143cda6240 0x56143cdaaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56143cd9b980 0x56143cdaaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56143ce219f0 0x56143ce23e10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 --2- 172.21.15.137:0/2825365972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56143cd9b060 0x56143cdae6c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa88b7fe640 0 client.16278 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 >> 172.21.15.137:0/2825365972 conn(0x56143cd975f0 msgr2=0x56143cd95f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 shutdown_connections 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 32, 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 28, 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "8063b8e7": { 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 28, 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:20:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 136.6, 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 1 -- 172.21.15.137:0/2825365972 wait complete. 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b4963740 20 client.16278 trim_cache size 0 max 16384 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8940a2d70 msgr2=0x7fa8940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b258a640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8940a2d70 0x7fa8940a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa8a409ba60 tx=0x7fa8a406bbc0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac106f10 msgr2=0x7fa8ac19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.001+0000 7fa8b258a640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac106f10 0x7fa8ac19cd20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400cce0 tx=0x7fa8a400a620 comp rx=0 tx=0).stop 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 shutdown_connections 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8940a2d70 0x7fa8940a5230 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8ac106f10 0x7fa8ac19cd20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8ac102790 0x7fa8ac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 --2- 172.21.15.137:0/3384003521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8ac101de0 0x7fa8ac198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:21.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 >> 172.21.15.137:0/3384003521 conn(0x7fa8ac0fd8c0 msgr2=0x7fa8ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:21.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 shutdown_connections 2024-09-12T12:20:21.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:21.002+0000 7fa8b258a640 1 -- 172.21.15.137:0/3384003521 wait complete. 2024-09-12T12:20:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:20 smithi189 ceph-mon[35475]: pgmap v220: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 2.2 MiB/s wr, 121 op/s 2024-09-12T12:20:21.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 566282 8.21 MB/sec execute 236 sec latency 15841.635 ms 2024-09-12T12:20:21.800 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8063b8e7', total seconds elapsed: 19.9 2024-09-12T12:20:22.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 568665 8.23 MB/sec execute 237 sec latency 12.366 ms 2024-09-12T12:20:22.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:22 smithi171 ceph-mon[35098]: pgmap v221: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-09-12T12:20:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:22 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:20:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:22 smithi137 ceph-mon[30950]: pgmap v221: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-09-12T12:20:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:22 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:20:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:22 smithi189 ceph-mon[35475]: pgmap v221: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-09-12T12:20:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:22 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:20:23.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 570205 8.22 MB/sec execute 238 sec latency 14.983 ms 2024-09-12T12:20:23.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:20:23.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:20:23.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:20:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:20:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:20:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:20:24.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:20:24.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:20:24.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:20:24.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 572338 8.24 MB/sec execute 239 sec latency 35.760 ms 2024-09-12T12:20:24.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:24 smithi171 ceph-mon[35098]: pgmap v222: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-09-12T12:20:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:24 smithi137 ceph-mon[30950]: pgmap v222: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-09-12T12:20:25.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:24 smithi189 ceph-mon[35475]: pgmap v222: 97 pgs: 97 active+clean; 167 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 1.3 MiB/s wr, 67 op/s 2024-09-12T12:20:25.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 574681 8.25 MB/sec execute 240 sec latency 26.956 ms 2024-09-12T12:20:26.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 576493 8.26 MB/sec execute 241 sec latency 17.605 ms 2024-09-12T12:20:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:26 smithi137 ceph-mon[30950]: pgmap v223: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 162 op/s 2024-09-12T12:20:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:26 smithi189 ceph-mon[35475]: pgmap v223: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 162 op/s 2024-09-12T12:20:27.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:26 smithi171 ceph-mon[35098]: pgmap v223: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 162 op/s 2024-09-12T12:20:27.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 579086 8.29 MB/sec execute 242 sec latency 14.510 ms 2024-09-12T12:20:28.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 580670 8.28 MB/sec execute 243 sec latency 14.564 ms 2024-09-12T12:20:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:28 smithi137 ceph-mon[30950]: pgmap v224: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-12T12:20:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:28 smithi189 ceph-mon[35475]: pgmap v224: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-12T12:20:29.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:28 smithi171 ceph-mon[35098]: pgmap v224: 97 pgs: 97 active+clean; 172 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-09-12T12:20:29.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 582988 8.31 MB/sec execute 244 sec latency 26.644 ms 2024-09-12T12:20:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 -- 172.21.15.137:0/4263796943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 msgr2=0x7fe6c0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/4263796943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c0106da0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009930 tx=0x7fe6b002a420 comp rx=0 tx=0).stop 2024-09-12T12:20:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 -- 172.21.15.137:0/4263796943 shutdown_connections 2024-09-12T12:20:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/4263796943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6c0101350 0x7fe6c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/4263796943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6c01009b0 0x7fe6c0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/4263796943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c0106da0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 -- 172.21.15.137:0/4263796943 >> 172.21.15.137:0/4263796943 conn(0x7fe6c0077990 msgr2=0x7fe6c00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 -- 172.21.15.137:0/4263796943 shutdown_connections 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.491+0000 7fe6c72c0640 1 -- 172.21.15.137:0/4263796943 wait complete. 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 Processor -- start 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 -- start start 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6c01009b0 0x7fe6c0197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6c0101350 0x7fe6c01977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0110e50 con 0x7fe6c01069c0 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0110cd0 con 0x7fe6c0101350 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c72c0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0110fd0 con 0x7fe6c01009b0 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c505c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6c01009b0 0x7fe6c0197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6c0101350 0x7fe6c01977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c019bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43620/0 (socket says 172.21.15.137:43620) 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 -- 172.21.15.137:0/3961828473 learned_addr learned my addr 172.21.15.137:0/3961828473 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c505c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6c01009b0 0x7fe6c0197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33080/0 (socket says 172.21.15.137:33080) 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 -- 172.21.15.137:0/3961828473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6c01009b0 msgr2=0x7fe6c0197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6c01009b0 0x7fe6c0197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 -- 172.21.15.137:0/3961828473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6c0101350 msgr2=0x7fe6c01977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6c0101350 0x7fe6c01977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 -- 172.21.15.137:0/3961828473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c019c260 con 0x7fe6c01069c0 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.492+0000 7fe6c585d640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c019bb30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800eb10 tx=0x7fe6b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.493+0000 7fe6b67fc640 1 -- 172.21.15.137:0/3961828473 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004b20 con 0x7fe6c01069c0 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.493+0000 7fe6b67fc640 1 -- 172.21.15.137:0/3961828473 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b8004cc0 con 0x7fe6c01069c0 2024-09-12T12:20:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.493+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c019c550 con 0x7fe6c01069c0 2024-09-12T12:20:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.493+0000 7fe6b67fc640 1 -- 172.21.15.137:0/3961828473 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8010760 con 0x7fe6c01069c0 2024-09-12T12:20:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.493+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a3de0 con 0x7fe6c01069c0 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.495+0000 7fe6b67fc640 1 -- 172.21.15.137:0/3961828473 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe6b8005020 con 0x7fe6c01069c0 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 Processor -- start 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 -- start start 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56123a8322f0 con 0x56123a8cdaf0 2024-09-12T12:20:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56123a7da270 con 0x56123a8d8cb0 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6ca054740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56123a82e2b0 con 0x56123a8ce410 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c505c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6b67fc640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6ac0a2e20 0x7fe6ac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8cded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43636/0 (socket says 172.21.15.137:43636) 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c485b640 1 -- 172.21.15.137:0/2068411189 learned_addr learned my addr 172.21.15.137:0/2068411189 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c505c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33082/0 (socket says 172.21.15.137:33082) 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c485b640 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c585d640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6ac0a2e20 0x7fe6ac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c505c640 1 -- 172.21.15.137:0/2068411189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 msgr2=0x56123a8db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c505c640 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c505c640 1 -- 172.21.15.137:0/2068411189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 msgr2=0x56123a8cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c505c640 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c505c640 1 -- 172.21.15.137:0/2068411189 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123a7d9340 con 0x56123a8ce410 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.496+0000 7fe6c485b640 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.497+0000 7fe6b67fc640 1 -- 172.21.15.137:0/3961828473 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7fe6b80cf920 con 0x7fe6c01069c0 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.497+0000 7fe6c505c640 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d8770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6b002a940 tx=0x7fe6b0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72d229893c031c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.497+0000 7fe6c585d640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6ac0a2e20 0x7fe6ac0a52e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80a4540 tx=0x7fe6b8097a00 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.497+0000 7fe6ab7fe640 1 -- 172.21.15.137:0/2068411189 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0035040 con 0x56123a8ce410 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.497+0000 7fe6ab7fe640 1 -- 172.21.15.137:0/2068411189 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b0005520 con 0x56123a8ce410 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.497+0000 7fe6ab7fe640 1 -- 172.21.15.137:0/2068411189 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00035f0 con 0x56123a8ce410 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 -- 172.21.15.137:0/2068411189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 msgr2=0x56123a8d8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d8770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6b002a940 tx=0x7fe6b0005de0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 -- 172.21.15.137:0/2068411189 shutdown_connections 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d8770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 --2- 172.21.15.137:0/2068411189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 -- 172.21.15.137:0/2068411189 >> 172.21.15.137:0/2068411189 conn(0x56123a8ca060 msgr2=0x56123a8ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 -- 172.21.15.137:0/2068411189 shutdown_connections 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 -- 172.21.15.137:0/2068411189 wait complete. 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 -- start start 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.498+0000 7fe6ca054740 1 Processor -- start 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56123a8db880 con 0x56123a8cdaf0 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56123a7bb290 con 0x56123a8d8cb0 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56123a8dba00 con 0x56123a8ce410 2024-09-12T12:20:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 10 client.0 _ll_register_callbacks cb 0x56123a7ec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43644/0 (socket says 172.21.15.137:43644) 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c485b640 1 -- 172.21.15.137:0/778516167 learned_addr learned my addr 172.21.15.137:0/778516167 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c505c640 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c485b640 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c505c640 1 -- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 msgr2=0x56123a8d66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c505c640 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8d66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c505c640 1 -- 172.21.15.137:0/778516167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 msgr2=0x56123a8d5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c505c640 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c505c640 1 -- 172.21.15.137:0/778516167 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123a545210 con 0x56123a8ce410 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6c505c640 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d6160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe6b002aa10 tx=0x7fe6b00048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eedf32d6c38c3ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:20:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/778516167 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0052060 con 0x56123a8ce410 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b0005520 con 0x56123a8ce410 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.499+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/778516167 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7fe6b0004390 con 0x56123a8ce410 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/778516167 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe6b0035070 con 0x56123a8ce410 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 10 client.25771 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56123a611620 con 0x56123a8ce410 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/778516167 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b0005520 con 0x56123a8ce410 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/778516167 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fe6b00039c0 con 0x56123a8ce410 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 10 client.25771 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 10 client.25771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 20 client.25771 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a8e8b00 0x56123a8eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 4 client.25771 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56123a83fa20 con 0x56123a8e8b00 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.500+0000 7fe6c485b640 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a8e8b00 0x56123a8eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.501+0000 7fe6a8ff9640 10 client.25771 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.501+0000 7fe6c485b640 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a8e8b00 0x56123a8eaef0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f272442e8e122c0 server_cookie=1573e7d7032460ce in_seq=0 out_seq=0 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.502+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/778516167 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56123a83fa20 con 0x56123a8e8b00 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.502+0000 7fe6a8ff9640 10 client.25771 handle_command_reply: tid=0 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.502+0000 7fe6ca054740 1 client.25771 shutdown 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.502+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 msgr2=0x56123a8d6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.502+0000 7fe6ca054740 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d6160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe6b002aa10 tx=0x7fe6b00048f0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 shutdown_connections 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8d8cb0 0x56123a8d66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8ce410 0x56123a8d6160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a8e8b00 0x56123a8eaef0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 --2- 172.21.15.137:0/778516167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6a8ff9640 0 client.25771 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 >> 172.21.15.137:0/778516167 conn(0x56123a8ca060 msgr2=0x56123a8cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 shutdown_connections 2024-09-12T12:20:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 1 -- 172.21.15.137:0/778516167 wait complete. 2024-09-12T12:20:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.503+0000 7fe6ca054740 20 client.25771 trim_cache size 0 max 16384 2024-09-12T12:20:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 Processor -- start 2024-09-12T12:20:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 -- start start 2024-09-12T12:20:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8d64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8d8cb0 0x56123a8edf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01addf0 con 0x56123a8cdaf0 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01ab690 con 0x56123a8ce410 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6ca054740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01a9500 con 0x56123a8d8cb0 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43656/0 (socket says 172.21.15.137:43656) 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 -- 172.21.15.137:0/246014992 learned_addr learned my addr 172.21.15.137:0/246014992 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c505c640 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8d64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 -- 172.21.15.137:0/246014992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8d8cb0 msgr2=0x56123a8edf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8d8cb0 0x56123a8edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 -- 172.21.15.137:0/246014992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 msgr2=0x56123a8d64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8d64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 -- 172.21.15.137:0/246014992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123a8e5660 con 0x56123a8cdaf0 2024-09-12T12:20:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.518+0000 7fe6c485b640 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d0560 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009710 tx=0x7fe6bc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a779a82daa61cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6aaffd640 1 -- 172.21.15.137:0/246014992 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc027070 con 0x56123a8cdaf0 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6aaffd640 1 -- 172.21.15.137:0/246014992 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc01d040 con 0x56123a8cdaf0 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6aaffd640 1 -- 172.21.15.137:0/246014992 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0127b0 con 0x56123a8cdaf0 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 -- 172.21.15.137:0/246014992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 msgr2=0x56123a8d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d0560 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009710 tx=0x7fe6bc005f90 comp rx=0 tx=0).stop 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 -- 172.21.15.137:0/246014992 shutdown_connections 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8d8cb0 0x56123a8edf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8d64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 --2- 172.21.15.137:0/246014992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8cdaf0 0x56123a8d0560 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 -- 172.21.15.137:0/246014992 >> 172.21.15.137:0/246014992 conn(0x56123a8ca060 msgr2=0x56123a8d7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 -- 172.21.15.137:0/246014992 shutdown_connections 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.519+0000 7fe6ca054740 1 -- 172.21.15.137:0/246014992 wait complete. 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 -- start start 2024-09-12T12:20:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 Processor -- start 2024-09-12T12:20:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8cdaf0 0x56123a8ee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8eb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8d8cb0 0x56123a8ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01a9b50 con 0x56123a8d8cb0 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01aafc0 con 0x56123a8ce410 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c01aa220 con 0x56123a8cdaf0 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6ca054740 10 client.0 _ll_register_callbacks cb 0x7fe6b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8d8cb0 0x56123a8ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8d8cb0 0x56123a8ebe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43658/0 (socket says 172.21.15.137:43658) 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 -- 172.21.15.137:0/1254736988 learned_addr learned my addr 172.21.15.137:0/1254736988 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c505c640 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8eb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8cdaf0 0x56123a8ee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 -- 172.21.15.137:0/1254736988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8cdaf0 msgr2=0x56123a8ee570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8cdaf0 0x56123a8ee570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 -- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 msgr2=0x56123a8eb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8eb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.520+0000 7fe6c485b640 1 -- 172.21.15.137:0/1254736988 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123a8ec6c0 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6c485b640 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8d8cb0 0x56123a8ebe90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc011970 tx=0x7fe6bc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2845769dafc78ea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/1254736988 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc033060 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/1254736988 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8484+0+0 (secure 0 0 0) 0x7fe6bc003bd0 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6bc01d040 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/1254736988 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe6bc027030 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 10 client.16296 fetch_fsmap learned FSMap version 32 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56123a953350 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/1254736988 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc01d040 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/1254736988 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7fe6bc02c070 con 0x56123a8d8cb0 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 10 client.16296 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T12:20:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 20 client.16296 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a9539c0 0x56123a955de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 4 client.16296 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:20:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56123a839d70 con 0x56123a9539c0 2024-09-12T12:20:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.521+0000 7fe6c505c640 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a9539c0 0x56123a955de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.522+0000 7fe6c505c640 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a9539c0 0x56123a955de0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd87da9dce4b03c server_cookie=2b8c43355cf142f in_seq=0 out_seq=0 2024-09-12T12:20:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.522+0000 7fe6a8ff9640 10 client.16296 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.581+0000 7fe6a8ff9640 1 -- 172.21.15.137:0/1254736988 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56123a839d70 con 0x56123a9539c0 2024-09-12T12:20:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.581+0000 7fe6a8ff9640 10 client.16296 handle_command_reply: tid=0 2024-09-12T12:20:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 client.16296 shutdown 2024-09-12T12:20:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8d8cb0 msgr2=0x56123a8ebe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8d8cb0 0x56123a8ebe90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc011970 tx=0x7fe6bc003f10 comp rx=0 tx=0).stop 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 shutdown_connections 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56123a9539c0 0x56123a955de0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56123a8d8cb0 0x56123a8ebe90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56123a8ce410 0x56123a8eb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 --2- 172.21.15.137:0/1254736988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56123a8cdaf0 0x56123a8ee570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 >> 172.21.15.137:0/1254736988 conn(0x56123a8ca060 msgr2=0x56123a8c89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6a8ff9640 0 client.16296 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 shutdown_connections 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 1 -- 172.21.15.137:0/1254736988 wait complete. 2024-09-12T12:20:30.584 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.582+0000 7fe6ca054740 20 client.16296 trim_cache size 0 max 16384 2024-09-12T12:20:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:20:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:20:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:20:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6ac0a2e20 msgr2=0x7fe6ac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6ac0a2e20 0x7fe6ac0a52e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80a4540 tx=0x7fe6b8097a00 comp rx=0 tx=0).stop 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 msgr2=0x7fe6c019bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c019bb30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800eb10 tx=0x7fe6b800c2d0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 shutdown_connections 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe6ac0a2e20 0x7fe6ac0a52e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c01069c0 0x7fe6c019bb30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6c0101350 0x7fe6c01977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 --2- 172.21.15.137:0/3961828473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6c01009b0 0x7fe6c0197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 >> 172.21.15.137:0/3961828473 conn(0x7fe6c0077990 msgr2=0x7fe6c01037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 shutdown_connections 2024-09-12T12:20:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:30.583+0000 7fe6c72c0640 1 -- 172.21.15.137:0/3961828473 wait complete. 2024-09-12T12:20:30.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 585511 8.33 MB/sec execute 245 sec latency 34.290 ms 2024-09-12T12:20:30.995 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:30 smithi189 ceph-mon[35475]: pgmap v225: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-09-12T12:20:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:30 smithi137 ceph-mon[30950]: pgmap v225: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-09-12T12:20:31.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:30 smithi171 ceph-mon[35098]: pgmap v225: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-09-12T12:20:31.340 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:20:31.341 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:20:31.341 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:30 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/2098638608 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca36cb3860 con 0x55ca36c91000 2024-09-12T12:20:31.341 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:20:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:20:31.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 587456 8.34 MB/sec execute 246 sec latency 17.901 ms 2024-09-12T12:20:32.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:31 smithi137 ceph-mon[30950]: daemon mds.f restarted 2024-09-12T12:20:32.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:31 smithi189 ceph-mon[35475]: daemon mds.f restarted 2024-09-12T12:20:32.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:31 smithi171 ceph-mon[35098]: daemon mds.f restarted 2024-09-12T12:20:32.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 589728 8.36 MB/sec execute 247 sec latency 17.913 ms 2024-09-12T12:20:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:32 smithi137 ceph-mon[30950]: pgmap v226: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-09-12T12:20:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:32 smithi137 ceph-mon[30950]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T12:20:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:32 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/2747788615,v1:172.21.15.189:6835/2747788615] up:boot 2024-09-12T12:20:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:32 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:20:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:32 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:20:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:32 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:20:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:32 smithi189 ceph-mon[35475]: pgmap v226: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-09-12T12:20:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:32 smithi189 ceph-mon[35475]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T12:20:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:32 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/2747788615,v1:172.21.15.189:6835/2747788615] up:boot 2024-09-12T12:20:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:32 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:20:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:32 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:20:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:32 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:32 smithi171 ceph-mon[35098]: pgmap v226: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 223 op/s 2024-09-12T12:20:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:32 smithi171 ceph-mon[35098]: osdmap e90: 12 total, 12 up, 12 in 2024-09-12T12:20:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:32 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/2747788615,v1:172.21.15.189:6835/2747788615] up:boot 2024-09-12T12:20:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:32 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:20:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:32 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:20:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:32 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:20:33.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 591313 8.35 MB/sec execute 248 sec latency 14.362 ms 2024-09-12T12:20:34.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 593629 8.37 MB/sec execute 249 sec latency 13.350 ms 2024-09-12T12:20:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:34 smithi137 ceph-mon[30950]: pgmap v228: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:20:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:34 smithi189 ceph-mon[35475]: pgmap v228: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:20:35.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:34 smithi171 ceph-mon[35098]: pgmap v228: 97 pgs: 97 active+clean; 180 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.4 MiB/s wr, 268 op/s 2024-09-12T12:20:35.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 595937 8.38 MB/sec execute 250 sec latency 36.653 ms 2024-09-12T12:20:36.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 597626 8.38 MB/sec execute 251 sec latency 20.994 ms 2024-09-12T12:20:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:36 smithi137 ceph-mon[30950]: pgmap v229: 97 pgs: 97 active+clean; 176 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-12T12:20:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:36 smithi189 ceph-mon[35475]: pgmap v229: 97 pgs: 97 active+clean; 176 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-12T12:20:37.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:36 smithi171 ceph-mon[35098]: pgmap v229: 97 pgs: 97 active+clean; 176 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-12T12:20:37.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 600415 8.42 MB/sec execute 252 sec latency 15.859 ms 2024-09-12T12:20:38.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 601923 8.41 MB/sec execute 253 sec latency 16.898 ms 2024-09-12T12:20:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:38 smithi137 ceph-mon[30950]: pgmap v230: 97 pgs: 97 active+clean; 176 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-12T12:20:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:38 smithi189 ceph-mon[35475]: pgmap v230: 97 pgs: 97 active+clean; 176 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-12T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:38 smithi171 ceph-mon[35098]: pgmap v230: 97 pgs: 97 active+clean; 176 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 340 op/s 2024-09-12T12:20:39.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 604264 8.43 MB/sec execute 254 sec latency 13.068 ms 2024-09-12T12:20:40.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 606703 8.45 MB/sec execute 255 sec latency 20.769 ms 2024-09-12T12:20:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:40 smithi137 ceph-mon[30950]: pgmap v231: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-12T12:20:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:40 smithi189 ceph-mon[35475]: pgmap v231: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-12T12:20:41.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:40 smithi171 ceph-mon[35098]: pgmap v231: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-12T12:20:41.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 608067 8.44 MB/sec execute 256 sec latency 130.828 ms 2024-09-12T12:20:42.737 INFO:tasks.workunit.client.0.smithi137.stdout: 1 610343 8.46 MB/sec execute 257 sec latency 58.128 ms 2024-09-12T12:20:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:42 smithi137 ceph-mon[30950]: pgmap v232: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-12T12:20:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:42 smithi189 ceph-mon[35475]: pgmap v232: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-12T12:20:43.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:42 smithi171 ceph-mon[35098]: pgmap v232: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 334 op/s 2024-09-12T12:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 -- 172.21.15.137:0/3156099400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4101ac0 msgr2=0x7fc8a4101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/3156099400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4101ac0 0x7fc8a4101f20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009930 tx=0x7fc88c02a420 comp rx=0 tx=0).stop 2024-09-12T12:20:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 -- 172.21.15.137:0/3156099400 shutdown_connections 2024-09-12T12:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/3156099400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8a4102460 0x7fc8a410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/3156099400 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4101ac0 0x7fc8a4101f20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/3156099400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8a4107ad0 0x7fc8a4107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 -- 172.21.15.137:0/3156099400 >> 172.21.15.137:0/3156099400 conn(0x7fc8a40fd820 msgr2=0x7fc8a40ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 -- 172.21.15.137:0/3156099400 shutdown_connections 2024-09-12T12:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 -- 172.21.15.137:0/3156099400 wait complete. 2024-09-12T12:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.676+0000 7fc8a8ded640 1 Processor -- start 2024-09-12T12:20:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a8ded640 1 -- start start 2024-09-12T12:20:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a8ded640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8a4101ac0 0x7fc8a41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a8ded640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4102460 0x7fc8a4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a8ded640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8a4107ad0 0x7fc8a419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a8ded640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4111f10 con 0x7fc8a4102460 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a8ded640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4111d90 con 0x7fc8a4107ad0 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a8ded640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4112090 con 0x7fc8a4101ac0 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4102460 0x7fc8a4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8a4101ac0 0x7fc8a41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4102460 0x7fc8a4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46334/0 (socket says 172.21.15.137:46334) 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 -- 172.21.15.137:0/1465595788 learned_addr learned my addr 172.21.15.137:0/1465595788 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8a4107ad0 0x7fc8a419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 -- 172.21.15.137:0/1465595788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8a4101ac0 msgr2=0x7fc8a41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8a4101ac0 0x7fc8a41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 -- 172.21.15.137:0/1465595788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8a4107ad0 msgr2=0x7fc8a419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8a4107ad0 0x7fc8a419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc899d9b640 1 -- 172.21.15.137:0/1465595788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419d3d0 con 0x7fc8a4102460 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.677+0000 7fc8a259c640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8a4101ac0 0x7fc8a41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.678+0000 7fc899d9b640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4102460 0x7fc8a4198920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009230 tx=0x7fc88c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.678+0000 7fc89bfff640 1 -- 172.21.15.137:0/1465595788 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c03f030 con 0x7fc8a4102460 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.678+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a419d660 con 0x7fc8a4102460 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.678+0000 7fc89bfff640 1 -- 172.21.15.137:0/1465595788 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88c003ad0 con 0x7fc8a4102460 2024-09-12T12:20:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.678+0000 7fc89bfff640 1 -- 172.21.15.137:0/1465595788 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c035070 con 0x7fc8a4102460 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.678+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41a4f50 con 0x7fc8a4102460 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.680+0000 7fc89bfff640 1 -- 172.21.15.137:0/1465595788 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc88c042030 con 0x7fc8a4102460 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.680+0000 7fc8abc5d740 1 Processor -- start 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8abc5d740 1 -- start start 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc54ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc5604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d8bc4b32f0 con 0x55d8bc55e0c0 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d8bc45b270 con 0x55d8bc54f2d0 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d8bc4af2b0 con 0x55d8bc54e9b0 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc89bfff640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8900a2da0 0x7fc8900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc899d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc5604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a2d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc54ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a2d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc54ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:45396/0 (socket says 172.21.15.137:45396) 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a2d9d640 1 -- 172.21.15.137:0/1037499329 learned_addr learned my addr 172.21.15.137:0/1037499329 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8900a2da0 0x7fc8900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc89bfff640 1 -- 172.21.15.137:0/1465595788 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc88c0ef990 con 0x7fc8a4102460 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a259c640 1 -- 172.21.15.137:0/1037499329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 msgr2=0x55d8bc54ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a259c640 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc54ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a259c640 1 -- 172.21.15.137:0/1037499329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 msgr2=0x55d8bc5604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a259c640 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc5604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a259c640 1 -- 172.21.15.137:0/1037499329 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bc45a340 con 0x55d8bc54f2d0 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc54ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8900a2da0 0x7fc8900a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0129b0 tx=0x7fc89c005cc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.681+0000 7fc8a259c640 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55db80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc894005fd0 tx=0x7fc894011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23620855e49ad0ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc898d99640 1 -- 172.21.15.137:0/1037499329 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894011e60 con 0x55d8bc54f2d0 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc898d99640 1 -- 172.21.15.137:0/1037499329 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc894004d20 con 0x55d8bc54f2d0 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc898d99640 1 -- 172.21.15.137:0/1037499329 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8940043d0 con 0x55d8bc54f2d0 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc8abc5d740 1 -- 172.21.15.137:0/1037499329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54f2d0 msgr2=0x55d8bc55db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55db80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc894005fd0 tx=0x7fc894011c70 comp rx=0 tx=0).stop 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc8abc5d740 1 -- 172.21.15.137:0/1037499329 shutdown_connections 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc5604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55db80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/1037499329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc54ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.682+0000 7fc8abc5d740 1 -- 172.21.15.137:0/1037499329 >> 172.21.15.137:0/1037499329 conn(0x55d8bc54b060 msgr2=0x55d8bc54b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 -- 172.21.15.137:0/1037499329 shutdown_connections 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 -- 172.21.15.137:0/1037499329 wait complete. 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 -- start start 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 Processor -- start 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54e9b0 0x55d8bc55afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc55ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d8bc560c90 con 0x55d8bc55e0c0 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d8bc43c290 con 0x55d8bc54e9b0 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d8bc560e10 con 0x55d8bc54f2d0 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8abc5d740 10 client.0 _ll_register_callbacks cb 0x55d8bc46d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8a2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc55ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54e9b0 0x55d8bc55afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8a2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc55ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46364/0 (socket says 172.21.15.137:46364) 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc8a2d9d640 1 -- 172.21.15.137:0/4173219130 learned_addr learned my addr 172.21.15.137:0/4173219130 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.683+0000 7fc899d9b640 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8a2d9d640 1 -- 172.21.15.137:0/4173219130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54f2d0 msgr2=0x55d8bc55b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8a2d9d640 1 -- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54e9b0 msgr2=0x55d8bc55afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54e9b0 0x55d8bc55afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8a2d9d640 1 -- 172.21.15.137:0/4173219130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bc1c6210 con 0x55d8bc55e0c0 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8a259c640 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54e9b0 0x55d8bc55afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc55ba60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc89c004320 tx=0x7fc89c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5764191ffbcc037d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc889ffb640 1 -- 172.21.15.137:0/4173219130 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0044d0 con 0x55d8bc55e0c0 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc894004d20 con 0x55d8bc55e0c0 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc889ffb640 1 -- 172.21.15.137:0/4173219130 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc89c005020 con 0x55d8bc55e0c0 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc889ffb640 1 -- 172.21.15.137:0/4173219130 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc89c005220 con 0x55d8bc55e0c0 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8abc5d740 10 client.16314 fetch_fsmap learned FSMap version 34 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8bc292620 con 0x55d8bc55e0c0 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc889ffb640 1 -- 172.21.15.137:0/4173219130 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc894004d20 con 0x55d8bc55e0c0 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.684+0000 7fc889ffb640 1 -- 172.21.15.137:0/4173219130 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fc89c0035f0 con 0x55d8bc55e0c0 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8abc5d740 10 client.16314 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:20:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8abc5d740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8abc5d740 20 client.16314 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc56e0b0 0x55d8bc5704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8abc5d740 4 client.16314 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8bc4c0a20 con 0x55d8bc56e0b0 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8a259c640 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc56e0b0 0x55d8bc5704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc8a259c640 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc56e0b0 0x55d8bc5704a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7683731d893d89 server_cookie=c54fe49ceb17c744 in_seq=0 out_seq=0 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.685+0000 7fc889ffb640 10 client.16314 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc889ffb640 1 -- 172.21.15.137:0/4173219130 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8bc4c0a20 con 0x55d8bc56e0b0 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc889ffb640 10 client.16314 handle_command_reply: tid=0 2024-09-12T12:20:43.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 client.16314 shutdown 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 msgr2=0x55d8bc55ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc55ba60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc89c004320 tx=0x7fc89c0046c0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 shutdown_connections 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc56e0b0 0x55d8bc5704a0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc55e0c0 0x55d8bc55ba60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54f2d0 0x55d8bc55b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/4173219130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc54e9b0 0x55d8bc55afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 >> 172.21.15.137:0/4173219130 conn(0x55d8bc54b060 msgr2=0x55d8bc550dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc889ffb640 0 client.16314 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 shutdown_connections 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 1 -- 172.21.15.137:0/4173219130 wait complete. 2024-09-12T12:20:43.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.686+0000 7fc8abc5d740 20 client.16314 trim_cache size 0 max 16384 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 Processor -- start 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 -- start start 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc575d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc5762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc558220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41add60 con 0x55d8bc54f2d0 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41ab600 con 0x55d8bc55e0c0 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.700+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41a9470 con 0x55d8bc54e9b0 2024-09-12T12:20:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc5762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc575d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc5762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46366/0 (socket says 172.21.15.137:46366) 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 -- 172.21.15.137:0/2249081732 learned_addr learned my addr 172.21.15.137:0/2249081732 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc558220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 -- 172.21.15.137:0/2249081732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 msgr2=0x55d8bc575d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc575d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 -- 172.21.15.137:0/2249081732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 msgr2=0x55d8bc558220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc558220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 -- 172.21.15.137:0/2249081732 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bc54ae90 con 0x55d8bc54f2d0 2024-09-12T12:20:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc899d9b640 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc5762d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0bbcb0 tx=0x7fc88c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9141125dc937c9b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc88bfff640 1 -- 172.21.15.137:0/2249081732 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c049040 con 0x55d8bc54f2d0 2024-09-12T12:20:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc88bfff640 1 -- 172.21.15.137:0/2249081732 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88c0c4b70 con 0x55d8bc54f2d0 2024-09-12T12:20:43.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.701+0000 7fc88bfff640 1 -- 172.21.15.137:0/2249081732 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0b8620 con 0x55d8bc54f2d0 2024-09-12T12:20:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 -- 172.21.15.137:0/2249081732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 msgr2=0x55d8bc5762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc5762d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0bbcb0 tx=0x7fc88c009260 comp rx=0 tx=0).stop 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 -- 172.21.15.137:0/2249081732 shutdown_connections 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc558220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc5762d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/2249081732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc575d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 -- 172.21.15.137:0/2249081732 >> 172.21.15.137:0/2249081732 conn(0x55d8bc54b060 msgr2=0x55d8bc55d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 -- 172.21.15.137:0/2249081732 shutdown_connections 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 -- 172.21.15.137:0/2249081732 wait complete. 2024-09-12T12:20:43.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 -- start start 2024-09-12T12:20:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.702+0000 7fc8abc5d740 1 Processor -- start 2024-09-12T12:20:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc5669c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc561ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc562470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41a9ac0 con 0x55d8bc54f2d0 2024-09-12T12:20:43.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41aaf30 con 0x55d8bc55e0c0 2024-09-12T12:20:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a41aa190 con 0x55d8bc54e9b0 2024-09-12T12:20:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 10 client.0 _ll_register_callbacks cb 0x7fc89c012630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc561ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8a259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc5669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc561ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46378/0 (socket says 172.21.15.137:46378) 2024-09-12T12:20:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 -- 172.21.15.137:0/3042433599 learned_addr learned my addr 172.21.15.137:0/3042433599 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:43.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8a2d9d640 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc562470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 -- 172.21.15.137:0/3042433599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 msgr2=0x55d8bc5669c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc5669c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 -- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 msgr2=0x55d8bc562470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc562470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 -- 172.21.15.137:0/3042433599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bc5d7d60 con 0x55d8bc54f2d0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc899d9b640 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc561ce0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0b8410 tx=0x7fc88c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d08cc55251c99b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc889ffb640 1 -- 172.21.15.137:0/3042433599 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c049050 con 0x55d8bc54f2d0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc889ffb640 1 -- 172.21.15.137:0/3042433599 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7fc88c0ed3b0 con 0x55d8bc54f2d0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc88c0c4b70 con 0x55d8bc54f2d0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc889ffb640 1 -- 172.21.15.137:0/3042433599 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc88c003c80 con 0x55d8bc54f2d0 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.703+0000 7fc8abc5d740 10 client.16320 fetch_fsmap learned FSMap version 34 2024-09-12T12:20:43.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8bc5d8f50 con 0x55d8bc54f2d0 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc889ffb640 1 -- 172.21.15.137:0/3042433599 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88c0c4b70 con 0x55d8bc54f2d0 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc889ffb640 1 -- 172.21.15.137:0/3042433599 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fc88c0c46e0 con 0x55d8bc54f2d0 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8abc5d740 10 client.16320 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8abc5d740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8abc5d740 20 client.16320 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc5d94f0 0x55d8bc5db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8abc5d740 4 client.16320 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d8bc4bad70 con 0x55d8bc5d94f0 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8a259c640 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc5d94f0 0x55d8bc5db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc889ffb640 10 client.16320 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:43.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.704+0000 7fc8a259c640 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc5d94f0 0x55d8bc5db910 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204d4d4564e34960 server_cookie=4322fd2ecfad4667 in_seq=0 out_seq=0 2024-09-12T12:20:43.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 611935 8.46 MB/sec execute 258 sec latency 14.894 ms 2024-09-12T12:20:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc889ffb640 1 -- 172.21.15.137:0/3042433599 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d8bc4bad70 con 0x55d8bc5d94f0 2024-09-12T12:20:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc889ffb640 10 client.16320 handle_command_reply: tid=0 2024-09-12T12:20:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 client.16320 shutdown 2024-09-12T12:20:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 msgr2=0x55d8bc561ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc561ce0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0b8410 tx=0x7fc88c02a420 comp rx=0 tx=0).stop 2024-09-12T12:20:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 shutdown_connections 2024-09-12T12:20:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d8bc55e0c0 0x55d8bc562470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d8bc5d94f0 0x55d8bc5db910 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc889ffb640 0 client.16320 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d8bc54f2d0 0x55d8bc561ce0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 --2- 172.21.15.137:0/3042433599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d8bc54e9b0 0x55d8bc5669c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 >> 172.21.15.137:0/3042433599 conn(0x55d8bc54b060 msgr2=0x55d8bc55d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 shutdown_connections 2024-09-12T12:20:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.774+0000 7fc8abc5d740 1 -- 172.21.15.137:0/3042433599 wait complete. 2024-09-12T12:20:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8abc5d740 20 client.16320 trim_cache size 0 max 16384 2024-09-12T12:20:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:20:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:20:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:20:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:20:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8900a2da0 msgr2=0x7fc8900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8900a2da0 0x7fc8900a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0129b0 tx=0x7fc89c005cc0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4102460 msgr2=0x7fc8a4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4102460 0x7fc8a4198920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009230 tx=0x7fc88c004270 comp rx=0 tx=0).stop 2024-09-12T12:20:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 shutdown_connections 2024-09-12T12:20:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8900a2da0 0x7fc8900a5260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc8a4107ad0 0x7fc8a419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8a4102460 0x7fc8a4198920 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 --2- 172.21.15.137:0/1465595788 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8a4101ac0 0x7fc8a41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.775+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 >> 172.21.15.137:0/1465595788 conn(0x7fc8a40fd820 msgr2=0x7fc8a40fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.776+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 shutdown_connections 2024-09-12T12:20:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:43.776+0000 7fc8a8ded640 1 -- 172.21.15.137:0/1465595788 wait complete. 2024-09-12T12:20:44.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 614594 8.49 MB/sec execute 259 sec latency 23.759 ms 2024-09-12T12:20:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:44 smithi137 ceph-mon[30950]: pgmap v233: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-09-12T12:20:45.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:44 smithi189 ceph-mon[35475]: pgmap v233: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-09-12T12:20:45.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:44 smithi171 ceph-mon[35098]: pgmap v233: 97 pgs: 97 active+clean; 185 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-09-12T12:20:45.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 615933 8.48 MB/sec execute 260 sec latency 18.631 ms 2024-09-12T12:20:46.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 618237 8.50 MB/sec execute 261 sec latency 13.139 ms 2024-09-12T12:20:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:46 smithi137 ceph-mon[30950]: pgmap v234: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-09-12T12:20:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:46 smithi189 ceph-mon[35475]: pgmap v234: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-09-12T12:20:47.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:46 smithi171 ceph-mon[35098]: pgmap v234: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-09-12T12:20:47.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 619678 8.49 MB/sec execute 262 sec latency 26.178 ms 2024-09-12T12:20:48.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 621999 8.51 MB/sec execute 263 sec latency 12.726 ms 2024-09-12T12:20:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:48 smithi137 ceph-mon[30950]: pgmap v235: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-12T12:20:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:48 smithi189 ceph-mon[35475]: pgmap v235: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-12T12:20:49.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:48 smithi171 ceph-mon[35098]: pgmap v235: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-12T12:20:49.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 624128 8.51 MB/sec execute 264 sec latency 15.097 ms 2024-09-12T12:20:50.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 625979 8.52 MB/sec execute 265 sec latency 19.663 ms 2024-09-12T12:20:51.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:50 smithi137 ceph-mon[30950]: pgmap v236: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 361 op/s 2024-09-12T12:20:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:50 smithi189 ceph-mon[35475]: pgmap v236: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 361 op/s 2024-09-12T12:20:51.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:50 smithi171 ceph-mon[35098]: pgmap v236: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.1 MiB/s wr, 361 op/s 2024-09-12T12:20:51.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 628841 8.55 MB/sec execute 266 sec latency 14.388 ms 2024-09-12T12:20:52.629 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T12:20:52.629 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T12:20:52.630 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:20:52.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 630276 8.54 MB/sec execute 267 sec latency 18.534 ms 2024-09-12T12:20:52.891 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:20:53.013 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:52 smithi137 ceph-mon[30950]: pgmap v237: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-12T12:20:53.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:52 smithi189 ceph-mon[35475]: pgmap v237: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-12T12:20:53.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:52 smithi171 ceph-mon[35098]: pgmap v237: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-12T12:20:53.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 -- 172.21.15.137:0/2656480827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af010e480 msgr2=0x7f5af0110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:53.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2656480827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af010e480 0x7f5af0110870 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009ca0 tx=0x7f5ae802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 -- 172.21.15.137:0/2656480827 shutdown_connections 2024-09-12T12:20:53.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2656480827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af010e480 0x7f5af0110870 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2656480827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5af0101280 0x7f5af010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2656480827 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af01008d0 0x7f5af0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 -- 172.21.15.137:0/2656480827 >> 172.21.15.137:0/2656480827 conn(0x7f5af00fc7a0 msgr2=0x7f5af00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 -- 172.21.15.137:0/2656480827 shutdown_connections 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.453+0000 7f5af68ac640 1 -- 172.21.15.137:0/2656480827 wait complete. 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.454+0000 7f5af68ac640 1 Processor -- start 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.454+0000 7f5af68ac640 1 -- start start 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.454+0000 7f5af68ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5af01008d0 0x7f5af019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.454+0000 7f5af68ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101280 0x7f5af019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5af68ac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af010e480 0x7f5af019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5af68ac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01141d0 con 0x7f5af0101280 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5af68ac640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0114050 con 0x7f5af010e480 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5af68ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0114350 con 0x7f5af01008d0 2024-09-12T12:20:53.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101280 0x7f5af019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101280 0x7f5af019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40256/0 (socket says 172.21.15.137:40256) 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 -- 172.21.15.137:0/2459772321 learned_addr learned my addr 172.21.15.137:0/2459772321 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aeffff640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5af01008d0 0x7f5af019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5af4e49640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af010e480 0x7f5af019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 -- 172.21.15.137:0/2459772321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5af01008d0 msgr2=0x7f5af019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5af01008d0 0x7f5af019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 -- 172.21.15.137:0/2459772321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af010e480 msgr2=0x7f5af019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af010e480 0x7f5af019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 -- 172.21.15.137:0/2459772321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af019f5e0 con 0x7f5af0101280 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aeffff640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5af01008d0 0x7f5af019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aef7fe640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101280 0x7f5af019ab70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9f0 tx=0x7f5ae400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aed7fa640 1 -- 172.21.15.137:0/2459772321 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae400cd80 con 0x7f5af0101280 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af019f8d0 con 0x7f5af0101280 2024-09-12T12:20:53.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aed7fa640 1 -- 172.21.15.137:0/2459772321 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae4005500 con 0x7f5af0101280 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5aed7fa640 1 -- 172.21.15.137:0/2459772321 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4010690 con 0x7f5af0101280 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.455+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01a71b0 con 0x7f5af0101280 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.457+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af0100cb0 con 0x7f5af0101280 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.458+0000 7f5aed7fa640 1 -- 172.21.15.137:0/2459772321 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5ae40046c0 con 0x7f5af0101280 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.458+0000 7f5aed7fa640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5ad80a2cf0 0x7f5ad80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.458+0000 7f5aeffff640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5ad80a2cf0 0x7f5ad80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.458+0000 7f5aed7fa640 1 -- 172.21.15.137:0/2459772321 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f5ae40cf7a0 con 0x7f5af0101280 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.459+0000 7f5aeffff640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5ad80a2cf0 0x7f5ad80a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009fc0 tx=0x7f5ae0005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:53.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.462+0000 7f5aed7fa640 1 -- 172.21.15.137:0/2459772321 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ae409c410 con 0x7f5af0101280 2024-09-12T12:20:53.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.675+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5af01a7450 con 0x7f5af0101280 2024-09-12T12:20:53.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.676+0000 7f5aed7fa640 1 -- 172.21.15.137:0/2459772321 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11937 (secure 0 0 0) 0x7f5ae409bd10 con 0x7f5af0101280 2024-09-12T12:20:53.677 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:20:53.678 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":34,"btime":"2024-09-12T12:20:31:715452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:20:31.715433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25780":{"gid":25780,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2747788615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2747788615},{"type":"v1","addr":"172.21.15.189:6835","nonce":2747788615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:20:53.678 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 34 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5ad80a2cf0 msgr2=0x7f5ad80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5ad80a2cf0 0x7f5ad80a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009fc0 tx=0x7f5ae0005f90 comp rx=0 tx=0).stop 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101280 msgr2=0x7f5af019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101280 0x7f5af019ab70 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400e9f0 tx=0x7f5ae400edf0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 shutdown_connections 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5ad80a2cf0 0x7f5ad80a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af010e480 0x7f5af019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5af0101280 0x7f5af019ab70 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 --2- 172.21.15.137:0/2459772321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5af01008d0 0x7f5af019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 >> 172.21.15.137:0/2459772321 conn(0x7f5af00fc7a0 msgr2=0x7f5af00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 shutdown_connections 2024-09-12T12:20:53.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:53.679+0000 7f5af68ac640 1 -- 172.21.15.137:0/2459772321 wait complete. 2024-09-12T12:20:53.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 632606 8.56 MB/sec execute 268 sec latency 12.389 ms 2024-09-12T12:20:53.989 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:53 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2459772321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:54.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:53 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2459772321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:54.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:53 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2459772321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:20:54.505 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T12:20:54.505 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T12:20:54.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 634628 8.56 MB/sec execute 269 sec latency 15.601 ms 2024-09-12T12:20:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:20:55.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:54 smithi137 ceph-mon[30950]: pgmap v238: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-12T12:20:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:54 smithi189 ceph-mon[35475]: pgmap v238: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-12T12:20:55.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:54 smithi171 ceph-mon[35098]: pgmap v238: 97 pgs: 97 active+clean; 190 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-09-12T12:20:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 -- 172.21.15.137:0/2352954758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 msgr2=0x7f569410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 --2- 172.21.15.137:0/2352954758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569410ec70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f568c009cd0 tx=0x7f568c02a8d0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 -- 172.21.15.137:0/2352954758 shutdown_connections 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 --2- 172.21.15.137:0/2352954758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569410ec70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 --2- 172.21.15.137:0/2352954758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 0x7f5694107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 --2- 172.21.15.137:0/2352954758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5694102dd0 0x7f56941031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 -- 172.21.15.137:0/2352954758 >> 172.21.15.137:0/2352954758 conn(0x7f56940fc7d0 msgr2=0x7f56940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 -- 172.21.15.137:0/2352954758 shutdown_connections 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.336+0000 7f569a943640 1 -- 172.21.15.137:0/2352954758 wait complete. 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 Processor -- start 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 -- start start 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5694102dd0 0x7f5694198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 0x7f5694198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56941120d0 con 0x7f5694108300 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5694111f50 con 0x7f5694103780 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f569a943640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5694112250 con 0x7f5694102dd0 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f5698ee0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f56937fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 0x7f5694198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f5693fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5694102dd0 0x7f5694198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f5698ee0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40272/0 (socket says 172.21.15.137:40272) 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f5698ee0640 1 -- 172.21.15.137:0/3114821794 learned_addr learned my addr 172.21.15.137:0/3114821794 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.337+0000 7f56937fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 0x7f5694198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:52466/0 (socket says 172.21.15.137:52466) 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f5698ee0640 1 -- 172.21.15.137:0/3114821794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5694102dd0 msgr2=0x7f5694198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f5698ee0640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5694102dd0 0x7f5694198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f5698ee0640 1 -- 172.21.15.137:0/3114821794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 msgr2=0x7f5694198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f5698ee0640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 0x7f5694198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f5698ee0640 1 -- 172.21.15.137:0/3114821794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569419d540 con 0x7f5694108300 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f5693fff640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5694102dd0 0x7f5694198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f56937fe640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 0x7f5694198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f5698ee0640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569419ce60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f568c009ca0 tx=0x7f568c02f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f56917fa640 1 -- 172.21.15.137:0/3114821794 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c031760 con 0x7f5694108300 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f56917fa640 1 -- 172.21.15.137:0/3114821794 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c003e50 con 0x7f5694108300 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569419d7d0 con 0x7f5694108300 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f56917fa640 1 -- 172.21.15.137:0/3114821794 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c030590 con 0x7f5694108300 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.338+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56941a5110 con 0x7f5694108300 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.341+0000 7f569cd1c740 1 Processor -- start 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.341+0000 7f56917fa640 1 -- 172.21.15.137:0/3114821794 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f568c004040 con 0x7f5694108300 2024-09-12T12:20:55.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.341+0000 7f569cd1c740 1 -- start start 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.341+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a4760 0x55bded2a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.341+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a5080 0x55bded2af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f569cd1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bded20a2a0 con 0x55bded2a4760 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f569cd1c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bded1b1b10 con 0x55bded2a5080 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f569cd1c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bded206260 con 0x55bded2af940 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f56937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a4760 0x55bded2a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f56937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a4760 0x55bded2a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40286/0 (socket says 172.21.15.137:40286) 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f56937fe640 1 -- 172.21.15.137:0/2871430711 learned_addr learned my addr 172.21.15.137:0/2871430711 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5693fff640 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a5080 0x55bded2af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f56937fe640 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f56917fa640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56800a2cf0 0x7f56800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5693fff640 1 -- 172.21.15.137:0/2871430711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 msgr2=0x55bded2b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5693fff640 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5693fff640 1 -- 172.21.15.137:0/2871430711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a4760 msgr2=0x55bded2a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5693fff640 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a4760 0x55bded2a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5693fff640 1 -- 172.21.15.137:0/2871430711 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdecf7bb90 con 0x55bded2a5080 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f56917fa640 1 -- 172.21.15.137:0/3114821794 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f568c0f0320 con 0x7f5694108300 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5698ee0640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56800a2cf0 0x7f56800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f56937fe640 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a4760 0x55bded2a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f5693fff640 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a5080 0x55bded2af400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f56840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eadbf42c71b00ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.342+0000 7f568a7fc640 1 -- 172.21.15.137:0/2871430711 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684004310 con 0x55bded2a5080 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f568a7fc640 1 -- 172.21.15.137:0/2871430711 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56840044b0 con 0x55bded2a5080 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f568a7fc640 1 -- 172.21.15.137:0/2871430711 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56840035f0 con 0x55bded2a5080 2024-09-12T12:20:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 -- 172.21.15.137:0/2871430711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a5080 msgr2=0x55bded2af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a5080 0x55bded2af400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f56840118d0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f5698ee0640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56800a2cf0 0x7f56800a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f568c006010 tx=0x7f568c005c30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 -- 172.21.15.137:0/2871430711 shutdown_connections 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a5080 0x55bded2af400 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2871430711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a4760 0x55bded2a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 -- 172.21.15.137:0/2871430711 >> 172.21.15.137:0/2871430711 conn(0x55bded2a0cf0 msgr2=0x55bded2a10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 -- 172.21.15.137:0/2871430711 shutdown_connections 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.343+0000 7f569cd1c740 1 -- 172.21.15.137:0/2871430711 wait complete. 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 -- start start 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 Processor -- start 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2a5080 0x55bded2acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2af940 0x55bded2ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bded2b2510 con 0x55bded2af940 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bded1926e0 con 0x55bded2a4760 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bded2b2690 con 0x55bded2a5080 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.344+0000 7f569cd1c740 10 client.0 _ll_register_callbacks cb 0x55bded1b6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2af940 0x55bded2ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2af940 0x55bded2ad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40298/0 (socket says 172.21.15.137:40298) 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 -- 172.21.15.137:0/1849390459 learned_addr learned my addr 172.21.15.137:0/1849390459 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f5693fff640 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2a5080 0x55bded2acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 -- 172.21.15.137:0/1849390459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2a5080 msgr2=0x55bded2acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2a5080 0x55bded2acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 -- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 msgr2=0x55bded2ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 -- 172.21.15.137:0/1849390459 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bded2bb980 con 0x55bded2af940 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56937fe640 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2af940 0x55bded2ad1c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f567c011950 tx=0x7f567c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f45a984afef854 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56737fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56737fe640 1 -- 172.21.15.137:0/1849390459 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c00fa90 con 0x55bded2af940 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56737fe640 1 -- 172.21.15.137:0/1849390459 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f567c00fc30 con 0x55bded2af940 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56840044b0 con 0x55bded2af940 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f56737fe640 1 -- 172.21.15.137:0/1849390459 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f567c02c050 con 0x55bded2af940 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f569cd1c740 10 client.16338 fetch_fsmap learned FSMap version 34 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.345+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bded2be310 con 0x55bded2af940 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f56737fe640 1 -- 172.21.15.137:0/1849390459 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56840044b0 con 0x55bded2af940 2024-09-12T12:20:55.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f56737fe640 1 -- 172.21.15.137:0/1849390459 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f567c003990 con 0x55bded2af940 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f569cd1c740 10 client.16338 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f569cd1c740 10 client.16338 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f569cd1c740 20 client.16338 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded2bf7d0 0x55bded2c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f569cd1c740 4 client.16338 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bded2179d0 con 0x55bded2bf7d0 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f5693fff640 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded2bf7d0 0x55bded2c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f5693fff640 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded2bf7d0 0x55bded2c1be0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c104f9425700298e server_cookie=4b2b48ad60b51e5b in_seq=0 out_seq=0 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.346+0000 7f56737fe640 10 client.16338 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f56737fe640 1 -- 172.21.15.137:0/1849390459 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bded2179d0 con 0x55bded2bf7d0 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f56737fe640 10 client.16338 handle_command_reply: tid=0 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 client.16338 shutdown 2024-09-12T12:20:55.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2af940 msgr2=0x55bded2ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2af940 0x55bded2ad1c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f567c011950 tx=0x7f567c005850 comp rx=0 tx=0).stop 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 shutdown_connections 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded2bf7d0 0x55bded2c1be0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2af940 0x55bded2ad1c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2a5080 0x55bded2acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f56737fe640 0 client.16338 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1849390459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 >> 172.21.15.137:0/1849390459 conn(0x55bded2a0cf0 msgr2=0x55bded29fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 shutdown_connections 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 1 -- 172.21.15.137:0/1849390459 wait complete. 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.347+0000 7f569cd1c740 20 client.16338 trim_cache size 0 max 16384 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.364+0000 7f569cd1c740 1 Processor -- start 2024-09-12T12:20:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.364+0000 7f569cd1c740 1 -- start start 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f569cd1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b0060 con 0x55bded2a5080 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f569cd1c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f56941ad900 con 0x55bded2a4760 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f569cd1c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56941ab750 con 0x55bded2af940 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f56937fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2c79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40308/0 (socket says 172.21.15.137:40308) 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 -- 172.21.15.137:0/2417862031 learned_addr learned my addr 172.21.15.137:0/2417862031 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f56937fe640 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 -- 172.21.15.137:0/2417862031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 msgr2=0x55bded2a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2a99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 -- 172.21.15.137:0/2417862031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 msgr2=0x55bded2c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 -- 172.21.15.137:0/2417862031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bded2a0b20 con 0x55bded2a5080 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5693fff640 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2c79f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f5684005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16a22527841dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5689ffb640 1 -- 172.21.15.137:0/2417862031 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684033070 con 0x55bded2a5080 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5689ffb640 1 -- 172.21.15.137:0/2417862031 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684004430 con 0x55bded2a5080 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.365+0000 7f5689ffb640 1 -- 172.21.15.137:0/2417862031 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568402e410 con 0x55bded2a5080 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 -- 172.21.15.137:0/2417862031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 msgr2=0x55bded2c79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2c79f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f5684005f90 comp rx=0 tx=0).stop 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 -- 172.21.15.137:0/2417862031 shutdown_connections 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2c79f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 --2- 172.21.15.137:0/2417862031 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 -- 172.21.15.137:0/2417862031 >> 172.21.15.137:0/2417862031 conn(0x55bded2a0cf0 msgr2=0x55bded2ae7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 -- 172.21.15.137:0/2417862031 shutdown_connections 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.366+0000 7f569cd1c740 1 -- 172.21.15.137:0/2417862031 wait complete. 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 -- start start 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 Processor -- start 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56941abdc0 con 0x55bded2a5080 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f56941ad230 con 0x55bded2a4760 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56941ac490 con 0x55bded2af940 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f569cd1c740 10 client.0 _ll_register_callbacks cb 0x7f567c0115a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f5693fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f56937fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f56937fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:52502/0 (socket says 172.21.15.137:52502) 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f5693fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40314/0 (socket says 172.21.15.137:40314) 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f56937fe640 1 -- 172.21.15.137:0/1433778552 learned_addr learned my addr 172.21.15.137:0/1433778552 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.367+0000 7f56937fe640 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56937fe640 1 -- 172.21.15.137:0/1433778552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 msgr2=0x55bded2b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56937fe640 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56937fe640 1 -- 172.21.15.137:0/1433778552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 msgr2=0x55bded2b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56937fe640 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56937fe640 1 -- 172.21.15.137:0/1433778552 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bded29c260 con 0x55bded2a4760 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f5693fff640 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56937fe640 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2b78e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f567c003d90 tx=0x7f567c015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a1ba2b4d077fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56737fe640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56737fe640 1 -- 172.21.15.137:0/1433778552 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c004390 con 0x55bded2a4760 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684004430 con 0x55bded2a4760 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56737fe640 1 -- 172.21.15.137:0/1433778552 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f567c0035f0 con 0x55bded2a4760 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56737fe640 1 -- 172.21.15.137:0/1433778552 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f567c040020 con 0x55bded2a4760 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 10 client.25754 fetch_fsmap learned FSMap version 34 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bded32aa30 con 0x55bded2a4760 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56737fe640 1 -- 172.21.15.137:0/1433778552 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684004430 con 0x55bded2a4760 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f56737fe640 1 -- 172.21.15.137:0/1433778552 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f567c045690 con 0x55bded2a4760 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 10 client.25754 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 10 client.25754 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 20 client.25754 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded32b170 0x55bded32d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:55.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 4 client.25754 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.368+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x55bded211d20 con 0x55bded32b170 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.369+0000 7f5693fff640 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded32b170 0x55bded32d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.369+0000 7f5693fff640 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded32b170 0x55bded32d580 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1b950e333f84e1b server_cookie=4ac75aa8a105c182 in_seq=0 out_seq=0 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.369+0000 7f56737fe640 10 client.25754 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f56737fe640 1 -- 172.21.15.137:0/1433778552 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bded211d20 con 0x55bded32b170 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f56737fe640 10 client.25754 handle_command_reply: tid=0 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 client.25754 shutdown 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 msgr2=0x55bded2b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2b78e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f567c003d90 tx=0x7f567c015040 comp rx=0 tx=0).stop 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 shutdown_connections 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bded2af940 0x55bded2b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bded32b170 0x55bded32d580 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bded2a5080 0x55bded2b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 --2- 172.21.15.137:0/1433778552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bded2a4760 0x55bded2b78e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f56737fe640 0 client.25754 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 >> 172.21.15.137:0/1433778552 conn(0x55bded2a0cf0 msgr2=0x55bded2a61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 shutdown_connections 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 1 -- 172.21.15.137:0/1433778552 wait complete. 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.444+0000 7f569cd1c740 20 client.25754 trim_cache size 0 max 16384 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 34, 2024-09-12T12:20:55.462 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 30, 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "b21614dd": { 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 30, 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 139.4, 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:20:55.463 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:20:55.464 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56800a2cf0 msgr2=0x7f56800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56800a2cf0 0x7f56800a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f568c006010 tx=0x7f568c005c30 comp rx=0 tx=0).stop 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 msgr2=0x7f569419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569419ce60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f568c009ca0 tx=0x7f568c02f350 comp rx=0 tx=0).stop 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 shutdown_connections 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56800a2cf0 0x7f56800a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5694108300 0x7f569419ce60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5694103780 0x7f5694198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 --2- 172.21.15.137:0/3114821794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5694102dd0 0x7f5694198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 >> 172.21.15.137:0/3114821794 conn(0x7f56940fc7d0 msgr2=0x7f5694105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 shutdown_connections 2024-09-12T12:20:55.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:20:55.445+0000 7f569a943640 1 -- 172.21.15.137:0/3114821794 wait complete. 2024-09-12T12:20:55.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.57 MB/sec execute 270 sec latency 296.047 ms 2024-09-12T12:20:56.195 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b21614dd', quiesce took 3.6 seconds. Will release after: 15.799999999999999 2024-09-12T12:20:56.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.54 MB/sec execute 271 sec latency 1296.088 ms 2024-09-12T12:20:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/1298350873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4106b70 msgr2=0x7f4dc4106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/1298350873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4106b70 0x7f4dc4106f50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009900 tx=0x7f4dc002a410 comp rx=0 tx=0).stop 2024-09-12T12:20:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/1298350873 shutdown_connections 2024-09-12T12:20:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/1298350873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc41020c0 0x7f4dc410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/1298350873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dc4101740 0x7f4dc4101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/1298350873 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4106b70 0x7f4dc4106f50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/1298350873 >> 172.21.15.137:0/1298350873 conn(0x7f4dc4075ef0 msgr2=0x7f4dc4076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/1298350873 shutdown_connections 2024-09-12T12:20:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.864+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/1298350873 wait complete. 2024-09-12T12:20:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 Processor -- start 2024-09-12T12:20:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 -- start start 2024-09-12T12:20:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4101740 0x7f4dc41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dc41020c0 0x7f4dc4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc4106b70 0x7f4dc419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4110ed0 con 0x7f4dc4101740 2024-09-12T12:20:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4110d50 con 0x7f4dc41020c0 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dcb7f1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc4111050 con 0x7f4dc4106b70 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4101740 0x7f4dc41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4101740 0x7f4dc41972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40336/0 (socket says 172.21.15.137:40336) 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 -- 172.21.15.137:0/390274525 learned_addr learned my addr 172.21.15.137:0/390274525 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc4106b70 0x7f4dc419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dc41020c0 0x7f4dc4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 -- 172.21.15.137:0/390274525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc4106b70 msgr2=0x7f4dc419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc4106b70 0x7f4dc419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 -- 172.21.15.137:0/390274525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dc41020c0 msgr2=0x7f4dc4197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dc41020c0 0x7f4dc4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 -- 172.21.15.137:0/390274525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc419c2d0 con 0x7f4dc4101740 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.865+0000 7f4dc958d640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4101740 0x7f4dc41972e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006010 tx=0x7f4dc00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.866+0000 7f4dba7fc640 1 -- 172.21.15.137:0/390274525 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc003f030 con 0x7f4dc4101740 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.866+0000 7f4dba7fc640 1 -- 172.21.15.137:0/390274525 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dc0003ce0 con 0x7f4dc4101740 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.866+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc419c560 con 0x7f4dc4101740 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.866+0000 7f4dba7fc640 1 -- 172.21.15.137:0/390274525 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0035070 con 0x7f4dc4101740 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.866+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc41a3e50 con 0x7f4dc4101740 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.868+0000 7f4dba7fc640 1 -- 172.21.15.137:0/390274525 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4dc0042030 con 0x7f4dc4101740 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.868+0000 7f4dce663740 1 Processor -- start 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.868+0000 7f4dce663740 1 -- start start 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dce663740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1c970 0x55d331a1cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dce663740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1d290 0x55d331a29980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dce663740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dce663740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d3319812f0 con 0x55d331a1c970 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dce663740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d331929270 con 0x55d331a1d290 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dce663740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d33197d2b0 con 0x55d331a29ec0 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1c970 0x55d331a1cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1c970 0x55d331a1cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40338/0 (socket says 172.21.15.137:40338) 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/637909885 learned_addr learned my addr 172.21.15.137:0/637909885 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dba7fc640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4db00a2da0 0x7f4db00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc958d640 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1d290 0x55d331a29980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4db00a2da0 0x7f4db00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/637909885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 msgr2=0x55d331a2c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/637909885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1d290 msgr2=0x55d331a29980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1d290 0x55d331a29980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dba7fc640 1 -- 172.21.15.137:0/390274525 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4dc00efa90 con 0x7f4dc4101740 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/637909885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d331928340 con 0x55d331a1c970 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1c970 0x55d331a1cd50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc015970 tx=0x7f4dbc015e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0a5a51a738dfc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4db00a2da0 0x7f4db00a5260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc012760 tx=0x7f4dbc01b000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4daf7fe640 1 -- 172.21.15.137:0/637909885 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc022070 con 0x55d331a1c970 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4daf7fe640 1 -- 172.21.15.137:0/637909885 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc004d40 con 0x55d331a1c970 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.869+0000 7f4daf7fe640 1 -- 172.21.15.137:0/637909885 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc005230 con 0x55d331a1c970 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 -- 172.21.15.137:0/637909885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1c970 msgr2=0x55d331a1cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1c970 0x55d331a1cd50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc015970 tx=0x7f4dbc015e30 comp rx=0 tx=0).stop 2024-09-12T12:20:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 -- 172.21.15.137:0/637909885 shutdown_connections 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1d290 0x55d331a29980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 --2- 172.21.15.137:0/637909885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1c970 0x55d331a1cd50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 -- 172.21.15.137:0/637909885 >> 172.21.15.137:0/637909885 conn(0x55d331a19060 msgr2=0x55d331a19440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 -- 172.21.15.137:0/637909885 shutdown_connections 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.870+0000 7f4dce663740 1 -- 172.21.15.137:0/637909885 wait complete. 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 -- start start 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 Processor -- start 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a26e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1d290 0x55d331a273b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a29ec0 0x55d331a278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d331a2ca90 con 0x55d331a29ec0 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d33190a290 con 0x55d331a1c970 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d331a2cc10 con 0x55d331a1d290 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dce663740 10 client.0 _ll_register_callbacks cb 0x55d33193b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a29ec0 0x55d331a278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a29ec0 0x55d331a278f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40348/0 (socket says 172.21.15.137:40348) 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/1450863474 learned_addr learned my addr 172.21.15.137:0/1450863474 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc958d640 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a26e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1d290 0x55d331a273b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/1450863474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1d290 msgr2=0x55d331a273b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1d290 0x55d331a273b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 msgr2=0x55d331a26e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a26e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.871+0000 7f4dc9d8e640 1 -- 172.21.15.137:0/1450863474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d331694210 con 0x55d331a29ec0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a29ec0 0x55d331a278f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc018040 tx=0x7f4dbc010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83424cfcdec92cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dacff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dacff9640 1 -- 172.21.15.137:0/1450863474 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc02b070 con 0x55d331a29ec0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dacff9640 1 -- 172.21.15.137:0/1450863474 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4dbc01d440 con 0x55d331a29ec0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dbc004d40 con 0x55d331a29ec0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dacff9640 1 -- 172.21.15.137:0/1450863474 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc037050 con 0x55d331a29ec0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 10 client.16356 fetch_fsmap learned FSMap version 34 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d331760620 con 0x55d331a29ec0 2024-09-12T12:20:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dacff9640 1 -- 172.21.15.137:0/1450863474 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dbc004d40 con 0x55d331a29ec0 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dacff9640 1 -- 172.21.15.137:0/1450863474 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f4dbc027410 con 0x55d331a29ec0 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 10 client.16356 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 20 client.16356 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331a39dd0 0x55d331a3c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 4 client.16356 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d33198ea20 con 0x55d331a39dd0 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.872+0000 7f4dc958d640 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331a39dd0 0x55d331a3c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.873+0000 7f4dacff9640 10 client.16356 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.873+0000 7f4dc958d640 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331a39dd0 0x55d331a3c1c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412bf932573be7e6 server_cookie=c48ba84c478060d in_seq=0 out_seq=0 2024-09-12T12:20:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.873+0000 7f4dacff9640 1 -- 172.21.15.137:0/1450863474 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d33198ea20 con 0x55d331a39dd0 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.873+0000 7f4dacff9640 10 client.16356 handle_command_reply: tid=0 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.873+0000 7f4dce663740 1 client.16356 shutdown 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a29ec0 msgr2=0x55d331a278f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a29ec0 0x55d331a278f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc018040 tx=0x7f4dbc010b40 comp rx=0 tx=0).stop 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 shutdown_connections 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331a39dd0 0x55d331a3c1c0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a29ec0 0x55d331a278f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1d290 0x55d331a273b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dacff9640 0 client.16356 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 --2- 172.21.15.137:0/1450863474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a26e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 >> 172.21.15.137:0/1450863474 conn(0x55d331a19060 msgr2=0x55d331a1edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 shutdown_connections 2024-09-12T12:20:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 1 -- 172.21.15.137:0/1450863474 wait complete. 2024-09-12T12:20:56.890 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.874+0000 7f4dce663740 20 client.16356 trim_cache size 0 max 16384 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 Processor -- start 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 -- start start 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1c970 0x55d331a27720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a180b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a29ec0 0x55d331a419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41addc0 con 0x55d331a1d290 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41ab660 con 0x55d331a29ec0 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dce663740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41a94d0 con 0x55d331a1c970 2024-09-12T12:20:56.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a180b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40350/0 (socket says 172.21.15.137:40350) 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/4149641318 learned_addr learned my addr 172.21.15.137:0/4149641318 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc958d640 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1c970 0x55d331a27720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a29ec0 0x55d331a419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/4149641318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1c970 msgr2=0x55d331a27720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1c970 0x55d331a27720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/4149641318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a29ec0 msgr2=0x55d331a419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a29ec0 0x55d331a419a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/4149641318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d331a27c60 con 0x55d331a1d290 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.888+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a180b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009af0 tx=0x7f4db4013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b38822d48f3473a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4daeffd640 1 -- 172.21.15.137:0/4149641318 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db400f650 con 0x55d331a1d290 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4daeffd640 1 -- 172.21.15.137:0/4149641318 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db400fcd0 con 0x55d331a1d290 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4daeffd640 1 -- 172.21.15.137:0/4149641318 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db401b3e0 con 0x55d331a1d290 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 -- 172.21.15.137:0/4149641318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 msgr2=0x55d331a180b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a180b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009af0 tx=0x7f4db4013460 comp rx=0 tx=0).stop 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 -- 172.21.15.137:0/4149641318 shutdown_connections 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a29ec0 0x55d331a419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a180b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 --2- 172.21.15.137:0/4149641318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a1c970 0x55d331a27720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 -- 172.21.15.137:0/4149641318 >> 172.21.15.137:0/4149641318 conn(0x55d331a19060 msgr2=0x55d331a179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 -- 172.21.15.137:0/4149641318 shutdown_connections 2024-09-12T12:20:56.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 -- 172.21.15.137:0/4149641318 wait complete. 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.889+0000 7f4dce663740 1 -- start start 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 1 Processor -- start 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a32500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a2d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41a9b20 con 0x55d331a1d290 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41aaf90 con 0x55d331a1c970 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41aa1f0 con 0x55d331a29ec0 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dce663740 10 client.0 _ll_register_callbacks cb 0x7f4dbc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a2d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a2d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40364/0 (socket says 172.21.15.137:40364) 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/3800087834 learned_addr learned my addr 172.21.15.137:0/3800087834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc9d8e640 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc958d640 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a32500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/3800087834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 msgr2=0x55d331a2dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 msgr2=0x55d331a32500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a32500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.890+0000 7f4dc8d8c640 1 -- 172.21.15.137:0/3800087834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d331a2e590 con 0x55d331a1d290 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dc8d8c640 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a2d820 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4db40119e0 tx=0x7f4db400ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0ab18f67f798b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:20:56.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dacff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dacff9640 1 -- 172.21.15.137:0/3800087834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db402a030 con 0x55d331a1d290 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dacff9640 1 -- 172.21.15.137:0/3800087834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8440+0+0 (secure 0 0 0) 0x7f4db4021070 con 0x55d331a1d290 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db400fcd0 con 0x55d331a1d290 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dacff9640 1 -- 172.21.15.137:0/3800087834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4db4034050 con 0x55d331a1d290 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 10 client.16362 fetch_fsmap learned FSMap version 34 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d331aa48b0 con 0x55d331a1d290 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dacff9640 1 -- 172.21.15.137:0/3800087834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db400fcd0 con 0x55d331a1d290 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dacff9640 1 -- 172.21.15.137:0/3800087834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f4db401a070 con 0x55d331a1d290 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 10 client.16362 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:20:56.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:20:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 20 client.16362 populate_metadata read hostname 'smithi137' 2024-09-12T12:20:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331aa4fa0 0x55d331aa73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:20:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 4 client.16362 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:20:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.891+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d331988d70 con 0x55d331aa4fa0 2024-09-12T12:20:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.892+0000 7f4dc958d640 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331aa4fa0 0x55d331aa73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:20:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.892+0000 7f4dacff9640 10 client.16362 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.892+0000 7f4dc958d640 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331aa4fa0 0x55d331aa73c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f20e1490ac9badb1 server_cookie=552a331d365c7eb in_seq=0 out_seq=0 2024-09-12T12:20:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.957+0000 7f4dacff9640 1 -- 172.21.15.137:0/3800087834 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d331988d70 con 0x55d331aa4fa0 2024-09-12T12:20:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.957+0000 7f4dacff9640 10 client.16362 handle_command_reply: tid=0 2024-09-12T12:20:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.957+0000 7f4dce663740 1 client.16362 shutdown 2024-09-12T12:20:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 msgr2=0x55d331a2d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a2d820 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4db40119e0 tx=0x7f4db400ff90 comp rx=0 tx=0).stop 2024-09-12T12:20:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 shutdown_connections 2024-09-12T12:20:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d331a29ec0 0x55d331a2dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d331aa4fa0 0x55d331aa73c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d331a1d290 0x55d331a2d820 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dacff9640 0 client.16362 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 --2- 172.21.15.137:0/3800087834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d331a1c970 0x55d331a32500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 >> 172.21.15.137:0/3800087834 conn(0x55d331a19060 msgr2=0x55d331a179a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 shutdown_connections 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 1 -- 172.21.15.137:0/3800087834 wait complete. 2024-09-12T12:20:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.958+0000 7f4dce663740 20 client.16362 trim_cache size 0 max 16384 2024-09-12T12:20:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:20:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:20:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:20:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:20:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4db00a2da0 msgr2=0x7f4db00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4db00a2da0 0x7f4db00a5260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc012760 tx=0x7f4dbc01b000 comp rx=0 tx=0).stop 2024-09-12T12:20:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4101740 msgr2=0x7f4dc41972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:20:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4101740 0x7f4dc41972e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006010 tx=0x7f4dc00042f0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 shutdown_connections 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4db00a2da0 0x7f4db00a5260 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dc4106b70 0x7f4dc419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dc41020c0 0x7f4dc4197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 --2- 172.21.15.137:0/390274525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dc4101740 0x7f4dc41972e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 >> 172.21.15.137:0/390274525 conn(0x7f4dc4075ef0 msgr2=0x7f4dc40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 shutdown_connections 2024-09-12T12:20:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:20:56.959+0000 7f4dcb7f1640 1 -- 172.21.15.137:0/390274525 wait complete. 2024-09-12T12:20:57.002 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:56 smithi171 ceph-mon[35098]: pgmap v239: 97 pgs: 97 active+clean; 199 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-09-12T12:20:57.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:56 smithi137 ceph-mon[30950]: pgmap v239: 97 pgs: 97 active+clean; 199 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-09-12T12:20:57.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:56 smithi189 ceph-mon[35475]: pgmap v239: 97 pgs: 97 active+clean; 199 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.6 MiB/s wr, 339 op/s 2024-09-12T12:20:57.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.51 MB/sec execute 272 sec latency 2296.120 ms 2024-09-12T12:20:58.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.47 MB/sec execute 273 sec latency 3296.156 ms 2024-09-12T12:20:59.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:20:58 smithi137 ceph-mon[30950]: pgmap v240: 97 pgs: 97 active+clean; 199 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-09-12T12:20:59.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:20:58 smithi189 ceph-mon[35475]: pgmap v240: 97 pgs: 97 active+clean; 199 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-09-12T12:20:59.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:20:58 smithi171 ceph-mon[35098]: pgmap v240: 97 pgs: 97 active+clean; 199 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-09-12T12:20:59.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.44 MB/sec execute 274 sec latency 4296.197 ms 2024-09-12T12:21:00.738 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.41 MB/sec execute 275 sec latency 5296.252 ms 2024-09-12T12:21:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:00 smithi137 ceph-mon[30950]: pgmap v241: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-09-12T12:21:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:00 smithi189 ceph-mon[35475]: pgmap v241: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-09-12T12:21:01.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:00 smithi171 ceph-mon[35098]: pgmap v241: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2024-09-12T12:21:01.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.38 MB/sec execute 276 sec latency 6296.317 ms 2024-09-12T12:21:02.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.35 MB/sec execute 277 sec latency 7296.439 ms 2024-09-12T12:21:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:02 smithi137 ceph-mon[30950]: pgmap v242: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-12T12:21:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:02 smithi189 ceph-mon[35475]: pgmap v242: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-12T12:21:03.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:02 smithi171 ceph-mon[35098]: pgmap v242: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-12T12:21:03.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.32 MB/sec execute 278 sec latency 8296.578 ms 2024-09-12T12:21:04.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.29 MB/sec execute 279 sec latency 9296.697 ms 2024-09-12T12:21:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:04 smithi137 ceph-mon[30950]: pgmap v243: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-12T12:21:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:04 smithi189 ceph-mon[35475]: pgmap v243: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-12T12:21:05.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:04 smithi171 ceph-mon[35098]: pgmap v243: 97 pgs: 97 active+clean; 188 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 183 op/s 2024-09-12T12:21:05.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.26 MB/sec execute 280 sec latency 10296.803 ms 2024-09-12T12:21:06.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.23 MB/sec execute 281 sec latency 11296.902 ms 2024-09-12T12:21:07.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:06 smithi137 ceph-mon[30950]: pgmap v244: 97 pgs: 97 active+clean; 187 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-09-12T12:21:07.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:06 smithi189 ceph-mon[35475]: pgmap v244: 97 pgs: 97 active+clean; 187 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-09-12T12:21:07.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:06 smithi171 ceph-mon[35098]: pgmap v244: 97 pgs: 97 active+clean; 187 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-09-12T12:21:07.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.20 MB/sec execute 282 sec latency 12296.975 ms 2024-09-12T12:21:08.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.18 MB/sec execute 283 sec latency 13297.055 ms 2024-09-12T12:21:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:08 smithi137 ceph-mon[30950]: pgmap v245: 97 pgs: 97 active+clean; 187 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-12T12:21:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:08 smithi137 ceph-mon[30950]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T12:21:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:08 smithi189 ceph-mon[35475]: pgmap v245: 97 pgs: 97 active+clean; 187 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-12T12:21:09.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:08 smithi189 ceph-mon[35475]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T12:21:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:08 smithi171 ceph-mon[35098]: pgmap v245: 97 pgs: 97 active+clean; 187 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-12T12:21:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:08 smithi171 ceph-mon[35098]: osdmap e91: 12 total, 12 up, 12 in 2024-09-12T12:21:09.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.15 MB/sec execute 284 sec latency 14297.154 ms 2024-09-12T12:21:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 -- 172.21.15.137:0/659946701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb00105eb0 msgr2=0x7fbb00106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 --2- 172.21.15.137:0/659946701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb00105eb0 0x7fbb00106320 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009ca0 tx=0x7fbaf802a950 comp rx=0 tx=0).stop 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 -- 172.21.15.137:0/659946701 shutdown_connections 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 --2- 172.21.15.137:0/659946701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb00105eb0 0x7fbb00106320 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 --2- 172.21.15.137:0/659946701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb001013a0 0x7fbb00105860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 --2- 172.21.15.137:0/659946701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb00100a80 0x7fbb00100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 -- 172.21.15.137:0/659946701 >> 172.21.15.137:0/659946701 conn(0x7fbb000fc700 msgr2=0x7fbb000feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 -- 172.21.15.137:0/659946701 shutdown_connections 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.059+0000 7fbb080f1640 1 -- 172.21.15.137:0/659946701 wait complete. 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 Processor -- start 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 -- start start 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb00100a80 0x7fbb001a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb001013a0 0x7fbb001a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb00105eb0 0x7fbb0019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001184a0 con 0x7fbb001013a0 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00118320 con 0x7fbb00105eb0 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb080f1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb00118620 con 0x7fbb00100a80 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb0568c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb001013a0 0x7fbb001a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb0568c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb001013a0 0x7fbb001a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42364/0 (socket says 172.21.15.137:42364) 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.060+0000 7fbb0568c640 1 -- 172.21.15.137:0/2010344738 learned_addr learned my addr 172.21.15.137:0/2010344738 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0668e640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb00105eb0 0x7fbb0019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb00100a80 0x7fbb001a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0568c640 1 -- 172.21.15.137:0/2010344738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb00100a80 msgr2=0x7fbb001a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0568c640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb00100a80 0x7fbb001a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0568c640 1 -- 172.21.15.137:0/2010344738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb00105eb0 msgr2=0x7fbb0019e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0568c640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb00105eb0 0x7fbb0019e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0568c640 1 -- 172.21.15.137:0/2010344738 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0019ef70 con 0x7fbb001013a0 2024-09-12T12:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0668e640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb00105eb0 0x7fbb0019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb0568c640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb001013a0 0x7fbb001a4b30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d740 tx=0x7fbaf000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbaf6ffd640 1 -- 172.21.15.137:0/2010344738 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0017070 con 0x7fbb001013a0 2024-09-12T12:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbaf6ffd640 1 -- 172.21.15.137:0/2010344738 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf000de50 con 0x7fbb001013a0 2024-09-12T12:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0019f260 con 0x7fbb001013a0 2024-09-12T12:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbaf6ffd640 1 -- 172.21.15.137:0/2010344738 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000f620 con 0x7fbb001013a0 2024-09-12T12:21:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.061+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb001ab570 con 0x7fbb001013a0 2024-09-12T12:21:10.070 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbaf6ffd640 1 -- 172.21.15.137:0/2010344738 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbaf0009c30 con 0x7fbb001013a0 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 Processor -- start 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 -- start start 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e77f5062f0 con 0x55e77f5a1af0 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e77f4ae270 con 0x55e77f5accb0 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.064+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e77f5022b0 con 0x55e77f5a2410 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42368/0 (socket says 172.21.15.137:42368) 2024-09-12T12:21:10.071 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 -- 172.21.15.137:0/993706343 learned_addr learned my addr 172.21.15.137:0/993706343 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0568c640 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 -- 172.21.15.137:0/993706343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 msgr2=0x55e77f5ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 -- 172.21.15.137:0/993706343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 msgr2=0x55e77f5af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbaf6ffd640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbaec0a2e20 0x7fbaec0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 -- 172.21.15.137:0/993706343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e77f4ad340 con 0x55e77f5a1af0 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0568c640 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbaf6ffd640 1 -- 172.21.15.137:0/2010344738 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8528+0+0 (secure 0 0 0) 0x7fbaf00cefd0 con 0x7fbb001013a0 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a1ed0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009a50 tx=0x7fbaf802f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd47db033bdb8cfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:10.072 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbb0668e640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbaec0a2e20 0x7fbaec0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbaebfff640 1 -- 172.21.15.137:0/993706343 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8034040 con 0x55e77f5a1af0 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbaebfff640 1 -- 172.21.15.137:0/993706343 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf8003e50 con 0x55e77f5a1af0 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.065+0000 7fbaebfff640 1 -- 172.21.15.137:0/993706343 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8044440 con 0x55e77f5a1af0 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/993706343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 msgr2=0x55e77f5a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0668e640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbaec0a2e20 0x7fbaec0a52e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80049d0 tx=0x7fbaf8004940 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a1ed0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009a50 tx=0x7fbaf802f940 comp rx=0 tx=0).stop 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/993706343 shutdown_connections 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/993706343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a1ed0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/993706343 >> 172.21.15.137:0/993706343 conn(0x55e77f59e060 msgr2=0x55e77f59e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/993706343 shutdown_connections 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.066+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/993706343 wait complete. 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 -- start start 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 Processor -- start 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a2410 0x55e77f5aa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5accb0 0x55e77f5aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e77f5af880 con 0x55e77f5a1af0 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e77f48f290 con 0x55e77f5a2410 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e77f5afa00 con 0x55e77f5accb0 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0ae8f740 10 client.0 _ll_register_callbacks cb 0x55e77f4c0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:10.074 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb05e8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb05e8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42382/0 (socket says 172.21.15.137:42382) 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb05e8d640 1 -- 172.21.15.137:0/387676526 learned_addr learned my addr 172.21.15.137:0/387676526 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.067+0000 7fbb0568c640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a2410 0x55e77f5aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb0668e640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5accb0 0x55e77f5aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb05e8d640 1 -- 172.21.15.137:0/387676526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5accb0 msgr2=0x55e77f5aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5accb0 0x55e77f5aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb05e8d640 1 -- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a2410 msgr2=0x55e77f5aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a2410 0x55e77f5aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb05e8d640 1 -- 172.21.15.137:0/387676526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e77f219210 con 0x55e77f5a1af0 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb0668e640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5accb0 0x55e77f5aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a9be0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbafc010840 tx=0x7fbafc010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7be67b58c336a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:10.075 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbae97fa640 1 -- 172.21.15.137:0/387676526 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00ed40 con 0x55e77f5a1af0 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbae97fa640 1 -- 172.21.15.137:0/387676526 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8528+0+0 (secure 0 0 0) 0x7fbafc01c480 con 0x55e77f5a1af0 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaf8003e50 con 0x55e77f5a1af0 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbae97fa640 1 -- 172.21.15.137:0/387676526 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbafc010e90 con 0x55e77f5a1af0 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb0ae8f740 10 client.16374 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e77f2e5620 con 0x55e77f5a1af0 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbae97fa640 1 -- 172.21.15.137:0/387676526 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaf8003e50 con 0x55e77f5a1af0 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.068+0000 7fbae97fa640 1 -- 172.21.15.137:0/387676526 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fbafc0117a0 con 0x55e77f5a1af0 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0ae8f740 10 client.16374 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0ae8f740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0ae8f740 20 client.16374 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f5bcca0 0x55e77f5bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.076 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0ae8f740 4 client.16374 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e77f513a20 con 0x55e77f5bcca0 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0568c640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f5bcca0 0x55e77f5bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbae97fa640 10 client.16374 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.069+0000 7fbb0568c640 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f5bcca0 0x55e77f5bf090 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=893ce275d37deec7 server_cookie=35758b1b9a2a0295 in_seq=0 out_seq=0 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbae97fa640 1 -- 172.21.15.137:0/387676526 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e77f513a20 con 0x55e77f5bcca0 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbae97fa640 10 client.16374 handle_command_reply: tid=0 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 client.16374 shutdown 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 msgr2=0x55e77f5a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a9be0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbafc010840 tx=0x7fbafc010c40 comp rx=0 tx=0).stop 2024-09-12T12:21:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 shutdown_connections 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5accb0 0x55e77f5aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a2410 0x55e77f5aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f5bcca0 0x55e77f5bf090 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/387676526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a9be0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbae97fa640 0 client.16374 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 >> 172.21.15.137:0/387676526 conn(0x55e77f59e060 msgr2=0x55e77f5a3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 shutdown_connections 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/387676526 wait complete. 2024-09-12T12:21:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.070+0000 7fbb0ae8f740 20 client.16374 trim_cache size 0 max 16384 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 Processor -- start 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 -- start start 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5a4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001b6750 con 0x55e77f5a1af0 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001b3ff0 con 0x55e77f5accb0 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001b1e40 con 0x55e77f5a2410 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42388/0 (socket says 172.21.15.137:42388) 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 -- 172.21.15.137:0/1674238285 learned_addr learned my addr 172.21.15.137:0/1674238285 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0568c640 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb0668e640 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 -- 172.21.15.137:0/1674238285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 msgr2=0x55e77f5a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 -- 172.21.15.137:0/1674238285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 msgr2=0x55e77f5aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 -- 172.21.15.137:0/1674238285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e77f5a8110 con 0x55e77f5a1af0 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a8b00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbafc011bc0 tx=0x7fbafc0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00bf19d001af6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.085+0000 7fbaeb7fe640 1 -- 172.21.15.137:0/1674238285 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc032070 con 0x55e77f5a1af0 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbaeb7fe640 1 -- 172.21.15.137:0/1674238285 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc011920 con 0x55e77f5a1af0 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbaeb7fe640 1 -- 172.21.15.137:0/1674238285 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc029040 con 0x55e77f5a1af0 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1674238285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 msgr2=0x55e77f5a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a8b00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbafc011bc0 tx=0x7fbafc0104a0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1674238285 shutdown_connections 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5accb0 0x55e77f5aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1674238285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5a1af0 0x55e77f5a8b00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1674238285 >> 172.21.15.137:0/1674238285 conn(0x55e77f59e060 msgr2=0x55e77f5abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1674238285 shutdown_connections 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1674238285 wait complete. 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.086+0000 7fbb0ae8f740 1 -- start start 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 1 Processor -- start 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a1af0 0x55e77f5b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5accb0 0x55e77f5a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001b24b0 con 0x55e77f5accb0 2024-09-12T12:21:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001b3920 con 0x55e77f5a1af0 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb001b2b80 con 0x55e77f5a2410 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0ae8f740 10 client.0 _ll_register_callbacks cb 0x7fbafc00c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5accb0 0x55e77f5a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5accb0 0x55e77f5a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42398/0 (socket says 172.21.15.137:42398) 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 -- 172.21.15.137:0/1077755259 learned_addr learned my addr 172.21.15.137:0/1077755259 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0568c640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a1af0 0x55e77f5b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 -- 172.21.15.137:0/1077755259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 msgr2=0x55e77f5b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.092 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 -- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a1af0 msgr2=0x55e77f5b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a1af0 0x55e77f5b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0668e640 1 -- 172.21.15.137:0/1077755259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e77f5a6ea0 con 0x55e77f5accb0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb0568c640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.087+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a1af0 0x55e77f5b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0668e640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5accb0 0x55e77f5a6670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbaf802ae70 tx=0x7fbaf804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914eed1fe42a4bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbae97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbae97fa640 1 -- 172.21.15.137:0/1077755259 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf805f060 con 0x55e77f5accb0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbae97fa640 1 -- 172.21.15.137:0/1077755259 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8528+0+0 (secure 0 0 0) 0x7fbaf805a680 con 0x55e77f5accb0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc011920 con 0x55e77f5accb0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbae97fa640 1 -- 172.21.15.137:0/1077755259 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbaf8034040 con 0x55e77f5accb0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 10 client.16380 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e77f6275a0 con 0x55e77f5accb0 2024-09-12T12:21:10.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbae97fa640 1 -- 172.21.15.137:0/1077755259 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc011920 con 0x55e77f5accb0 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbae97fa640 1 -- 172.21.15.137:0/1077755259 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fbaf8042020 con 0x55e77f5accb0 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 10 client.16380 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 20 client.16380 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f627930 0x55e77f629d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 4 client.16380 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.088+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e77f50dd70 con 0x55e77f627930 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.089+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f627930 0x55e77f629d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.089+0000 7fbae97fa640 10 client.16380 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:10.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.089+0000 7fbb05e8d640 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f627930 0x55e77f629d20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c9f31acbc772e9 server_cookie=53de6fb91179531a in_seq=0 out_seq=0 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.121+0000 7fbae97fa640 1 -- 172.21.15.137:0/1077755259 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e77f50dd70 con 0x55e77f627930 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.121+0000 7fbae97fa640 10 client.16380 handle_command_reply: tid=0 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.121+0000 7fbb0ae8f740 1 client.16380 shutdown 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.121+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5accb0 msgr2=0x55e77f5a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.121+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5accb0 0x55e77f5a6670 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbaf802ae70 tx=0x7fbaf804b000 comp rx=0 tx=0).stop 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 shutdown_connections 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e77f627930 0x55e77f629d20 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e77f5accb0 0x55e77f5a6670 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbae97fa640 0 client.16380 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:10.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e77f5a2410 0x55e77f5b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 --2- 172.21.15.137:0/1077755259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e77f5a1af0 0x55e77f5b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 >> 172.21.15.137:0/1077755259 conn(0x55e77f59e060 msgr2=0x55e77f59c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 shutdown_connections 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 1 -- 172.21.15.137:0/1077755259 wait complete. 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb0ae8f740 20 client.16380 trim_cache size 0 max 16384 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:21:10.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:21:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbaec0a2e20 msgr2=0x7fbaec0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbaec0a2e20 0x7fbaec0a52e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80049d0 tx=0x7fbaf8004940 comp rx=0 tx=0).stop 2024-09-12T12:21:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb001013a0 msgr2=0x7fbb001a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb001013a0 0x7fbb001a4b30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000d740 tx=0x7fbaf000dc00 comp rx=0 tx=0).stop 2024-09-12T12:21:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 shutdown_connections 2024-09-12T12:21:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbaec0a2e20 0x7fbaec0a52e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb00105eb0 0x7fbb0019e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb001013a0 0x7fbb001a4b30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 --2- 172.21.15.137:0/2010344738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb00100a80 0x7fbb001a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 >> 172.21.15.137:0/2010344738 conn(0x7fbb000fc700 msgr2=0x7fbb00103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 shutdown_connections 2024-09-12T12:21:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:10.122+0000 7fbb080f1640 1 -- 172.21.15.137:0/2010344738 wait complete. 2024-09-12T12:21:10.739 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.12 MB/sec execute 285 sec latency 15297.227 ms 2024-09-12T12:21:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:10 smithi137 ceph-mon[30950]: pgmap v247: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-12T12:21:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:10 smithi189 ceph-mon[35475]: pgmap v247: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-12T12:21:11.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:10 smithi171 ceph-mon[35098]: pgmap v247: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-12T12:21:11.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.09 MB/sec execute 286 sec latency 16297.344 ms 2024-09-12T12:21:11.996 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:21:12.271 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:21:12.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.06 MB/sec execute 287 sec latency 17297.429 ms 2024-09-12T12:21:12.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 -- 172.21.15.137:0/2214364245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 msgr2=0x7f3cec1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:12.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/2214364245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 0x7f3cec1067f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009900 tx=0x7f3ce002a410 comp rx=0 tx=0).stop 2024-09-12T12:21:12.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 -- 172.21.15.137:0/2214364245 shutdown_connections 2024-09-12T12:21:12.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/2214364245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cec106f20 0x7f3cec10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:12.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/2214364245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 0x7f3cec1067f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:12.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/2214364245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec101df0 0x7f3cec1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:12.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 -- 172.21.15.137:0/2214364245 >> 172.21.15.137:0/2214364245 conn(0x7f3cec0fd8f0 msgr2=0x7f3cec0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 -- 172.21.15.137:0/2214364245 shutdown_connections 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.852+0000 7f3cf12a0640 1 -- 172.21.15.137:0/2214364245 wait complete. 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.853+0000 7f3cf12a0640 1 Processor -- start 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.853+0000 7f3cf12a0640 1 -- start start 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cf12a0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cec101df0 0x7f3cec1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cf12a0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 0x7f3cec1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cf12a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec106f20 0x7f3cec19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cf12a0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec111f80 con 0x7f3cec106f20 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cf12a0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec111e00 con 0x7f3cec101df0 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cf12a0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec112100 con 0x7f3cec1027a0 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec106f20 0x7f3cec19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec106f20 0x7f3cec19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42410/0 (socket says 172.21.15.137:42410) 2024-09-12T12:21:12.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 -- 172.21.15.137:0/1097684642 learned_addr learned my addr 172.21.15.137:0/1097684642 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceaffd640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cec101df0 0x7f3cec1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cea7fc640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 0x7f3cec1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 -- 172.21.15.137:0/1097684642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 msgr2=0x7f3cec1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 0x7f3cec1989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 -- 172.21.15.137:0/1097684642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cec101df0 msgr2=0x7f3cec1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cec101df0 0x7f3cec1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 -- 172.21.15.137:0/1097684642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec19d450 con 0x7f3cec106f20 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3ceb7fe640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec106f20 0x7f3cec19cd70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400eb10 tx=0x7f3ce400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cd3fff640 1 -- 172.21.15.137:0/1097684642 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4004b20 con 0x7f3cec106f20 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cd3fff640 1 -- 172.21.15.137:0/1097684642 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce4004cc0 con 0x7f3cec106f20 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.854+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec19d740 con 0x7f3cec106f20 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.855+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec1a5020 con 0x7f3cec106f20 2024-09-12T12:21:12.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.855+0000 7f3cd3fff640 1 -- 172.21.15.137:0/1097684642 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4010760 con 0x7f3cec106f20 2024-09-12T12:21:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.856+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd8005290 con 0x7f3cec106f20 2024-09-12T12:21:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.857+0000 7f3cd3fff640 1 -- 172.21.15.137:0/1097684642 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3ce4005000 con 0x7f3cec106f20 2024-09-12T12:21:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.857+0000 7f3cd3fff640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3cd40a2cf0 0x7f3cd40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.858+0000 7f3ceaffd640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3cd40a2cf0 0x7f3cd40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.858+0000 7f3cd3fff640 1 -- 172.21.15.137:0/1097684642 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f3ce40cfaa0 con 0x7f3cec106f20 2024-09-12T12:21:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.858+0000 7f3ceaffd640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3cd40a2cf0 0x7f3cd40a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009bd0 tx=0x7f3cdc005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:12.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:12.861+0000 7f3cd3fff640 1 -- 172.21.15.137:0/1097684642 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ce40541b0 con 0x7f3cec106f20 2024-09-12T12:21:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:12 smithi137 ceph-mon[30950]: pgmap v248: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-12T12:21:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:12 smithi137 ceph-mon[30950]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T12:21:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.063+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cd8005580 con 0x7f3cec106f20 2024-09-12T12:21:13.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.064+0000 7f3cd3fff640 1 -- 172.21.15.137:0/1097684642 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11937 (secure 0 0 0) 0x7f3ce409c680 con 0x7f3cec106f20 2024-09-12T12:21:13.065 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:21:13.065 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":34,"btime":"2024-09-12T12:20:31:715452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:20:31.715433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25780":{"gid":25780,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2747788615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2747788615},{"type":"v1","addr":"172.21.15.189:6835","nonce":2747788615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:21:13.065 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 34 2024-09-12T12:21:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3cd40a2cf0 msgr2=0x7f3cd40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3cd40a2cf0 0x7f3cd40a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009bd0 tx=0x7f3cdc005f90 comp rx=0 tx=0).stop 2024-09-12T12:21:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec106f20 msgr2=0x7f3cec19cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec106f20 0x7f3cec19cd70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400eb10 tx=0x7f3ce400c2d0 comp rx=0 tx=0).stop 2024-09-12T12:21:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 shutdown_connections 2024-09-12T12:21:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3cd40a2cf0 0x7f3cd40a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cec106f20 0x7f3cec19cd70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3cec1027a0 0x7f3cec1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 --2- 172.21.15.137:0/1097684642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cec101df0 0x7f3cec1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 >> 172.21.15.137:0/1097684642 conn(0x7f3cec0fd8f0 msgr2=0x7f3cec0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.066+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 shutdown_connections 2024-09-12T12:21:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:13.067+0000 7f3cf12a0640 1 -- 172.21.15.137:0/1097684642 wait complete. 2024-09-12T12:21:13.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:12 smithi189 ceph-mon[35475]: pgmap v248: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-12T12:21:13.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:12 smithi189 ceph-mon[35475]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T12:21:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:12 smithi171 ceph-mon[35098]: pgmap v248: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-12T12:21:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:12 smithi171 ceph-mon[35098]: osdmap e92: 12 total, 12 up, 12 in 2024-09-12T12:21:13.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636015 8.03 MB/sec execute 288 sec latency 18297.520 ms 2024-09-12T12:21:13.871 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id b21614dd --release --await' 2024-09-12T12:21:13.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id b21614dd --release --await 2024-09-12T12:21:14.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:13 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1097684642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:14.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:13 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1097684642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:14.124 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:21:14.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:13 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1097684642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:14.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 -- 172.21.15.137:0/2264141768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72301002f0 msgr2=0x7f72301006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 --2- 172.21.15.137:0/2264141768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72301002f0 0x7f72301006d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7220009930 tx=0x7f722002a420 comp rx=0 tx=0).stop 2024-09-12T12:21:14.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 -- 172.21.15.137:0/2264141768 shutdown_connections 2024-09-12T12:21:14.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 --2- 172.21.15.137:0/2264141768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7230106f50 0x7f723010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 --2- 172.21.15.137:0/2264141768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7230100ca0 0x7f7230106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 --2- 172.21.15.137:0/2264141768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72301002f0 0x7f72301006d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 -- 172.21.15.137:0/2264141768 >> 172.21.15.137:0/2264141768 conn(0x7f7230077210 msgr2=0x7f72300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 -- 172.21.15.137:0/2264141768 shutdown_connections 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.634+0000 7f7235d95640 1 -- 172.21.15.137:0/2264141768 wait complete. 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 Processor -- start 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 -- start start 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72301002f0 0x7f723019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7230100ca0 0x7f723019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7230106f50 0x7f723019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7230111fa0 con 0x7f7230100ca0 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7230111e20 con 0x7f72301002f0 2024-09-12T12:21:14.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f7235d95640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7230112120 con 0x7f7230106f50 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f722effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7230100ca0 0x7f723019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f722effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7230100ca0 0x7f723019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42420/0 (socket says 172.21.15.137:42420) 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.635+0000 7f722effd640 1 -- 172.21.15.137:0/1703413307 learned_addr learned my addr 172.21.15.137:0/1703413307 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722f7fe640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72301002f0 0x7f723019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722ffff640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7230106f50 0x7f723019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722effd640 1 -- 172.21.15.137:0/1703413307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7230106f50 msgr2=0x7f723019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722effd640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7230106f50 0x7f723019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722effd640 1 -- 172.21.15.137:0/1703413307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72301002f0 msgr2=0x7f723019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722effd640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72301002f0 0x7f723019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722effd640 1 -- 172.21.15.137:0/1703413307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723019f600 con 0x7f7230100ca0 2024-09-12T12:21:14.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722ffff640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7230106f50 0x7f723019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722effd640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7230100ca0 0x7f723019ab90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f722400c970 tx=0x7f722400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722cff9640 1 -- 172.21.15.137:0/1703413307 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7224014070 con 0x7f7230100ca0 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722cff9640 1 -- 172.21.15.137:0/1703413307 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7224004440 con 0x7f7230100ca0 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723019f8f0 con 0x7f7230100ca0 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f722cff9640 1 -- 172.21.15.137:0/1703413307 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722400f450 con 0x7f7230100ca0 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.636+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72301a71d0 con 0x7f7230100ca0 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.638+0000 7f722cff9640 1 -- 172.21.15.137:0/1703413307 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7224017070 con 0x7f7230100ca0 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 Processor -- start 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 -- start start 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b11d060 0x56335b11d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b127d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b128240 0x56335b12a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56335b082620 con 0x56335b128240 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56335b029e50 con 0x56335b11d980 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f723816e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56335b07e5e0 con 0x56335b11d060 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f722cff9640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f72180a2d70 0x7f72180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f722effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b128240 0x56335b12a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f722effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b128240 0x56335b12a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42426/0 (socket says 172.21.15.137:42426) 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f722effd640 1 -- 172.21.15.137:0/1048422244 learned_addr learned my addr 172.21.15.137:0/1048422244 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f722f7fe640 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b127d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f722ffff640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f72180a2d70 0x7f72180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.639+0000 7f722ffff640 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b11d060 0x56335b11d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722cff9640 1 -- 172.21.15.137:0/1703413307 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f72240cf220 con 0x7f7230100ca0 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722effd640 1 -- 172.21.15.137:0/1048422244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b11d060 msgr2=0x56335b11d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722effd640 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b11d060 0x56335b11d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722effd640 1 -- 172.21.15.137:0/1048422244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 msgr2=0x56335b127d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722effd640 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b127d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722effd640 1 -- 172.21.15.137:0/1048422244 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335b01ad40 con 0x56335b128240 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722f7fe640 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b127d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722ffff640 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b11d060 0x56335b11d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722ffff640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f72180a2d70 0x7f72180a5230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7228012710 tx=0x7f722800a2f0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f722effd640 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b128240 0x56335b12a630 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7224098130 tx=0x7f722406e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8fe85fd983a2ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f7215ffb640 1 -- 172.21.15.137:0/1048422244 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722409be60 con 0x56335b128240 2024-09-12T12:21:14.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f7215ffb640 1 -- 172.21.15.137:0/1048422244 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72240667f0 con 0x56335b128240 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.640+0000 7f7215ffb640 1 -- 172.21.15.137:0/1048422244 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72240a43d0 con 0x56335b128240 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 -- 172.21.15.137:0/1048422244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b128240 msgr2=0x56335b12a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b128240 0x56335b12a630 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7224098130 tx=0x7f722406e7d0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 -- 172.21.15.137:0/1048422244 shutdown_connections 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b128240 0x56335b12a630 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b127d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 --2- 172.21.15.137:0/1048422244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b11d060 0x56335b11d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 -- 172.21.15.137:0/1048422244 >> 172.21.15.137:0/1048422244 conn(0x56335b1195f0 msgr2=0x56335b1199d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 -- 172.21.15.137:0/1048422244 shutdown_connections 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 -- 172.21.15.137:0/1048422244 wait complete. 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 -- start start 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.641+0000 7f723816e740 1 Processor -- start 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f723816e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b125150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f723816e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b125690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f723816e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b125bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f723816e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56335b12ae10 con 0x56335b11d060 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f723816e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56335b00af10 con 0x56335b11d980 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f723816e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56335b12af90 con 0x56335b128240 2024-09-12T12:21:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f723816e740 10 client.0 _ll_register_callbacks cb 0x56335b02edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b125150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b125150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42442/0 (socket says 172.21.15.137:42442) 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 -- 172.21.15.137:0/1923254652 learned_addr learned my addr 172.21.15.137:0/1923254652 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722effd640 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b125690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 -- 172.21.15.137:0/1923254652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 msgr2=0x56335b125bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722ffff640 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b125bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b125bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 -- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 msgr2=0x56335b125690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b125690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 -- 172.21.15.137:0/1923254652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335ad91580 con 0x56335b11d060 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f722f7fe640 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b125150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f722002a940 tx=0x7f7220031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6477bfce51c5494 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.642+0000 7f720effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f720effd640 1 -- 172.21.15.137:0/1923254652 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220035040 con 0x56335b11d060 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f720effd640 1 -- 172.21.15.137:0/1923254652 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7220004380 con 0x56335b11d060 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72240667f0 con 0x56335b11d060 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f720effd640 1 -- 172.21.15.137:0/1923254652 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7220005020 con 0x56335b11d060 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 10 client.16398 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56335ae5b960 con 0x56335b11d060 2024-09-12T12:21:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f720effd640 1 -- 172.21.15.137:0/1923254652 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72240667f0 con 0x56335b11d060 2024-09-12T12:21:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f720effd640 1 -- 172.21.15.137:0/1923254652 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f722004d020 con 0x56335b11d060 2024-09-12T12:21:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 10 client.16398 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 10 client.16398 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:21:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 20 client.16398 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b137d30 0x56335b13a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 4 client.16398 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56335b08fb90 con 0x56335b137d30 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.643+0000 7f722effd640 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b137d30 0x56335b13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.644+0000 7f722effd640 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b137d30 0x56335b13a160 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30cf06ef8c654518 server_cookie=7a411149a2a9a30a in_seq=0 out_seq=0 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.644+0000 7f720effd640 10 client.16398 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.644+0000 7f720effd640 1 -- 172.21.15.137:0/1923254652 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56335b08fb90 con 0x56335b137d30 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.644+0000 7f720effd640 10 client.16398 handle_command_reply: tid=0 2024-09-12T12:21:14.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.644+0000 7f723816e740 1 client.16398 shutdown 2024-09-12T12:21:14.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 msgr2=0x56335b125150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b125150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f722002a940 tx=0x7f7220031a10 comp rx=0 tx=0).stop 2024-09-12T12:21:14.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 shutdown_connections 2024-09-12T12:21:14.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b125bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b125690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b137d30 0x56335b13a160 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 --2- 172.21.15.137:0/1923254652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b125150 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f720effd640 0 client.16398 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 >> 172.21.15.137:0/1923254652 conn(0x56335b1195f0 msgr2=0x56335b11f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 shutdown_connections 2024-09-12T12:21:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 1 -- 172.21.15.137:0/1923254652 wait complete. 2024-09-12T12:21:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.645+0000 7f723816e740 20 client.16398 trim_cache size 0 max 16384 2024-09-12T12:21:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 Processor -- start 2024-09-12T12:21:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 -- start start 2024-09-12T12:21:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b1270c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b13ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72301b2330 con 0x56335b11d060 2024-09-12T12:21:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f72301afbd0 con 0x56335b11d980 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f723816e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72301ada40 con 0x56335b128240 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b1270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b1270c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42448/0 (socket says 172.21.15.137:42448) 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 -- 172.21.15.137:0/878496908 learned_addr learned my addr 172.21.15.137:0/878496908 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722effd640 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 -- 172.21.15.137:0/878496908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 msgr2=0x56335b13ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722ffff640 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b13ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b13ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 -- 172.21.15.137:0/878496908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 msgr2=0x56335b13fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b13fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 -- 172.21.15.137:0/878496908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335b124100 con 0x56335b11d060 2024-09-12T12:21:14.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722effd640 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:14.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722ffff640 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b13ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:14.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.662+0000 7f722f7fe640 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b1270c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7220009850 tx=0x7f7220003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dd2db248530049 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f72157fa640 1 -- 172.21.15.137:0/878496908 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220035030 con 0x56335b11d060 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f72157fa640 1 -- 172.21.15.137:0/878496908 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7220004380 con 0x56335b11d060 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f72157fa640 1 -- 172.21.15.137:0/878496908 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220004a90 con 0x56335b11d060 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 -- 172.21.15.137:0/878496908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 msgr2=0x56335b1270c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b1270c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7220009850 tx=0x7f7220003a20 comp rx=0 tx=0).stop 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 -- 172.21.15.137:0/878496908 shutdown_connections 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b13ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d980 0x56335b13fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 --2- 172.21.15.137:0/878496908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d060 0x56335b1270c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 -- 172.21.15.137:0/878496908 >> 172.21.15.137:0/878496908 conn(0x56335b1195f0 msgr2=0x56335b1269b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 -- 172.21.15.137:0/878496908 shutdown_connections 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.663+0000 7f723816e740 1 -- 172.21.15.137:0/878496908 wait complete. 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 -- start start 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 Processor -- start 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d060 0x56335b12c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d980 0x56335b12c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b12cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f72301ae090 con 0x56335b11d980 2024-09-12T12:21:14.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f72301af500 con 0x56335b11d060 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f72301ae760 con 0x56335b128240 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f723816e740 10 client.0 _ll_register_callbacks cb 0x7f7220031620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f722effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d980 0x56335b12c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f722effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d980 0x56335b12c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42450/0 (socket says 172.21.15.137:42450) 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f722effd640 1 -- 172.21.15.137:0/3518020508 learned_addr learned my addr 172.21.15.137:0/3518020508 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f722ffff640 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b12cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.664+0000 7f722f7fe640 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d060 0x56335b12c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f722effd640 1 -- 172.21.15.137:0/3518020508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 msgr2=0x56335b12cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f722effd640 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b12cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f722effd640 1 -- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d060 msgr2=0x56335b12c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f722effd640 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d060 0x56335b12c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f722effd640 1 -- 172.21.15.137:0/3518020508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56335b1a1ca0 con 0x56335b11d980 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f722effd640 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d980 0x56335b12c730 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f72240cf1f0 tx=0x7f7224003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ec45f9be1b8068 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f720effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f720effd640 1 -- 172.21.15.137:0/3518020508 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722409be60 con 0x56335b11d980 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f720effd640 1 -- 172.21.15.137:0/3518020508 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f7224099a60 con 0x56335b11d980 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7220004380 con 0x56335b11d980 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f720effd640 1 -- 172.21.15.137:0/3518020508 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f72240a43d0 con 0x56335b11d980 2024-09-12T12:21:14.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f723816e740 10 client.16404 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56335b1a2f50 con 0x56335b11d980 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f720effd640 1 -- 172.21.15.137:0/3518020508 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7220004380 con 0x56335b11d980 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f720effd640 1 -- 172.21.15.137:0/3518020508 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f722407a1a0 con 0x56335b11d980 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f723816e740 10 client.16404 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f723816e740 10 client.16404 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.665+0000 7f723816e740 20 client.16404 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.666+0000 7f723816e740 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b1a31e0 0x56335b1a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:14.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.666+0000 7f723816e740 4 client.16404 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b21614dd", "release": true, "await": true}] 2024-09-12T12:21:14.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.666+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b21614dd", "release": true, "await": true}) -- 0x56335b08a0a0 con 0x56335b1a31e0 2024-09-12T12:21:14.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.666+0000 7f722f7fe640 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b1a31e0 0x56335b1a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:14.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.666+0000 7f722f7fe640 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b1a31e0 0x56335b1a55d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43364f72385b987e server_cookie=82d6caebbbef7c53 in_seq=0 out_seq=0 2024-09-12T12:21:14.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.666+0000 7f720effd640 10 client.16404 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:14.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.677+0000 7f720effd640 1 -- 172.21.15.137:0/3518020508 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56335b08a0a0 con 0x56335b1a31e0 2024-09-12T12:21:14.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.677+0000 7f720effd640 10 client.16404 handle_command_reply: tid=0 2024-09-12T12:21:14.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 client.16404 shutdown 2024-09-12T12:21:14.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d980 msgr2=0x56335b12c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d980 0x56335b12c730 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f72240cf1f0 tx=0x7f7224003920 comp rx=0 tx=0).stop 2024-09-12T12:21:14.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 shutdown_connections 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56335b128240 0x56335b12cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56335b1a31e0 0x56335b1a55d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56335b11d980 0x56335b12c730 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 --2- 172.21.15.137:0/3518020508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56335b11d060 0x56335b12c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f720effd640 0 client.16404 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 >> 172.21.15.137:0/3518020508 conn(0x56335b1195f0 msgr2=0x56335b11f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 shutdown_connections 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 1 -- 172.21.15.137:0/3518020508 wait complete. 2024-09-12T12:21:14.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f723816e740 20 client.16404 trim_cache size 0 max 16384 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 34, 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 32, 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "b21614dd": { 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 32, 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:21:14.681 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 139.4, 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:21:14.682 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:14.683 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:21:14.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f72180a2d70 msgr2=0x7f72180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f7235d95640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f72180a2d70 0x7f72180a5230 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7228012710 tx=0x7f722800a2f0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7230100ca0 msgr2=0x7f723019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.678+0000 7f7235d95640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7230100ca0 0x7f723019ab90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f722400c970 tx=0x7f722400ce30 comp rx=0 tx=0).stop 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 shutdown_connections 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f72180a2d70 0x7f72180a5230 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7230106f50 0x7f723019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7230100ca0 0x7f723019ab90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 --2- 172.21.15.137:0/1703413307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72301002f0 0x7f723019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 >> 172.21.15.137:0/1703413307 conn(0x7f7230077210 msgr2=0x7f72301053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:14.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 shutdown_connections 2024-09-12T12:21:14.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:14.679+0000 7f7235d95640 1 -- 172.21.15.137:0/1703413307 wait complete. 2024-09-12T12:21:14.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 636066 8.01 MB/sec execute 289 sec latency 19238.393 ms 2024-09-12T12:21:15.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:14 smithi137 ceph-mon[30950]: pgmap v250: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.1 MiB/s wr, 45 op/s 2024-09-12T12:21:15.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:14 smithi189 ceph-mon[35475]: pgmap v250: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.1 MiB/s wr, 45 op/s 2024-09-12T12:21:15.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:14 smithi171 ceph-mon[35098]: pgmap v250: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.1 MiB/s wr, 45 op/s 2024-09-12T12:21:15.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 638112 8.00 MB/sec execute 290 sec latency 17.449 ms 2024-09-12T12:21:15.893 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b21614dd', total seconds elapsed: 23.3 2024-09-12T12:21:15.893 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T12:21:15.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:21:16.173 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:21:16.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 639839 8.02 MB/sec execute 291 sec latency 26.836 ms 2024-09-12T12:21:16.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 -- 172.21.15.137:0/100016515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8102570 msgr2=0x7f71a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 --2- 172.21.15.137:0/100016515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8102570 0x7f71a81029b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7198009930 tx=0x7f719802a4d0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 -- 172.21.15.137:0/100016515 shutdown_connections 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 --2- 172.21.15.137:0/100016515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a8102ef0 0x7f71a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 --2- 172.21.15.137:0/100016515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8102570 0x7f71a81029b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 --2- 172.21.15.137:0/100016515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8100e20 0x7f71a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 -- 172.21.15.137:0/100016515 >> 172.21.15.137:0/100016515 conn(0x7f71a80fc7d0 msgr2=0x7f71a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 -- 172.21.15.137:0/100016515 shutdown_connections 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.852+0000 7f71af508640 1 -- 172.21.15.137:0/100016515 wait complete. 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 Processor -- start 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 -- start start 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8100e20 0x7f71a8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102570 0x7f71a8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a8102ef0 0x7f71a819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f71a81120a0 con 0x7f71a8102570 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8111f20 con 0x7f71a8100e20 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71af508640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71a8112220 con 0x7f71a8102ef0 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71ad2a4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8100e20 0x7f71a8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71ad2a4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8100e20 0x7f71a8198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37766/0 (socket says 172.21.15.137:37766) 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71ad2a4640 1 -- 172.21.15.137:0/2664923009 learned_addr learned my addr 172.21.15.137:0/2664923009 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71adaa5640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a8102ef0 0x7f71a819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.853+0000 7f71acaa3640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102570 0x7f71a8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71ad2a4640 1 -- 172.21.15.137:0/2664923009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a8102ef0 msgr2=0x7f71a819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71ad2a4640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a8102ef0 0x7f71a819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71ad2a4640 1 -- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102570 msgr2=0x7f71a8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71ad2a4640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102570 0x7f71a8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71ad2a4640 1 -- 172.21.15.137:0/2664923009 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a819d4d0 con 0x7f71a8100e20 2024-09-12T12:21:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71ad2a4640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8100e20 0x7f71a8198520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e9f0 tx=0x7f71a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f719e7fc640 1 -- 172.21.15.137:0/2664923009 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400cd80 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f719e7fc640 1 -- 172.21.15.137:0/2664923009 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4010910 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a819d7c0 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f719e7fc640 1 -- 172.21.15.137:0/2664923009 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4018990 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.854+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a810e6e0 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.855+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a8101200 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.857+0000 7f719e7fc640 1 -- 172.21.15.137:0/2664923009 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f71a4010ac0 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.858+0000 7f719e7fc640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71940a2d70 0x7f71940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.858+0000 7f71acaa3640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71940a2d70 0x7f71940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.858+0000 7f719e7fc640 1 -- 172.21.15.137:0/2664923009 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f71a40cfbd0 con 0x7f71a8100e20 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.858+0000 7f71acaa3640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71940a2d70 0x7f71940a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7198006010 tx=0x7f7198000bc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:16.861+0000 7f719e7fc640 1 -- 172.21.15.137:0/2664923009 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71a409c1c0 con 0x7f71a8100e20 2024-09-12T12:21:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:16 smithi137 ceph-mon[30950]: pgmap v251: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.3 MiB/s wr, 100 op/s 2024-09-12T12:21:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.088+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71a8109090 con 0x7f71a8100e20 2024-09-12T12:21:17.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.089+0000 7f719e7fc640 1 -- 172.21.15.137:0/2664923009 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11937 (secure 0 0 0) 0x7f71a40a04c0 con 0x7f71a8100e20 2024-09-12T12:21:17.090 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:21:17.090 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":34,"btime":"2024-09-12T12:20:31:715452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:20:31.715433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25780":{"gid":25780,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2747788615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2747788615},{"type":"v1","addr":"172.21.15.189:6835","nonce":2747788615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:21:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 34 2024-09-12T12:21:17.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:16 smithi189 ceph-mon[35475]: pgmap v251: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.3 MiB/s wr, 100 op/s 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71940a2d70 msgr2=0x7f71940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71940a2d70 0x7f71940a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7198006010 tx=0x7f7198000bc0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8100e20 msgr2=0x7f71a8198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8100e20 0x7f71a8198520 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71a400e9f0 tx=0x7f71a400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 shutdown_connections 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f71940a2d70 0x7f71940a5230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71a8102ef0 0x7f71a819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102570 0x7f71a8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 --2- 172.21.15.137:0/2664923009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71a8100e20 0x7f71a8198520 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:17.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 >> 172.21.15.137:0/2664923009 conn(0x7f71a80fc7d0 msgr2=0x7f71a8104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:17.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 shutdown_connections 2024-09-12T12:21:17.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:17.091+0000 7f71af508640 1 -- 172.21.15.137:0/2664923009 wait complete. 2024-09-12T12:21:17.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:16 smithi171 ceph-mon[35098]: pgmap v251: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.3 MiB/s wr, 100 op/s 2024-09-12T12:21:17.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 642432 8.03 MB/sec execute 292 sec latency 17.192 ms 2024-09-12T12:21:17.803 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T12:21:17.803 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T12:21:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:17 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2664923009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:21:18.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:17 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2664923009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:18.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:17 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2664923009' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:18.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 -- 172.21.15.137:0/798270846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 msgr2=0x7fc618100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 --2- 172.21.15.137:0/798270846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc618100f00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc608009900 tx=0x7fc60802a410 comp rx=0 tx=0).stop 2024-09-12T12:21:18.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 -- 172.21.15.137:0/798270846 shutdown_connections 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 --2- 172.21.15.137:0/798270846 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc61810e500 0x7fc618110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 --2- 172.21.15.137:0/798270846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc618101440 0x7fc61810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 --2- 172.21.15.137:0/798270846 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc618100f00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 -- 172.21.15.137:0/798270846 >> 172.21.15.137:0/798270846 conn(0x7fc6180fc7d0 msgr2=0x7fc6180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 -- 172.21.15.137:0/798270846 shutdown_connections 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.665+0000 7fc61ce50640 1 -- 172.21.15.137:0/798270846 wait complete. 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 Processor -- start 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 -- start start 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc61819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc618101440 0x7fc61819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc61810e500 0x7fc61819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc618114280 con 0x7fc618100b20 2024-09-12T12:21:18.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc618114100 con 0x7fc61810e500 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61ce50640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc618114400 con 0x7fc618101440 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc61819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc61819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42486/0 (socket says 172.21.15.137:42486) 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 -- 172.21.15.137:0/2699601313 learned_addr learned my addr 172.21.15.137:0/2699601313 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc615d9b640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc618101440 0x7fc61819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc616d9d640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc61810e500 0x7fc61819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 -- 172.21.15.137:0/2699601313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc618101440 msgr2=0x7fc61819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc618101440 0x7fc61819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 -- 172.21.15.137:0/2699601313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc61810e500 msgr2=0x7fc61819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc61810e500 0x7fc61819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc61659c640 1 -- 172.21.15.137:0/2699601313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61819e730 con 0x7fc618100b20 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc615d9b640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc618101440 0x7fc61819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.666+0000 7fc616d9d640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc61810e500 0x7fc61819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:18.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.667+0000 7fc61659c640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc61819a7e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc608006010 tx=0x7fc608004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.667+0000 7fc6077fe640 1 -- 172.21.15.137:0/2699601313 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60803f030 con 0x7fc618100b20 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.667+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61819e9c0 con 0x7fc618100b20 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.667+0000 7fc6077fe640 1 -- 172.21.15.137:0/2699601313 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc608004440 con 0x7fc618100b20 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.667+0000 7fc6077fe640 1 -- 172.21.15.137:0/2699601313 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608035070 con 0x7fc618100b20 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.667+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6181a7390 con 0x7fc618100b20 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.669+0000 7fc61f229740 1 Processor -- start 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.669+0000 7fc6077fe640 1 -- 172.21.15.137:0/2699601313 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc608042030 con 0x7fc618100b20 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61f229740 1 -- start start 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61f229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x560119019b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61f229740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56011901a080 0x560119024400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61f229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560119024940 0x560119026d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61f229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560118f7f2a0 con 0x560119024940 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61f229740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560118f26b10 con 0x56011901a080 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61f229740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560118f7b260 con 0x560119019760 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56011901a080 0x560119024400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc615d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560119024940 0x560119026d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc616d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x560119019b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56011901a080 0x560119024400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37794/0 (socket says 172.21.15.137:37794) 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc616d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x560119019b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:38900/0 (socket says 172.21.15.137:38900) 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.670+0000 7fc61659c640 1 -- 172.21.15.137:0/2144755118 learned_addr learned my addr 172.21.15.137:0/2144755118 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc61659c640 1 -- 172.21.15.137:0/2144755118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 msgr2=0x560119019b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc61659c640 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x560119019b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc61659c640 1 -- 172.21.15.137:0/2144755118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560119024940 msgr2=0x560119026d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc61659c640 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560119024940 0x560119026d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc61659c640 1 -- 172.21.15.137:0/2144755118 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560118cf0b90 con 0x56011901a080 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc615d9b640 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560119024940 0x560119026d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc6077fe640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6000a2cf0 0x7fc6000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc61659c640 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56011901a080 0x560119024400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ece70 tx=0x7fc6080f5cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69700f3c5231cfbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc616d9d640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6000a2cf0 0x7fc6000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc604ff9640 1 -- 172.21.15.137:0/2144755118 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080f10d0 con 0x56011901a080 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.671+0000 7fc604ff9640 1 -- 172.21.15.137:0/2144755118 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6080ff020 con 0x56011901a080 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc604ff9640 1 -- 172.21.15.137:0/2144755118 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080f9860 con 0x56011901a080 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc6077fe640 1 -- 172.21.15.137:0/2699601313 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc6080ef790 con 0x7fc618100b20 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 -- 172.21.15.137:0/2144755118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56011901a080 msgr2=0x560119024400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc616d9d640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6000a2cf0 0x7fc6000a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc610009fd0 tx=0x7fc610009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56011901a080 0x560119024400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ece70 tx=0x7fc6080f5cd0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 -- 172.21.15.137:0/2144755118 shutdown_connections 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560119024940 0x560119026d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56011901a080 0x560119024400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 --2- 172.21.15.137:0/2144755118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x560119019b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 -- 172.21.15.137:0/2144755118 >> 172.21.15.137:0/2144755118 conn(0x560119015cf0 msgr2=0x5601190160d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 -- 172.21.15.137:0/2144755118 shutdown_connections 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.672+0000 7fc61f229740 1 -- 172.21.15.137:0/2144755118 wait complete. 2024-09-12T12:21:18.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 -- start start 2024-09-12T12:21:18.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 Processor -- start 2024-09-12T12:21:18.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119021740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119021c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x5601190221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5601190272d0 con 0x56011901a080 2024-09-12T12:21:18.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560118f076e0 con 0x560119019760 2024-09-12T12:21:18.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560119027450 con 0x560119024940 2024-09-12T12:21:18.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61f229740 10 client.0 _ll_register_callbacks cb 0x560118f2ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:18.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119021740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119021740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:37808/0 (socket says 172.21.15.137:37808) 2024-09-12T12:21:18.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc61659c640 1 -- 172.21.15.137:0/3123785257 learned_addr learned my addr 172.21.15.137:0/3123785257 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc616d9d640 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x5601190221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.673+0000 7fc615d9b640 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119021c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61659c640 1 -- 172.21.15.137:0/3123785257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 msgr2=0x5601190221c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61659c640 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x5601190221c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61659c640 1 -- 172.21.15.137:0/3123785257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 msgr2=0x560119021c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61659c640 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119021c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61659c640 1 -- 172.21.15.137:0/3123785257 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560119030730 con 0x560119019760 2024-09-12T12:21:18.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61659c640 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119021740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc608101040 tx=0x7fc6080f9460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95144a9d7a0e8e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc5fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3123785257 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608049030 con 0x560119019760 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6080ff020 con 0x560119019760 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3123785257 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc6080ba080 con 0x560119019760 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3123785257 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc6080fe070 con 0x560119019760 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61f229740 10 client.25820 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5601190272d0 con 0x560119019760 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.674+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3123785257 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6080ff020 con 0x560119019760 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3123785257 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fc6080ed180 con 0x560119019760 2024-09-12T12:21:18.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc61f229740 10 client.25820 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc61f229740 10 client.25820 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc61f229740 20 client.25820 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc61f229740 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560119034340 0x560119036760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc61f229740 4 client.25820 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560118f8c9d0 con 0x560119034340 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc615d9b640 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560119034340 0x560119036760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc615d9b640 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560119034340 0x560119036760 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb5547f4aa96529 server_cookie=9597d3f66f5ccdbe in_seq=0 out_seq=0 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.675+0000 7fc5fdffb640 10 client.25820 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3123785257 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560118f8c9d0 con 0x560119034340 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc5fdffb640 10 client.25820 handle_command_reply: tid=0 2024-09-12T12:21:18.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 client.25820 shutdown 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 msgr2=0x560119021740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119021740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc608101040 tx=0x7fc6080f9460 comp rx=0 tx=0).stop 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 shutdown_connections 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x5601190221c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560119034340 0x560119036760 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119021c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 --2- 172.21.15.137:0/3123785257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119021740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc5fdffb640 0 client.25820 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 >> 172.21.15.137:0/3123785257 conn(0x560119015cf0 msgr2=0x56011901b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 shutdown_connections 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 1 -- 172.21.15.137:0/3123785257 wait complete. 2024-09-12T12:21:18.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.676+0000 7fc61f229740 20 client.25820 trim_cache size 0 max 16384 2024-09-12T12:21:18.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 Processor -- start 2024-09-12T12:21:18.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 -- start start 2024-09-12T12:21:18.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119023610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119022010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x56011903c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181b23a0 con 0x56011901a080 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181afc40 con 0x560119019760 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61f229740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181adab0 con 0x560119024940 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc615d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119022010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc615d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119022010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42526/0 (socket says 172.21.15.137:42526) 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc61659c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119023610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc615d9b640 1 -- 172.21.15.137:0/2302894956 learned_addr learned my addr 172.21.15.137:0/2302894956 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.695+0000 7fc616d9d640 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x56011903c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc615d9b640 1 -- 172.21.15.137:0/2302894956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 msgr2=0x56011903c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc615d9b640 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x56011903c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc615d9b640 1 -- 172.21.15.137:0/2302894956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 msgr2=0x560119023610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc615d9b640 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119023610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc615d9b640 1 -- 172.21.15.137:0/2302894956 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560119015b20 con 0x56011901a080 2024-09-12T12:21:18.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc61659c640 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119023610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:18.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc616d9d640 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x56011903c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:18.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc615d9b640 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119022010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc61810ec60 tx=0x7fc60c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ea51c51f942978 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc5fffff640 1 -- 172.21.15.137:0/2302894956 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c022070 con 0x56011901a080 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc5fffff640 1 -- 172.21.15.137:0/2302894956 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc60c01e420 con 0x56011901a080 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.696+0000 7fc5fffff640 1 -- 172.21.15.137:0/2302894956 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c01d5e0 con 0x56011901a080 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 -- 172.21.15.137:0/2302894956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 msgr2=0x560119022010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119022010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc61810ec60 tx=0x7fc60c005f90 comp rx=0 tx=0).stop 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 -- 172.21.15.137:0/2302894956 shutdown_connections 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119024940 0x56011903c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119022010 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 --2- 172.21.15.137:0/2302894956 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119019760 0x560119023610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 -- 172.21.15.137:0/2302894956 >> 172.21.15.137:0/2302894956 conn(0x560119015cf0 msgr2=0x560119014e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 -- 172.21.15.137:0/2302894956 shutdown_connections 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 -- 172.21.15.137:0/2302894956 wait complete. 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 -- start start 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.697+0000 7fc61f229740 1 Processor -- start 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x56011902ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119027690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119024940 0x560119027c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181ae100 con 0x56011901a080 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181af570 con 0x560119024940 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6181ae7d0 con 0x560119019760 2024-09-12T12:21:18.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 10 client.0 _ll_register_callbacks cb 0x7fc6080f5a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc615d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119027690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x56011902ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x56011902ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:38930/0 (socket says 172.21.15.137:38930) 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc616d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119024940 0x560119027c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 -- 172.21.15.137:0/3647471395 learned_addr learned my addr 172.21.15.137:0/3647471395 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 -- 172.21.15.137:0/3647471395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119024940 msgr2=0x560119027c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119024940 0x560119027c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 -- 172.21.15.137:0/3647471395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 msgr2=0x560119027690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119027690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 -- 172.21.15.137:0/3647471395 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560119030560 con 0x560119019760 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61659c640 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x56011902ce30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6080b8750 tx=0x7fc608005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845dfd5bdbd36244 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc5fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3647471395 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608049040 con 0x560119019760 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc60c01e420 con 0x560119019760 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.698+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3647471395 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc6080f5320 con 0x560119019760 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3647471395 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc6080b8020 con 0x560119019760 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 10 client.25885 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56011901ace0 con 0x560119019760 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3647471395 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc60c01e420 con 0x560119019760 2024-09-12T12:21:18.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3647471395 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fc6080cb070 con 0x560119019760 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 10 client.25885 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 10 client.25885 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 20 client.25885 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5601190a00f0 0x5601190a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 4 client.25885 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x560118f86d20 con 0x5601190a00f0 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.699+0000 7fc615d9b640 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5601190a00f0 0x5601190a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:18.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.700+0000 7fc615d9b640 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5601190a00f0 0x5601190a24e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0ba28e838f4c88 server_cookie=88dd2ef4bef9081b in_seq=0 out_seq=0 2024-09-12T12:21:18.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.700+0000 7fc5fdffb640 10 client.25885 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:18.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 8.03 MB/sec execute 293 sec latency 11.117 ms 2024-09-12T12:21:18.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc5fdffb640 1 -- 172.21.15.137:0/3647471395 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560118f86d20 con 0x5601190a00f0 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc5fdffb640 10 client.25885 handle_command_reply: tid=0 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 client.25885 shutdown 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 msgr2=0x56011902ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x56011902ce30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6080b8750 tx=0x7fc608005c90 comp rx=0 tx=0).stop 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 shutdown_connections 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560119024940 0x560119027c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5601190a00f0 0x5601190a24e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56011901a080 0x560119027690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc5fdffb640 0 client.25885 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 --2- 172.21.15.137:0/3647471395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560119019760 0x56011902ce30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 >> 172.21.15.137:0/3647471395 conn(0x560119015cf0 msgr2=0x560119014650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 shutdown_connections 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.764+0000 7fc61f229740 1 -- 172.21.15.137:0/3647471395 wait complete. 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61f229740 20 client.25885 trim_cache size 0 max 16384 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 34, 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:21:18.767 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 34, 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "627d6bb5": { 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 34, 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 139.4, 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:21:18.768 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:18.769 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:21:18.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6000a2cf0 msgr2=0x7fc6000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6000a2cf0 0x7fc6000a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc610009fd0 tx=0x7fc610009500 comp rx=0 tx=0).stop 2024-09-12T12:21:18.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 msgr2=0x7fc61819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:18.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc61819a7e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc608006010 tx=0x7fc608004040 comp rx=0 tx=0).stop 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 shutdown_connections 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6000a2cf0 0x7fc6000a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc61810e500 0x7fc61819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc618101440 0x7fc61819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 --2- 172.21.15.137:0/2699601313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc618100b20 0x7fc61819a7e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 >> 172.21.15.137:0/2699601313 conn(0x7fc6180fc7d0 msgr2=0x7fc6180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 shutdown_connections 2024-09-12T12:21:18.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:18.765+0000 7fc61ce50640 1 -- 172.21.15.137:0/2699601313 wait complete. 2024-09-12T12:21:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:18 smithi137 ceph-mon[30950]: pgmap v252: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-09-12T12:21:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:18 smithi189 ceph-mon[35475]: pgmap v252: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-09-12T12:21:19.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:18 smithi171 ceph-mon[35098]: pgmap v252: 97 pgs: 97 active+clean; 181 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 1.9 MiB/s wr, 82 op/s 2024-09-12T12:21:19.580 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '627d6bb5', quiesce took 3.7 seconds. Will release after: 15.7 2024-09-12T12:21:19.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 8.01 MB/sec execute 294 sec latency 1001.056 ms 2024-09-12T12:21:20.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.98 MB/sec execute 295 sec latency 2001.151 ms 2024-09-12T12:21:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:20 smithi137 ceph-mon[30950]: pgmap v253: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-09-12T12:21:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:20 smithi189 ceph-mon[35475]: pgmap v253: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-09-12T12:21:21.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:20 smithi171 ceph-mon[35098]: pgmap v253: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-09-12T12:21:21.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.95 MB/sec execute 296 sec latency 3001.256 ms 2024-09-12T12:21:22.740 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.93 MB/sec execute 297 sec latency 4001.362 ms 2024-09-12T12:21:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:22 smithi137 ceph-mon[30950]: pgmap v254: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-09-12T12:21:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:22 smithi189 ceph-mon[35475]: pgmap v254: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-09-12T12:21:23.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:22 smithi171 ceph-mon[35098]: pgmap v254: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 173 op/s 2024-09-12T12:21:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/2254048033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 msgr2=0x7f0ebc106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/2254048033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc106dc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009930 tx=0x7f0eb002a420 comp rx=0 tx=0).stop 2024-09-12T12:21:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/2254048033 shutdown_connections 2024-09-12T12:21:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/2254048033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc101370 0x7f0ebc10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/2254048033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc1009d0 0x7f0ebc100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/2254048033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc106dc0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/2254048033 >> 172.21.15.137:0/2254048033 conn(0x7f0ebc0fc6f0 msgr2=0x7f0ebc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/2254048033 shutdown_connections 2024-09-12T12:21:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.215+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/2254048033 wait complete. 2024-09-12T12:21:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 Processor -- start 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 -- start start 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc1009d0 0x7f0ebc1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc101370 0x7f0ebc197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc110e70 con 0x7f0ebc1069e0 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc110cf0 con 0x7f0ebc101370 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec2f6d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc110ff0 con 0x7f0ebc1009d0 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec150a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec150a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32836/0 (socket says 172.21.15.137:32836) 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec150a640 1 -- 172.21.15.137:0/1373469077 learned_addr learned my addr 172.21.15.137:0/1373469077 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc101370 0x7f0ebc197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.216+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc1009d0 0x7f0ebc1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec150a640 1 -- 172.21.15.137:0/1373469077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc1009d0 msgr2=0x7f0ebc1972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec150a640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc1009d0 0x7f0ebc1972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec150a640 1 -- 172.21.15.137:0/1373469077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc101370 msgr2=0x7f0ebc197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec150a640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc101370 0x7f0ebc197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec150a640 1 -- 172.21.15.137:0/1373469077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc19c2d0 con 0x7f0ebc1069e0 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec150a640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc19bba0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400efd0 tx=0x7f0eb400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0eb9ffb640 1 -- 172.21.15.137:0/1373469077 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4003b70 con 0x7f0ebc1069e0 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0eb9ffb640 1 -- 172.21.15.137:0/1373469077 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb4003d10 con 0x7f0ebc1069e0 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc19c5c0 con 0x7f0ebc1069e0 2024-09-12T12:21:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0eb9ffb640 1 -- 172.21.15.137:0/1373469077 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40109f0 con 0x7f0ebc1069e0 2024-09-12T12:21:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.217+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc1a3e50 con 0x7f0ebc1069e0 2024-09-12T12:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0eb9ffb640 1 -- 172.21.15.137:0/1373469077 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0eb4004020 con 0x7f0ebc1069e0 2024-09-12T12:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 Processor -- start 2024-09-12T12:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 -- start start 2024-09-12T12:21:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e048ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0498770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e0498cb0 0x5599e049b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0eb9ffb640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2da0 0x7f0eac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5599e03f22f0 con 0x5599e048e410 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5599e039a270 con 0x5599e0498cb0 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5599e03ee2b0 con 0x5599e048daf0 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec150a640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2da0 0x7f0eac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec0d09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0498770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e048ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec0d09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0498770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32850/0 (socket says 172.21.15.137:32850) 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ec0d09640 1 -- 172.21.15.137:0/2938579077 learned_addr learned my addr 172.21.15.137:0/2938579077 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.220+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e0498cb0 0x5599e049b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0eb9ffb640 1 -- 172.21.15.137:0/1373469077 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f0eb40cf7f0 con 0x7f0ebc1069e0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ebbfff640 1 -- 172.21.15.137:0/2938579077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e0498cb0 msgr2=0x5599e049b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e0498cb0 0x5599e049b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ebbfff640 1 -- 172.21.15.137:0/2938579077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 msgr2=0x5599e0498770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0498770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ebbfff640 1 -- 172.21.15.137:0/2938579077 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e0399340 con 0x5599e048daf0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ec150a640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2da0 0x7f0eac0a5260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40cd010 tx=0x7f0eb4086860 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e048ded0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8005d00 tx=0x7f0ea8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8d50d64d3740d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ea6ffd640 1 -- 172.21.15.137:0/2938579077 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8012d20 con 0x5599e048daf0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ea6ffd640 1 -- 172.21.15.137:0/2938579077 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8005020 con 0x5599e048daf0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.221+0000 7f0ea6ffd640 1 -- 172.21.15.137:0/2938579077 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80049a0 con 0x5599e048daf0 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 -- 172.21.15.137:0/2938579077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 msgr2=0x5599e048ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e048ded0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8005d00 tx=0x7f0ea8005c90 comp rx=0 tx=0).stop 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 -- 172.21.15.137:0/2938579077 shutdown_connections 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e0498cb0 0x5599e049b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0498770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/2938579077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e048ded0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 -- 172.21.15.137:0/2938579077 >> 172.21.15.137:0/2938579077 conn(0x5599e048a060 msgr2=0x5599e048a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 -- 172.21.15.137:0/2938579077 shutdown_connections 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 -- 172.21.15.137:0/2938579077 wait complete. 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 -- start start 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.222+0000 7f0ec5de3740 1 Processor -- start 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048daf0 0x5599e0495c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0496190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 0x5599e04966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5599e049b880 con 0x5599e048e410 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5599e037b290 con 0x5599e048daf0 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5599e049ba00 con 0x5599e0498cb0 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec5de3740 10 client.0 _ll_register_callbacks cb 0x5599e03ac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0496190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0496190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32852/0 (socket says 172.21.15.137:32852) 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048daf0 0x5599e0495c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 -- 172.21.15.137:0/1537845540 learned_addr learned my addr 172.21.15.137:0/1537845540 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 0x5599e04966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 -- 172.21.15.137:0/1537845540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 msgr2=0x5599e04966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 0x5599e04966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 -- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048daf0 msgr2=0x5599e0495c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048daf0 0x5599e0495c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 -- 172.21.15.137:0/1537845540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e0105210 con 0x5599e048e410 2024-09-12T12:21:23.230 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048daf0 0x5599e0495c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0496190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0033000 tx=0x7f0eb0004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d107d779713d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0e9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.223+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1537845540 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0035040 con 0x5599e048e410 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1537845540 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f0eb0005450 con 0x5599e048e410 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea8005020 con 0x5599e048e410 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1537845540 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0005650 con 0x5599e048e410 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 10 client.16440 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599e01d1620 con 0x5599e048e410 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1537845540 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8005020 con 0x5599e048e410 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1537845540 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f0eb0003a60 con 0x5599e048e410 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 10 client.16440 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 20 client.16440 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e04a8ce0 0x5599e04ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 4 client.16440 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599e03ffa20 con 0x5599e04a8ce0 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.224+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e04a8ce0 0x5599e04ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0e9bfff640 10 client.16440 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e04a8ce0 0x5599e04ab0d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48dd904070de057c server_cookie=19bdb87c863f658b in_seq=0 out_seq=0 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1537845540 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599e03ffa20 con 0x5599e04a8ce0 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0e9bfff640 10 client.16440 handle_command_reply: tid=0 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 client.16440 shutdown 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 msgr2=0x5599e0496190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0496190 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0033000 tx=0x7f0eb0004420 comp rx=0 tx=0).stop 2024-09-12T12:21:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 shutdown_connections 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 0x5599e04966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e04a8ce0 0x5599e04ab0d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048e410 0x5599e0496190 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0e9bfff640 0 client.16440 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1537845540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048daf0 0x5599e0495c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.225+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 >> 172.21.15.137:0/1537845540 conn(0x5599e048a060 msgr2=0x5599e048fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.226+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 shutdown_connections 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.226+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1537845540 wait complete. 2024-09-12T12:21:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.226+0000 7f0ec5de3740 20 client.16440 trim_cache size 0 max 16384 2024-09-12T12:21:23.244 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 Processor -- start 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 -- start start 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e0490560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0496500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e0498cb0 0x5599e0497bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1aeeb0 con 0x5599e0498cb0 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1ac750 con 0x5599e048e410 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1aa5c0 con 0x5599e048daf0 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e0498cb0 0x5599e0497bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e0498cb0 0x5599e0497bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32860/0 (socket says 172.21.15.137:32860) 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 -- 172.21.15.137:0/3159117867 learned_addr learned my addr 172.21.15.137:0/3159117867 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e0490560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0496500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 -- 172.21.15.137:0/3159117867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 msgr2=0x5599e0490560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e0490560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 -- 172.21.15.137:0/3159117867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 msgr2=0x5599e0496500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0496500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 -- 172.21.15.137:0/3159117867 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599e04ab440 con 0x5599e0498cb0 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0496500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.240+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e0498cb0 0x5599e0497bc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b6b0 tx=0x7f0ea8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f717c35495d68384 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ea67fc640 1 -- 172.21.15.137:0/3159117867 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8010740 con 0x5599e0498cb0 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ea67fc640 1 -- 172.21.15.137:0/3159117867 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8033030 con 0x5599e0498cb0 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ea67fc640 1 -- 172.21.15.137:0/3159117867 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8003b30 con 0x5599e0498cb0 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 -- 172.21.15.137:0/3159117867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e0498cb0 msgr2=0x5599e0497bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e0498cb0 0x5599e0497bc0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b6b0 tx=0x7f0ea8014040 comp rx=0 tx=0).stop 2024-09-12T12:21:23.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 -- 172.21.15.137:0/3159117867 shutdown_connections 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e0498cb0 0x5599e0497bc0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0496500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/3159117867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e048daf0 0x5599e0490560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 -- 172.21.15.137:0/3159117867 >> 172.21.15.137:0/3159117867 conn(0x5599e048a060 msgr2=0x5599e04974b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 -- 172.21.15.137:0/3159117867 shutdown_connections 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.241+0000 7f0ec5de3740 1 -- 172.21.15.137:0/3159117867 wait complete. 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 -- start start 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 Processor -- start 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048daf0 0x5599e0493490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0491ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 0x5599e0492530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1aac10 con 0x5599e048daf0 2024-09-12T12:21:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1ac080 con 0x5599e048e410 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1ab2e0 con 0x5599e0498cb0 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec5de3740 10 client.0 _ll_register_callbacks cb 0x7f0eb0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048daf0 0x5599e0493490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048daf0 0x5599e0493490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32874/0 (socket says 172.21.15.137:32874) 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 -- 172.21.15.137:0/1030373737 learned_addr learned my addr 172.21.15.137:0/1030373737 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 -- 172.21.15.137:0/1030373737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 msgr2=0x5599e0492530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 0x5599e0492530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 -- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 msgr2=0x5599e0491ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0491ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0491ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.242+0000 7f0ebbfff640 1 -- 172.21.15.137:0/1030373737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599e0492a70 con 0x5599e048daf0 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ebbfff640 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048daf0 0x5599e0493490 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8021040 tx=0x7f0ea8012480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2921b07ea10e6d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:23.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0e9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1030373737 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8005850 con 0x5599e048daf0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1030373737 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f0ea80037d0 con 0x5599e048daf0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea8033030 con 0x5599e048daf0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1030373737 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0ea80039d0 con 0x5599e048daf0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 10 client.16446 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599e0515620 con 0x5599e048daf0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1030373737 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea8033030 con 0x5599e048daf0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1030373737 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f0ea8037070 con 0x5599e048daf0 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 10 client.16446 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 20 client.16446 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e0515a90 0x5599e0517e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 4 client.16446 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:21:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.243+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599e03f9d70 con 0x5599e0515a90 2024-09-12T12:21:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.244+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e0515a90 0x5599e0517e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.244+0000 7f0e9bfff640 10 client.16446 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:23.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.244+0000 7f0ec0d09640 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e0515a90 0x5599e0517e80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbddeb577daa6b1 server_cookie=1e9398525d9ca05d in_seq=0 out_seq=0 2024-09-12T12:21:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1030373737 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599e03f9d70 con 0x5599e0515a90 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0e9bfff640 10 client.16446 handle_command_reply: tid=0 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 client.16446 shutdown 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048daf0 msgr2=0x5599e0493490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048daf0 0x5599e0493490 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8021040 tx=0x7f0ea8012480 comp rx=0 tx=0).stop 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 shutdown_connections 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5599e0498cb0 0x5599e0492530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5599e048e410 0x5599e0491ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5599e0515a90 0x5599e0517e80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 --2- 172.21.15.137:0/1030373737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5599e048daf0 0x5599e0493490 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 >> 172.21.15.137:0/1030373737 conn(0x5599e048a060 msgr2=0x5599e04891c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:23.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0e9bfff640 0 client.16446 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 shutdown_connections 2024-09-12T12:21:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.286+0000 7f0ec5de3740 1 -- 172.21.15.137:0/1030373737 wait complete. 2024-09-12T12:21:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec5de3740 20 client.16446 trim_cache size 0 max 16384 2024-09-12T12:21:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:21:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:21:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:21:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2da0 msgr2=0x7f0eac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2da0 0x7f0eac0a5260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40cd010 tx=0x7f0eb4086860 comp rx=0 tx=0).stop 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 msgr2=0x7f0ebc19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc19bba0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400efd0 tx=0x7f0eb400c6a0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 shutdown_connections 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2da0 0x7f0eac0a5260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ebc1069e0 0x7f0ebc19bba0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc101370 0x7f0ebc197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 --2- 172.21.15.137:0/1373469077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ebc1009d0 0x7f0ebc1972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 >> 172.21.15.137:0/1373469077 conn(0x7f0ebc0fc6f0 msgr2=0x7f0ebc0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 shutdown_connections 2024-09-12T12:21:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:23.287+0000 7f0ec2f6d640 1 -- 172.21.15.137:0/1373469077 wait complete. 2024-09-12T12:21:23.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.90 MB/sec execute 298 sec latency 5001.427 ms 2024-09-12T12:21:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:21:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:21:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:21:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:21:24.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:21:24.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:21:24.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:21:24.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:21:24.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:21:24.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:21:24.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:21:24.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:21:24.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.87 MB/sec execute 299 sec latency 6001.536 ms 2024-09-12T12:21:25.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:25 smithi137 ceph-mon[30950]: pgmap v255: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-09-12T12:21:25.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:25 smithi189 ceph-mon[35475]: pgmap v255: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-09-12T12:21:25.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:25 smithi171 ceph-mon[35098]: pgmap v255: 97 pgs: 97 active+clean; 188 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-09-12T12:21:25.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.85 MB/sec execute 300 sec latency 7001.641 ms 2024-09-12T12:21:26.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.82 MB/sec execute 301 sec latency 8001.731 ms 2024-09-12T12:21:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:27 smithi137 ceph-mon[30950]: pgmap v256: 97 pgs: 97 active+clean; 183 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 195 op/s 2024-09-12T12:21:27.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:27 smithi189 ceph-mon[35475]: pgmap v256: 97 pgs: 97 active+clean; 183 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 195 op/s 2024-09-12T12:21:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:27 smithi171 ceph-mon[35098]: pgmap v256: 97 pgs: 97 active+clean; 183 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 195 op/s 2024-09-12T12:21:27.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.80 MB/sec execute 302 sec latency 9001.845 ms 2024-09-12T12:21:28.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.77 MB/sec execute 303 sec latency 10001.969 ms 2024-09-12T12:21:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:29 smithi137 ceph-mon[30950]: pgmap v257: 97 pgs: 97 active+clean; 183 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-12T12:21:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:29 smithi189 ceph-mon[35475]: pgmap v257: 97 pgs: 97 active+clean; 183 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-12T12:21:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:29 smithi171 ceph-mon[35098]: pgmap v257: 97 pgs: 97 active+clean; 183 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-12T12:21:29.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.74 MB/sec execute 304 sec latency 11002.091 ms 2024-09-12T12:21:30.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.72 MB/sec execute 305 sec latency 12002.212 ms 2024-09-12T12:21:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:31 smithi137 ceph-mon[30950]: pgmap v258: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-09-12T12:21:31.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:31 smithi189 ceph-mon[35475]: pgmap v258: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-09-12T12:21:31.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:31 smithi171 ceph-mon[35098]: pgmap v258: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-09-12T12:21:31.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.69 MB/sec execute 306 sec latency 13002.310 ms 2024-09-12T12:21:32.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.67 MB/sec execute 307 sec latency 14002.416 ms 2024-09-12T12:21:33.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:33 smithi137 ceph-mon[30950]: pgmap v259: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-12T12:21:33.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:33 smithi189 ceph-mon[35475]: pgmap v259: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-12T12:21:33.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:33 smithi171 ceph-mon[35098]: pgmap v259: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-12T12:21:33.741 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.64 MB/sec execute 308 sec latency 15002.516 ms 2024-09-12T12:21:34.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.62 MB/sec execute 309 sec latency 16002.579 ms 2024-09-12T12:21:35.282 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:21:35.546 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:21:35.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:35 smithi137 ceph-mon[30950]: pgmap v260: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-12T12:21:35.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:35 smithi189 ceph-mon[35475]: pgmap v260: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-12T12:21:35.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:35 smithi171 ceph-mon[35098]: pgmap v260: 97 pgs: 97 active+clean; 185 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.5 MiB/s wr, 80 op/s 2024-09-12T12:21:35.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.59 MB/sec execute 310 sec latency 17002.675 ms 2024-09-12T12:21:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 -- 172.21.15.137:0/229782903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344106a80 msgr2=0x7fc344106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/229782903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344106a80 0x7fc344106e60 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc340009930 tx=0x7fc34002a420 comp rx=0 tx=0).stop 2024-09-12T12:21:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 -- 172.21.15.137:0/229782903 shutdown_connections 2024-09-12T12:21:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/229782903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344101420 0x7fc34410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/229782903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc344100a80 0x7fc344100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/229782903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344106a80 0x7fc344106e60 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 -- 172.21.15.137:0/229782903 >> 172.21.15.137:0/229782903 conn(0x7fc3440fc7a0 msgr2=0x7fc3440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 -- 172.21.15.137:0/229782903 shutdown_connections 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.027+0000 7fc34b2eb640 1 -- 172.21.15.137:0/229782903 wait complete. 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 Processor -- start 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 -- start start 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344100a80 0x7fc344197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc344101420 0x7fc3441978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344106a80 0x7fc34419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc344110e80 con 0x7fc344100a80 2024-09-12T12:21:36.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc344110d00 con 0x7fc344106a80 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.028+0000 7fc34b2eb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc344111000 con 0x7fc344101420 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344100a80 0x7fc344197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344100a80 0x7fc344197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52706/0 (socket says 172.21.15.137:52706) 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 -- 172.21.15.137:0/1981253682 learned_addr learned my addr 172.21.15.137:0/1981253682 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc348886640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc344101420 0x7fc3441978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349888640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344106a80 0x7fc34419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 -- 172.21.15.137:0/1981253682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc344101420 msgr2=0x7fc3441978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc344101420 0x7fc3441978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 -- 172.21.15.137:0/1981253682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344106a80 msgr2=0x7fc34419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344106a80 0x7fc34419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 -- 172.21.15.137:0/1981253682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34419c310 con 0x7fc344100a80 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349888640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344106a80 0x7fc34419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:36.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc349087640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344100a80 0x7fc344197360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc340009a50 tx=0x7fc340004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc33a7fc640 1 -- 172.21.15.137:0/1981253682 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34003f030 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc33a7fc640 1 -- 172.21.15.137:0/1981253682 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc340003ad0 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34419c5a0 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc33a7fc640 1 -- 172.21.15.137:0/1981253682 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340035070 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.029+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3441a3ee0 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.030+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc344100ee0 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.031+0000 7fc33a7fc640 1 -- 172.21.15.137:0/1981253682 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc340042030 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.032+0000 7fc33a7fc640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc3300a2d70 0x7fc3300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.032+0000 7fc348886640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc3300a2d70 0x7fc3300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.033+0000 7fc33a7fc640 1 -- 172.21.15.137:0/1981253682 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fc3400efbb0 con 0x7fc344100a80 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.033+0000 7fc348886640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc3300a2d70 0x7fc3300a5230 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc344105f90 tx=0x7fc33400a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.037+0000 7fc33a7fc640 1 -- 172.21.15.137:0/1981253682 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3400bc790 con 0x7fc344100a80 2024-09-12T12:21:36.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.237+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc344108ff0 con 0x7fc344100a80 2024-09-12T12:21:36.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.237+0000 7fc33a7fc640 1 -- 172.21.15.137:0/1981253682 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11937 (secure 0 0 0) 0x7fc3400bc090 con 0x7fc344100a80 2024-09-12T12:21:36.238 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:21:36.239 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":34,"btime":"2024-09-12T12:20:31:715452+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:20:31.715433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25780":{"gid":25780,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2747788615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2747788615},{"type":"v1","addr":"172.21.15.189:6835","nonce":2747788615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:21:36.239 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 34 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc3300a2d70 msgr2=0x7fc3300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc3300a2d70 0x7fc3300a5230 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc344105f90 tx=0x7fc33400a2d0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344100a80 msgr2=0x7fc344197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344100a80 0x7fc344197360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc340009a50 tx=0x7fc340004270 comp rx=0 tx=0).stop 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 shutdown_connections 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc3300a2d70 0x7fc3300a5230 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc344106a80 0x7fc34419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc344101420 0x7fc3441978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 --2- 172.21.15.137:0/1981253682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc344100a80 0x7fc344197360 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 >> 172.21.15.137:0/1981253682 conn(0x7fc3440fc7a0 msgr2=0x7fc3440fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 shutdown_connections 2024-09-12T12:21:36.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:36.240+0000 7fc34b2eb640 1 -- 172.21.15.137:0/1981253682 wait complete. 2024-09-12T12:21:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 -- 172.21.15.137:0/687297821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1069c0 msgr2=0x7f01bc106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 --2- 172.21.15.137:0/687297821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1069c0 0x7f01bc106da0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009930 tx=0x7f01b802a420 comp rx=0 tx=0).stop 2024-09-12T12:21:36.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 -- 172.21.15.137:0/687297821 shutdown_connections 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 --2- 172.21.15.137:0/687297821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01bc101350 0x7f01bc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 --2- 172.21.15.137:0/687297821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01bc1009b0 0x7f01bc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 --2- 172.21.15.137:0/687297821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1069c0 0x7f01bc106da0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 -- 172.21.15.137:0/687297821 >> 172.21.15.137:0/687297821 conn(0x7f01bc077990 msgr2=0x7f01bc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 -- 172.21.15.137:0/687297821 shutdown_connections 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.378+0000 7f01c45fd640 1 -- 172.21.15.137:0/687297821 wait complete. 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 Processor -- start 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 -- start start 2024-09-12T12:21:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1009b0 0x7f01bc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01bc101350 0x7f01bc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01bc1069c0 0x7f01bc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc110e50 con 0x7f01bc1009b0 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc110cd0 con 0x7f01bc101350 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c45fd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc110fd0 con 0x7f01bc1069c0 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1009b0 0x7f01bc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1009b0 0x7f01bc198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52720/0 (socket says 172.21.15.137:52720) 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 -- 172.21.15.137:0/2033704259 learned_addr learned my addr 172.21.15.137:0/2033704259 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01bc1069c0 0x7f01bc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c1b98640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01bc101350 0x7f01bc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 -- 172.21.15.137:0/2033704259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01bc1069c0 msgr2=0x7f01bc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01bc1069c0 0x7f01bc19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 -- 172.21.15.137:0/2033704259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01bc101350 msgr2=0x7f01bc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01bc101350 0x7f01bc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 -- 172.21.15.137:0/2033704259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01bc19d410 con 0x7f01bc1009b0 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.379+0000 7f01c2399640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1009b0 0x7f01bc198420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f01b802a940 tx=0x7f01b8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.380+0000 7f01b37fe640 1 -- 172.21.15.137:0/2033704259 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b803f070 con 0x7f01bc1009b0 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.380+0000 7f01b37fe640 1 -- 172.21.15.137:0/2033704259 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b8031c90 con 0x7f01bc1009b0 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.380+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01bc19d6a0 con 0x7f01bc1009b0 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.380+0000 7f01b37fe640 1 -- 172.21.15.137:0/2033704259 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8035030 con 0x7f01bc1009b0 2024-09-12T12:21:36.384 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.380+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01bc1a4f90 con 0x7f01bc1009b0 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.382+0000 7f01b37fe640 1 -- 172.21.15.137:0/2033704259 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f01b8042070 con 0x7f01bc1009b0 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.383+0000 7f01b37fe640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f01a80a2e20 0x7f01a80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.383+0000 7f01c1b98640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f01a80a2e20 0x7f01a80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.383+0000 7f01b37fe640 1 -- 172.21.15.137:0/2033704259 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f01b80efaf0 con 0x7f01bc1009b0 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.383+0000 7f01c1b98640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f01a80a2e20 0x7f01a80a52e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f01bc100000 tx=0x7f01ac005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.383+0000 7f01c7480740 1 Processor -- start 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c7480740 1 -- start start 2024-09-12T12:21:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c7480740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465625eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c7480740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x564656269770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c7480740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c7480740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5646561c32f0 con 0x56465625f410 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c7480740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56465616b270 con 0x564656269cb0 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c7480740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5646561bf2b0 con 0x56465625eaf0 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x564656269770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x564656269770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52722/0 (socket says 172.21.15.137:52722) 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 -- 172.21.15.137:0/146100317 learned_addr learned my addr 172.21.15.137:0/146100317 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c2399640 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465625eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 -- 172.21.15.137:0/146100317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 msgr2=0x56465625eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465625eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 -- 172.21.15.137:0/146100317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 msgr2=0x56465626c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c1b98640 1 -- 172.21.15.137:0/146100317 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465616a340 con 0x56465625f410 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.384+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.385+0000 7f01c1b98640 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x564656269770 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0058c0 tx=0x7f01ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e359dd18b4329e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.385+0000 7f01b0bfe640 1 -- 172.21.15.137:0/146100317 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac005020 con 0x56465625f410 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.385+0000 7f01b0bfe640 1 -- 172.21.15.137:0/146100317 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ac0047a0 con 0x56465625f410 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.385+0000 7f01b0bfe640 1 -- 172.21.15.137:0/146100317 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac01e840 con 0x56465625f410 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- 172.21.15.137:0/146100317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 msgr2=0x564656269770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x564656269770 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0058c0 tx=0x7f01ac014000 comp rx=0 tx=0).stop 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- 172.21.15.137:0/146100317 shutdown_connections 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x564656269770 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 --2- 172.21.15.137:0/146100317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465625eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- 172.21.15.137:0/146100317 >> 172.21.15.137:0/146100317 conn(0x56465625b060 msgr2=0x56465625b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- 172.21.15.137:0/146100317 shutdown_connections 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- 172.21.15.137:0/146100317 wait complete. 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- start start 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 Processor -- start 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625eaf0 0x564656266bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x564656267100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564656269cb0 0x564656267640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56465626c880 con 0x56465625eaf0 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56465614c290 con 0x56465625f410 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.386+0000 7f01c7480740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56465626ca00 con 0x564656269cb0 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c7480740 10 client.0 _ll_register_callbacks cb 0x56465617d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625eaf0 0x564656266bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625eaf0 0x564656266bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52736/0 (socket says 172.21.15.137:52736) 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 -- 172.21.15.137:0/642384504 learned_addr learned my addr 172.21.15.137:0/642384504 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564656269cb0 0x564656267640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c1b98640 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x564656267100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 -- 172.21.15.137:0/642384504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564656269cb0 msgr2=0x564656267640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564656269cb0 0x564656267640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 -- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 msgr2=0x564656267100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x564656267100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 -- 172.21.15.137:0/642384504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564655ed6210 con 0x56465625eaf0 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c2399640 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625eaf0 0x564656266bc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009900 tx=0x7f01b802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e680d4a691ab8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:36.397 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01a5ffb640 1 -- 172.21.15.137:0/642384504 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8049070 con 0x56465625eaf0 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01a5ffb640 1 -- 172.21.15.137:0/642384504 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f01b80ed480 con 0x56465625eaf0 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ac0047a0 con 0x56465625eaf0 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01a5ffb640 1 -- 172.21.15.137:0/642384504 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f01b809f050 con 0x56465625eaf0 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c7480740 10 client.16464 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.387+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564655fa2620 con 0x56465625eaf0 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01a5ffb640 1 -- 172.21.15.137:0/642384504 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ac0047a0 con 0x56465625eaf0 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01a5ffb640 1 -- 172.21.15.137:0/642384504 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f01b803bce0 con 0x56465625eaf0 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01c7480740 10 client.16464 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01c7480740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01c7480740 20 client.16464 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01c7480740 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564656279c00 0x56465627bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01c7480740 4 client.16464 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646561d0a20 con 0x564656279c00 2024-09-12T12:21:36.398 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.388+0000 7f01c1b98640 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564656279c00 0x56465627bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01a5ffb640 10 client.16464 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c1b98640 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564656279c00 0x56465627bff0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c897e43335b16dfd server_cookie=2fe374e63c9230e7 in_seq=0 out_seq=0 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01a5ffb640 1 -- 172.21.15.137:0/642384504 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646561d0a20 con 0x564656279c00 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01a5ffb640 10 client.16464 handle_command_reply: tid=0 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 client.16464 shutdown 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625eaf0 msgr2=0x564656266bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625eaf0 0x564656266bc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009900 tx=0x7f01b802a420 comp rx=0 tx=0).stop 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 shutdown_connections 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564656269cb0 0x564656267640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x564656267100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564656279c00 0x56465627bff0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.390+0000 7f01a5ffb640 0 client.16464 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.389+0000 7f01c7480740 1 --2- 172.21.15.137:0/642384504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625eaf0 0x564656266bc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.390+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 >> 172.21.15.137:0/642384504 conn(0x56465625b060 msgr2=0x564656260d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.390+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 shutdown_connections 2024-09-12T12:21:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.390+0000 7f01c7480740 1 -- 172.21.15.137:0/642384504 wait complete. 2024-09-12T12:21:36.400 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.390+0000 7f01c7480740 20 client.16464 trim_cache size 0 max 16384 2024-09-12T12:21:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 Processor -- start 2024-09-12T12:21:36.410 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 -- start start 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x564656267470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x5646562818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564656269cb0 0x564656281de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1b00d0 con 0x564656269cb0 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1ad970 con 0x56465625f410 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c7480740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1ab7e0 con 0x56465625eaf0 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c2b9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564656269cb0 0x564656281de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c2b9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564656269cb0 0x564656281de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52744/0 (socket says 172.21.15.137:52744) 2024-09-12T12:21:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c2b9a640 1 -- 172.21.15.137:0/1206807195 learned_addr learned my addr 172.21.15.137:0/1206807195 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c2399640 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x564656267470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c1b98640 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x5646562818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c2b9a640 1 -- 172.21.15.137:0/1206807195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 msgr2=0x564656267470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.404+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x564656267470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01c2b9a640 1 -- 172.21.15.137:0/1206807195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 msgr2=0x5646562818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x5646562818a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01c2b9a640 1 -- 172.21.15.137:0/1206807195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56465626c3e0 con 0x564656269cb0 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01c1b98640 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x5646562818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564656269cb0 0x564656281de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009f70 tx=0x7f01b40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b909f96399f1b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01a7fff640 1 -- 172.21.15.137:0/1206807195 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4003a30 con 0x564656269cb0 2024-09-12T12:21:36.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01a7fff640 1 -- 172.21.15.137:0/1206807195 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b4003bd0 con 0x564656269cb0 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.405+0000 7f01a7fff640 1 -- 172.21.15.137:0/1206807195 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b401c8e0 con 0x564656269cb0 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- 172.21.15.137:0/1206807195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564656269cb0 msgr2=0x564656281de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564656269cb0 0x564656281de0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009f70 tx=0x7f01b40128e0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- 172.21.15.137:0/1206807195 shutdown_connections 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564656269cb0 0x564656281de0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56465625f410 0x5646562818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 --2- 172.21.15.137:0/1206807195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x564656267470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- 172.21.15.137:0/1206807195 >> 172.21.15.137:0/1206807195 conn(0x56465625b060 msgr2=0x564656268c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- 172.21.15.137:0/1206807195 shutdown_connections 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- 172.21.15.137:0/1206807195 wait complete. 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- start start 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 Processor -- start 2024-09-12T12:21:36.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465626da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x56465626df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1abe30 con 0x56465625f410 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1ad2a0 con 0x564656269cb0 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01bc1ac500 con 0x56465625eaf0 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c7480740 10 client.0 _ll_register_callbacks cb 0x7f01b8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c1b98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x56465626df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c1b98640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x56465626df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52758/0 (socket says 172.21.15.137:52758) 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.406+0000 7f01c1b98640 1 -- 172.21.15.137:0/1374192689 learned_addr learned my addr 172.21.15.137:0/1374192689 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c2b9a640 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c2399640 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465626da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c1b98640 1 -- 172.21.15.137:0/1374192689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 msgr2=0x56465626da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c1b98640 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465626da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c1b98640 1 -- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 msgr2=0x56465626e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c1b98640 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c1b98640 1 -- 172.21.15.137:0/1374192689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646562e3870 con 0x56465625f410 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c1b98640 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x56465626df50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0058c0 tx=0x7f01ac0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43df39fcd9718eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01a5ffb640 1 -- 172.21.15.137:0/1374192689 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac01e650 con 0x56465625f410 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b4003bd0 con 0x56465625f410 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01a5ffb640 1 -- 172.21.15.137:0/1374192689 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7f01ac01e7f0 con 0x56465625f410 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01a5ffb640 1 -- 172.21.15.137:0/1374192689 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f01ac044050 con 0x56465625f410 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c7480740 10 client.16470 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.407+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5646562e4ad0 con 0x56465625f410 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01a5ffb640 1 -- 172.21.15.137:0/1374192689 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b4003bd0 con 0x56465625f410 2024-09-12T12:21:36.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01a5ffb640 1 -- 172.21.15.137:0/1374192689 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f01ac033bb0 con 0x56465625f410 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c7480740 10 client.16470 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c7480740 10 client.16470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c7480740 20 client.16470 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c7480740 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5646562e4fd0 0x5646562e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c7480740 4 client.16470 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646561cad70 con 0x5646562e4fd0 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c2399640 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5646562e4fd0 0x5646562e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01a5ffb640 10 client.16470 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.408+0000 7f01c2399640 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5646562e4fd0 0x5646562e73f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca685802e73659fa server_cookie=7fa99b54b18530c4 in_seq=0 out_seq=0 2024-09-12T12:21:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01a5ffb640 1 -- 172.21.15.137:0/1374192689 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646561cad70 con 0x5646562e4fd0 2024-09-12T12:21:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01a5ffb640 10 client.16470 handle_command_reply: tid=0 2024-09-12T12:21:36.438 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 client.16470 shutdown 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 msgr2=0x56465626df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x56465626df50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f01ac0058c0 tx=0x7f01ac0099c0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 shutdown_connections 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564656269cb0 0x56465626e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5646562e4fd0 0x5646562e73f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56465625f410 0x56465626df50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01a5ffb640 0 client.16470 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 --2- 172.21.15.137:0/1374192689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56465625eaf0 0x56465626da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 >> 172.21.15.137:0/1374192689 conn(0x56465625b060 msgr2=0x564656260550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 shutdown_connections 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 1 -- 172.21.15.137:0/1374192689 wait complete. 2024-09-12T12:21:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.437+0000 7f01c7480740 20 client.16470 trim_cache size 0 max 16384 2024-09-12T12:21:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:21:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:21:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:21:36.440 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f01a80a2e20 msgr2=0x7f01a80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f01a80a2e20 0x7f01a80a52e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f01bc100000 tx=0x7f01ac005f90 comp rx=0 tx=0).stop 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1009b0 msgr2=0x7f01bc198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1009b0 0x7f01bc198420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f01b802a940 tx=0x7f01b8003b90 comp rx=0 tx=0).stop 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 shutdown_connections 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f01a80a2e20 0x7f01a80a52e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01bc1069c0 0x7f01bc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01bc101350 0x7f01bc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 --2- 172.21.15.137:0/2033704259 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01bc1009b0 0x7f01bc198420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 >> 172.21.15.137:0/2033704259 conn(0x7f01bc077990 msgr2=0x7f01bc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:36.441 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 shutdown_connections 2024-09-12T12:21:36.442 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:36.438+0000 7f01c45fd640 1 -- 172.21.15.137:0/2033704259 wait complete. 2024-09-12T12:21:36.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:36 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1981253682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:36.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:36 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1981253682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:36.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:36 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1981253682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:21:36.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.57 MB/sec execute 311 sec latency 18002.717 ms 2024-09-12T12:21:36.975 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 627d6bb5 --release --await' 2024-09-12T12:21:36.975 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 627d6bb5 --release --await 2024-09-12T12:21:37.235 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:21:37.553 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:37 smithi137 ceph-mon[30950]: pgmap v261: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-12T12:21:37.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:37 smithi189 ceph-mon[35475]: pgmap v261: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-12T12:21:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:37 smithi171 ceph-mon[35098]: pgmap v261: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.6 MiB/s wr, 86 op/s 2024-09-12T12:21:37.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 643942 7.55 MB/sec execute 312 sec latency 19002.769 ms 2024-09-12T12:21:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.771+0000 7fd1e72f5640 1 -- 172.21.15.137:0/4073494222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0106f10 msgr2=0x7fd1e010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.771+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/4073494222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0106f10 0x7fd1e010ea90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009ca0 tx=0x7fd1d802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.771+0000 7fd1e72f5640 1 -- 172.21.15.137:0/4073494222 shutdown_connections 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.771+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/4073494222 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0106f10 0x7fd1e010ea90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.771+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/4073494222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1e0102790 0x7fd1e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/4073494222 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e0101de0 0x7fd1e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 -- 172.21.15.137:0/4073494222 >> 172.21.15.137:0/4073494222 conn(0x7fd1e00fd8c0 msgr2=0x7fd1e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 -- 172.21.15.137:0/4073494222 shutdown_connections 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 -- 172.21.15.137:0/4073494222 wait complete. 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 Processor -- start 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 -- start start 2024-09-12T12:21:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e0101de0 0x7fd1e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0102790 0x7fd1e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1e0106f10 0x7fd1e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0111f70 con 0x7fd1e0102790 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e0111df0 con 0x7fd1e0101de0 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.772+0000 7fd1e72f5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01120f0 con 0x7fd1e0106f10 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0102790 0x7fd1e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0102790 0x7fd1e0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52788/0 (socket says 172.21.15.137:52788) 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 -- 172.21.15.137:0/1588786528 learned_addr learned my addr 172.21.15.137:0/1588786528 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e5892640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1e0106f10 0x7fd1e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e5091640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e0101de0 0x7fd1e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 -- 172.21.15.137:0/1588786528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1e0106f10 msgr2=0x7fd1e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1e0106f10 0x7fd1e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 -- 172.21.15.137:0/1588786528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e0101de0 msgr2=0x7fd1e00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e0101de0 0x7fd1e00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 -- 172.21.15.137:0/1588786528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e01a6890 con 0x7fd1e0102790 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e5892640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1e0106f10 0x7fd1e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e5091640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e0101de0 0x7fd1e00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e4890640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0102790 0x7fd1e0074a30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000e9f0 tx=0x7fd1d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1d67fc640 1 -- 172.21.15.137:0/1588786528 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000cd80 con 0x7fd1e0102790 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e01a6b80 con 0x7fd1e0102790 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1d67fc640 1 -- 172.21.15.137:0/1588786528 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d0005500 con 0x7fd1e0102790 2024-09-12T12:21:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1d67fc640 1 -- 172.21.15.137:0/1588786528 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0010690 con 0x7fd1e0102790 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.773+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e01a70c0 con 0x7fd1e0102790 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.775+0000 7fd1d67fc640 1 -- 172.21.15.137:0/1588786528 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd1d00046c0 con 0x7fd1e0102790 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 Processor -- start 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 -- start start 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1d67fc640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1cc0a2cf0 0x7fd1cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347060 0x55ee52347440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52351d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52352240 0x55ee52354630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ee522ac620 con 0x55ee52347980 2024-09-12T12:21:37.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ee52253e50 con 0x55ee52347060 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee522a85e0 con 0x55ee52352240 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1d67fc640 1 -- 172.21.15.137:0/1588786528 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd1d00cf970 con 0x7fd1e0102790 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e4890640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52351d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e4890640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52351d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52798/0 (socket says 172.21.15.137:52798) 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.776+0000 7fd1e4890640 1 -- 172.21.15.137:0/4175932995 learned_addr learned my addr 172.21.15.137:0/4175932995 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e5892640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1cc0a2cf0 0x7fd1cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e5091640 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347060 0x55ee52347440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e5892640 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52352240 0x55ee52354630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e4890640 1 -- 172.21.15.137:0/4175932995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52352240 msgr2=0x55ee52354630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e4890640 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52352240 0x55ee52354630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e4890640 1 -- 172.21.15.137:0/4175932995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347060 msgr2=0x55ee52347440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e4890640 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347060 0x55ee52347440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e4890640 1 -- 172.21.15.137:0/4175932995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee52244d40 con 0x55ee52347980 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e5892640 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52352240 0x55ee52354630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e4890640 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52351d00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0098820 tx=0x7fd1d009bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1868575a73077c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1e5892640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1cc0a2cf0 0x7fd1cc0a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ee52344970 tx=0x7fd1d8005ef0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1cb7fe640 1 -- 172.21.15.137:0/4175932995 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d009c550 con 0x55ee52347980 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1cb7fe640 1 -- 172.21.15.137:0/4175932995 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d009e070 con 0x55ee52347980 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.777+0000 7fd1cb7fe640 1 -- 172.21.15.137:0/4175932995 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00a0200 con 0x55ee52347980 2024-09-12T12:21:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 -- 172.21.15.137:0/4175932995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 msgr2=0x55ee52351d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52351d00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0098820 tx=0x7fd1d009bce0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 -- 172.21.15.137:0/4175932995 shutdown_connections 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52352240 0x55ee52354630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52351d00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/4175932995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347060 0x55ee52347440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 -- 172.21.15.137:0/4175932995 >> 172.21.15.137:0/4175932995 conn(0x55ee523435f0 msgr2=0x55ee523439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 -- 172.21.15.137:0/4175932995 shutdown_connections 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 -- 172.21.15.137:0/4175932995 wait complete. 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 -- start start 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.778+0000 7fd1e96ce740 1 Processor -- start 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5234f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee5234f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee5234fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ee52354e10 con 0x55ee52352240 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ee52234f10 con 0x55ee52347980 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ee52354f90 con 0x55ee52347060 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e96ce740 10 client.0 _ll_register_callbacks cb 0x55ee52258db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee5234fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee5234fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52800/0 (socket says 172.21.15.137:52800) 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 -- 172.21.15.137:0/1804286845 learned_addr learned my addr 172.21.15.137:0/1804286845 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e4890640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee5234f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5091640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5234f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 -- 172.21.15.137:0/1804286845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 msgr2=0x55ee5234f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5234f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 -- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 msgr2=0x55ee5234f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee5234f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5892640 1 -- 172.21.15.137:0/1804286845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee51fbb580 con 0x55ee52352240 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e4890640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee5234f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.779+0000 7fd1e5091640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5234f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e5892640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee5234fbc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8031ab0 tx=0x7fd1d802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12807b4b1865a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/1804286845 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80305c0 con 0x55ee52352240 2024-09-12T12:21:37.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/1804286845 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd1d8030760 con 0x55ee52352240 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d009e070 con 0x55ee52352240 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/1804286845 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1d8030960 con 0x55ee52352240 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 10 client.16482 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee52085960 con 0x55ee52352240 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/1804286845 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d009e070 con 0x55ee52352240 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/1804286845 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fd1d805b020 con 0x55ee52352240 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 10 client.16482 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 10 client.16482 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 20 client.16482 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee52361d10 0x55ee52364140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 4 client.16482 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.780+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee522b9b90 con 0x55ee52361d10 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.781+0000 7fd1e5091640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee52361d10 0x55ee52364140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.781+0000 7fd1e5091640 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee52361d10 0x55ee52364140 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939ddd5073e48dbb server_cookie=fef9f6939d72159 in_seq=0 out_seq=0 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.781+0000 7fd1c8ff9640 10 client.16482 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/1804286845 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee522b9b90 con 0x55ee52361d10 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1c8ff9640 10 client.16482 handle_command_reply: tid=0 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 client.16482 shutdown 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 msgr2=0x55ee5234fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee5234fbc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8031ab0 tx=0x7fd1d802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 shutdown_connections 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee52361d10 0x55ee52364140 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee5234fbc0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee5234f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/1804286845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5234f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1c8ff9640 0 client.16482 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:37.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 >> 172.21.15.137:0/1804286845 conn(0x55ee523435f0 msgr2=0x55ee52349320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:37.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 shutdown_connections 2024-09-12T12:21:37.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 1 -- 172.21.15.137:0/1804286845 wait complete. 2024-09-12T12:21:37.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.782+0000 7fd1e96ce740 20 client.16482 trim_cache size 0 max 16384 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 Processor -- start 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 -- start start 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee52351010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52349b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52352240 0x55ee52369a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01b2210 con 0x55ee52347980 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01afab0 con 0x55ee52352240 2024-09-12T12:21:37.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01ad920 con 0x55ee52347060 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52349b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52349b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52804/0 (socket says 172.21.15.137:52804) 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 -- 172.21.15.137:0/724291122 learned_addr learned my addr 172.21.15.137:0/724291122 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e5091640 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee52351010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 -- 172.21.15.137:0/724291122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 msgr2=0x55ee52351010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee52351010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 -- 172.21.15.137:0/724291122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52352240 msgr2=0x55ee52369a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52352240 0x55ee52369a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.799+0000 7fd1e4890640 1 -- 172.21.15.137:0/724291122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee52344280 con 0x55ee52347980 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e4890640 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52349b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00987d0 tx=0x7fd1d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a265e5b117944f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1caffd640 1 -- 172.21.15.137:0/724291122 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00b9070 con 0x55ee52347980 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1caffd640 1 -- 172.21.15.137:0/724291122 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d009e050 con 0x55ee52347980 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1caffd640 1 -- 172.21.15.137:0/724291122 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d009be50 con 0x55ee52347980 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 -- 172.21.15.137:0/724291122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 msgr2=0x55ee52349b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52349b10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00987d0 tx=0x7fd1d0004a40 comp rx=0 tx=0).stop 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 -- 172.21.15.137:0/724291122 shutdown_connections 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52352240 0x55ee52369a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52347980 0x55ee52349b10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/724291122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee52351010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 -- 172.21.15.137:0/724291122 >> 172.21.15.137:0/724291122 conn(0x55ee523435f0 msgr2=0x55ee52342750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.800+0000 7fd1e96ce740 1 -- 172.21.15.137:0/724291122 shutdown_connections 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 -- 172.21.15.137:0/724291122 wait complete. 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 -- start start 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 Processor -- start 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5235a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee52356b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee523570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01adf70 con 0x55ee52352240 2024-09-12T12:21:37.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01af3e0 con 0x55ee52347980 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e01ae640 con 0x55ee52347060 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e96ce740 10 client.0 _ll_register_callbacks cb 0x7fd1d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e5892640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee523570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e5892640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee523570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52810/0 (socket says 172.21.15.137:52810) 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e5892640 1 -- 172.21.15.137:0/98266488 learned_addr learned my addr 172.21.15.137:0/98266488 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.801+0000 7fd1e5091640 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5235a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e4890640 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee52356b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e5892640 1 -- 172.21.15.137:0/98266488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 msgr2=0x55ee5235a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e5892640 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5235a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e5892640 1 -- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 msgr2=0x55ee52356b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e5892640 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee52356b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e5892640 1 -- 172.21.15.137:0/98266488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee52357790 con 0x55ee52352240 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e5892640 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee523570a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009bf0 tx=0x7fd1d8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f6637bdc244a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/98266488 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8030d00 con 0x55ee52352240 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/98266488 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd1d805d460 con 0x55ee52352240 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d009e050 con 0x55ee52352240 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/98266488 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd1d806e050 con 0x55ee52352240 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e96ce740 10 client.16488 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:37.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ee523cd910 con 0x55ee52352240 2024-09-12T12:21:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.802+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/98266488 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d009e050 con 0x55ee52352240 2024-09-12T12:21:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/98266488 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fd1d805b040 con 0x55ee52352240 2024-09-12T12:21:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e96ce740 10 client.16488 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e96ce740 10 client.16488 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:21:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e96ce740 20 client.16488 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee523ce030 0x55ee523d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:37.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e96ce740 4 client.16488 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "627d6bb5", "release": true, "await": true}] 2024-09-12T12:21:37.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "627d6bb5", "release": true, "await": true}) -- 0x55ee5233e950 con 0x55ee523ce030 2024-09-12T12:21:37.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e5091640 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee523ce030 0x55ee523d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:37.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1e5091640 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee523ce030 0x55ee523d0450 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218e1c056b6c54d server_cookie=d1ae181132a70f7b in_seq=0 out_seq=0 2024-09-12T12:21:37.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.803+0000 7fd1c8ff9640 10 client.16488 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:37.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.809+0000 7fd1c8ff9640 1 -- 172.21.15.137:0/98266488 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ee5233e950 con 0x55ee523ce030 2024-09-12T12:21:37.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1c8ff9640 10 client.16488 handle_command_reply: tid=0 2024-09-12T12:21:37.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 client.16488 shutdown 2024-09-12T12:21:37.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 msgr2=0x55ee523570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee523570a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009bf0 tx=0x7fd1d8005fd0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 shutdown_connections 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ee523ce030 0x55ee523d0450 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ee52352240 0x55ee523570a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ee52347980 0x55ee52356b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 --2- 172.21.15.137:0/98266488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee52347060 0x55ee5235a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1c8ff9640 0 client.16488 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 >> 172.21.15.137:0/98266488 conn(0x55ee523435f0 msgr2=0x55ee5235d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 shutdown_connections 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 1 -- 172.21.15.137:0/98266488 wait complete. 2024-09-12T12:21:37.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e96ce740 20 client.16488 trim_cache size 0 max 16384 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 34, 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 36, 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "627d6bb5": { 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 36, 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:21:37.819 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 139.4, 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:37.820 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:37.821 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:37.821 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:37.821 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:21:37.821 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:21:37.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1cc0a2cf0 msgr2=0x7fd1cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1cc0a2cf0 0x7fd1cc0a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55ee52344970 tx=0x7fd1d8005ef0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0102790 msgr2=0x7fd1e0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.810+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0102790 0x7fd1e0074a30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000e9f0 tx=0x7fd1d000edf0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 shutdown_connections 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1cc0a2cf0 0x7fd1cc0a51b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1e0106f10 0x7fd1e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd1e0102790 0x7fd1e0074a30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 --2- 172.21.15.137:0/1588786528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1e0101de0 0x7fd1e00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:37.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 >> 172.21.15.137:0/1588786528 conn(0x7fd1e00fd8c0 msgr2=0x7fd1e00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:37.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 shutdown_connections 2024-09-12T12:21:37.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:21:37.811+0000 7fd1e72f5640 1 -- 172.21.15.137:0/1588786528 wait complete. 2024-09-12T12:21:38.566 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '627d6bb5', total seconds elapsed: 22.7 2024-09-12T12:21:38.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 646568 7.57 MB/sec execute 313 sec latency 19074.276 ms 2024-09-12T12:21:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:39 smithi189 ceph-mon[35475]: pgmap v262: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 838 KiB/s wr, 35 op/s 2024-09-12T12:21:39.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:39 smithi171 ceph-mon[35098]: pgmap v262: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 838 KiB/s wr, 35 op/s 2024-09-12T12:21:39.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 648176 7.57 MB/sec execute 314 sec latency 18.500 ms 2024-09-12T12:21:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:39 smithi137 ceph-mon[30950]: pgmap v262: 97 pgs: 97 active+clean; 186 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 838 KiB/s wr, 35 op/s 2024-09-12T12:21:40.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 650341 7.59 MB/sec execute 315 sec latency 11.011 ms 2024-09-12T12:21:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:41 smithi189 ceph-mon[35475]: pgmap v263: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-09-12T12:21:41.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:41 smithi171 ceph-mon[35098]: pgmap v263: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-09-12T12:21:41.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 652450 7.59 MB/sec execute 316 sec latency 19.388 ms 2024-09-12T12:21:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:41 smithi137 ceph-mon[30950]: pgmap v263: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 78 op/s 2024-09-12T12:21:42.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 654314 7.60 MB/sec execute 317 sec latency 18.467 ms 2024-09-12T12:21:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:43 smithi189 ceph-mon[35475]: pgmap v264: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 430 KiB/s wr, 49 op/s 2024-09-12T12:21:43.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:43 smithi171 ceph-mon[35098]: pgmap v264: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 430 KiB/s wr, 49 op/s 2024-09-12T12:21:43.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 656764 7.62 MB/sec execute 318 sec latency 14.193 ms 2024-09-12T12:21:43.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:43 smithi137 ceph-mon[30950]: pgmap v264: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 430 KiB/s wr, 49 op/s 2024-09-12T12:21:44.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 658408 7.63 MB/sec execute 319 sec latency 21.490 ms 2024-09-12T12:21:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:45 smithi189 ceph-mon[35475]: pgmap v265: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 430 KiB/s wr, 49 op/s 2024-09-12T12:21:45.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:45 smithi171 ceph-mon[35098]: pgmap v265: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 430 KiB/s wr, 49 op/s 2024-09-12T12:21:45.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 660906 7.65 MB/sec execute 320 sec latency 14.906 ms 2024-09-12T12:21:45.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:45 smithi137 ceph-mon[30950]: pgmap v265: 97 pgs: 97 active+clean; 190 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 430 KiB/s wr, 49 op/s 2024-09-12T12:21:46.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 662961 7.65 MB/sec execute 321 sec latency 28.389 ms 2024-09-12T12:21:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:47 smithi189 ceph-mon[35475]: pgmap v266: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-09-12T12:21:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:47 smithi171 ceph-mon[35098]: pgmap v266: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-09-12T12:21:47.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 664795 7.66 MB/sec execute 322 sec latency 17.773 ms 2024-09-12T12:21:47.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:47 smithi137 ceph-mon[30950]: pgmap v266: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-09-12T12:21:48.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 667459 7.68 MB/sec execute 323 sec latency 15.960 ms 2024-09-12T12:21:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 -- 172.21.15.137:0/113920134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864106830 msgr2=0x7fd864106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 --2- 172.21.15.137:0/113920134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864106830 0x7fd864106c10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd854009930 tx=0x7fd85402a420 comp rx=0 tx=0).stop 2024-09-12T12:21:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 -- 172.21.15.137:0/113920134 shutdown_connections 2024-09-12T12:21:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 --2- 172.21.15.137:0/113920134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8641011c0 0x7fd86410d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 --2- 172.21.15.137:0/113920134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd864100820 0x7fd864100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 --2- 172.21.15.137:0/113920134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864106830 0x7fd864106c10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 -- 172.21.15.137:0/113920134 >> 172.21.15.137:0/113920134 conn(0x7fd8640fc500 msgr2=0x7fd8640fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 -- 172.21.15.137:0/113920134 shutdown_connections 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.532+0000 7fd869801640 1 -- 172.21.15.137:0/113920134 wait complete. 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 Processor -- start 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 -- start start 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864100820 0x7fd8641971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8641011c0 0x7fd8641976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd864106830 0x7fd86419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd864110c70 con 0x7fd864100820 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd864110af0 con 0x7fd864106830 2024-09-12T12:21:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd869801640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd864110df0 con 0x7fd8641011c0 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd862ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864100820 0x7fd8641971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd862ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864100820 0x7fd8641971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59986/0 (socket says 172.21.15.137:59986) 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd862ffd640 1 -- 172.21.15.137:0/1909206602 learned_addr learned my addr 172.21.15.137:0/1909206602 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd8627fc640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8641011c0 0x7fd8641976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.533+0000 7fd8637fe640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd864106830 0x7fd86419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd862ffd640 1 -- 172.21.15.137:0/1909206602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8641011c0 msgr2=0x7fd8641976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd862ffd640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8641011c0 0x7fd8641976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd862ffd640 1 -- 172.21.15.137:0/1909206602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd864106830 msgr2=0x7fd86419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd862ffd640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd864106830 0x7fd86419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd862ffd640 1 -- 172.21.15.137:0/1909206602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86419c190 con 0x7fd864100820 2024-09-12T12:21:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd862ffd640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864100820 0x7fd8641971a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd854009230 tx=0x7fd854004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd84bfff640 1 -- 172.21.15.137:0/1909206602 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85403f030 con 0x7fd864100820 2024-09-12T12:21:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86419c420 con 0x7fd864100820 2024-09-12T12:21:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd84bfff640 1 -- 172.21.15.137:0/1909206602 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd854004440 con 0x7fd864100820 2024-09-12T12:21:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd84bfff640 1 -- 172.21.15.137:0/1909206602 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854035070 con 0x7fd864100820 2024-09-12T12:21:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.534+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8641a3d10 con 0x7fd864100820 2024-09-12T12:21:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.536+0000 7fd84bfff640 1 -- 172.21.15.137:0/1909206602 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd854042030 con 0x7fd864100820 2024-09-12T12:21:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 Processor -- start 2024-09-12T12:21:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 -- start start 2024-09-12T12:21:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004eeff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004efcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004efd090 0x558004eff480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558004e542f0 con 0x558004eefb60 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558004dfc270 con 0x558004efd090 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd86c672740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558004e502b0 con 0x558004ef0480 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004eeff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004efd090 0x558004eff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004efd090 0x558004eff480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:33368/0 (socket says 172.21.15.137:33368) 2024-09-12T12:21:49.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 -- 172.21.15.137:0/4141049397 learned_addr learned my addr 172.21.15.137:0/4141049397 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004eeff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59994/0 (socket says 172.21.15.137:59994) 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd862ffd640 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004efcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd84bfff640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd84c0a2cf0 0x7fd84c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 -- 172.21.15.137:0/4141049397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 msgr2=0x558004efcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004efcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 -- 172.21.15.137:0/4141049397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 msgr2=0x558004eeff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004eeff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8627fc640 1 -- 172.21.15.137:0/4141049397 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558004dfb340 con 0x558004efd090 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd8637fe640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd84c0a2cf0 0x7fd84c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.537+0000 7fd84bfff640 1 -- 172.21.15.137:0/1909206602 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd8540ef700 con 0x7fd864100820 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.538+0000 7fd8627fc640 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004efd090 0x558004eff480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd858009c90 tx=0x7fd858011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=debaf70b8e7a026d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.538+0000 7fd8497fa640 1 -- 172.21.15.137:0/4141049397 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85801e070 con 0x558004efd090 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.538+0000 7fd8497fa640 1 -- 172.21.15.137:0/4141049397 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd858019b10 con 0x558004efd090 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.538+0000 7fd8497fa640 1 -- 172.21.15.137:0/4141049397 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85800ea90 con 0x558004efd090 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.538+0000 7fd86c672740 1 -- 172.21.15.137:0/4141049397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004efd090 msgr2=0x558004eff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.538+0000 7fd8637fe640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd84c0a2cf0 0x7fd84c0a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009b80 tx=0x7fd85c004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.538+0000 7fd86c672740 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004efd090 0x558004eff480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd858009c90 tx=0x7fd858011810 comp rx=0 tx=0).stop 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- 172.21.15.137:0/4141049397 shutdown_connections 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004efd090 0x558004eff480 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004efcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 --2- 172.21.15.137:0/4141049397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004eeff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- 172.21.15.137:0/4141049397 >> 172.21.15.137:0/4141049397 conn(0x558004eec230 msgr2=0x558004eec610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.548 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- 172.21.15.137:0/4141049397 shutdown_connections 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- 172.21.15.137:0/4141049397 wait complete. 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- start start 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 Processor -- start 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004eefb60 0x558004f00110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004f00650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f00bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558004effcc0 con 0x558004efd090 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558004ddd290 con 0x558004ef0480 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558004effe40 con 0x558004eefb60 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd86c672740 10 client.0 _ll_register_callbacks cb 0x558004e0e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f00bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.549 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd862ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004eefb60 0x558004f00110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f00bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60010/0 (socket says 172.21.15.137:60010) 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd8627fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004f00650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.539+0000 7fd8637fe640 1 -- 172.21.15.137:0/3915532223 learned_addr learned my addr 172.21.15.137:0/3915532223 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd8637fe640 1 -- 172.21.15.137:0/3915532223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004eefb60 msgr2=0x558004f00110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd8637fe640 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004eefb60 0x558004f00110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd8637fe640 1 -- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 msgr2=0x558004f00650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd8637fe640 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004f00650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd8637fe640 1 -- 172.21.15.137:0/3915532223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558004b67210 con 0x558004efd090 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd8627fc640 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004f00650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:21:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd8637fe640 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f00bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00fc70 tx=0x7fd85c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdfffc3f6a1ef10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd846ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd846ff5640 1 -- 172.21.15.137:0/3915532223 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c02c060 con 0x558004efd090 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd846ff5640 1 -- 172.21.15.137:0/3915532223 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd85c027600 con 0x558004efd090 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd858019b10 con 0x558004efd090 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd846ff5640 1 -- 172.21.15.137:0/3915532223 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd85c027800 con 0x558004efd090 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd86c672740 10 client.16500 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558004c33620 con 0x558004efd090 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.540+0000 7fd846ff5640 1 -- 172.21.15.137:0/3915532223 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd858019b10 con 0x558004efd090 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd846ff5640 1 -- 172.21.15.137:0/3915532223 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fd85c01b9f0 con 0x558004efd090 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd86c672740 10 client.16500 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd86c672740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd86c672740 20 client.16500 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd86c672740 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f0f320 0x558004f11730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd86c672740 4 client.16500 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558004e61a20 con 0x558004f0f320 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd862ffd640 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f0f320 0x558004f11730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd846ff5640 10 client.16500 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.541+0000 7fd862ffd640 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f0f320 0x558004f11730 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b02553dc04ce2c server_cookie=e64dadd8526d86ce in_seq=0 out_seq=0 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd846ff5640 1 -- 172.21.15.137:0/3915532223 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558004e61a20 con 0x558004f0f320 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd846ff5640 10 client.16500 handle_command_reply: tid=0 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 client.16500 shutdown 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 msgr2=0x558004f00bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f00bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00fc70 tx=0x7fd85c004ab0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 shutdown_connections 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f0f320 0x558004f11730 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f00bb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004f00650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd846ff5640 0 client.16500 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 --2- 172.21.15.137:0/3915532223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004eefb60 0x558004f00110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 >> 172.21.15.137:0/3915532223 conn(0x558004eec230 msgr2=0x558004ef1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 shutdown_connections 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 1 -- 172.21.15.137:0/3915532223 wait complete. 2024-09-12T12:21:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.542+0000 7fd86c672740 20 client.16500 trim_cache size 0 max 16384 2024-09-12T12:21:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 Processor -- start 2024-09-12T12:21:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 -- start start 2024-09-12T12:21:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004efaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004efaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.562 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004efd090 0x558004f17040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641add60 con 0x558004eefb60 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641ab600 con 0x558004ef0480 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd86c672740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641a9470 con 0x558004efd090 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd862ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004efaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd862ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004efaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60014/0 (socket says 172.21.15.137:60014) 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.557+0000 7fd862ffd640 1 -- 172.21.15.137:0/3512665718 learned_addr learned my addr 172.21.15.137:0/3512665718 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd8637fe640 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004efd090 0x558004f17040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd8627fc640 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004efaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd862ffd640 1 -- 172.21.15.137:0/3512665718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004efd090 msgr2=0x558004f17040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd862ffd640 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004efd090 0x558004f17040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd862ffd640 1 -- 172.21.15.137:0/3512665718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 msgr2=0x558004efaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.563 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd862ffd640 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004efaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd862ffd640 1 -- 172.21.15.137:0/3512665718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558004eff7c0 con 0x558004eefb60 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd8627fc640 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004efaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd862ffd640 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004efaab0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd8540b8610 tx=0x7fd8540b8370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130da32f7c72cf6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd848ff9640 1 -- 172.21.15.137:0/3512665718 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854049040 con 0x558004eefb60 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd848ff9640 1 -- 172.21.15.137:0/3512665718 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8540f5030 con 0x558004eefb60 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.558+0000 7fd848ff9640 1 -- 172.21.15.137:0/3512665718 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540bfff0 con 0x558004eefb60 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- 172.21.15.137:0/3512665718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 msgr2=0x558004efaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004efaab0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd8540b8610 tx=0x7fd8540b8370 comp rx=0 tx=0).stop 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- 172.21.15.137:0/3512665718 shutdown_connections 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004efd090 0x558004f17040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004ef0480 0x558004efaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 --2- 172.21.15.137:0/3512665718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004eefb60 0x558004efaab0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.564 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- 172.21.15.137:0/3512665718 >> 172.21.15.137:0/3512665718 conn(0x558004eec230 msgr2=0x558004ee6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- 172.21.15.137:0/3512665718 shutdown_connections 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- 172.21.15.137:0/3512665718 wait complete. 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- start start 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 Processor -- start 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004eefb60 0x558004f03a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004f03f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f70fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641a9ac0 con 0x558004efd090 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641aaf30 con 0x558004eefb60 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8641aa190 con 0x558004ef0480 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd86c672740 10 client.0 _ll_register_callbacks cb 0x7fd85c009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f70fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.565 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd8627fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004f03f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd8637fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f70fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60026/0 (socket says 172.21.15.137:60026) 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd8637fe640 1 -- 172.21.15.137:0/263380696 learned_addr learned my addr 172.21.15.137:0/263380696 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.559+0000 7fd862ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004eefb60 0x558004f03a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd8637fe640 1 -- 172.21.15.137:0/263380696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 msgr2=0x558004f03f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd8637fe640 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004f03f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd8637fe640 1 -- 172.21.15.137:0/263380696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004eefb60 msgr2=0x558004f03a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd8637fe640 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004eefb60 0x558004f03a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd8637fe640 1 -- 172.21.15.137:0/263380696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558004f71670 con 0x558004efd090 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd8637fe640 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f70fd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009f20 tx=0x7fd85c027690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f6dcc99457ad80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd846ff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd846ff5640 1 -- 172.21.15.137:0/263380696 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c03e060 con 0x558004efd090 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd846ff5640 1 -- 172.21.15.137:0/263380696 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8484+0+0 (secure 0 0 0) 0x7fd85c027db0 con 0x558004efd090 2024-09-12T12:21:49.566 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8540f5030 con 0x558004efd090 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd846ff5640 1 -- 172.21.15.137:0/263380696 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd85c00f830 con 0x558004efd090 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd86c672740 10 client.16506 fetch_fsmap learned FSMap version 34 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558004f79b80 con 0x558004efd090 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd846ff5640 1 -- 172.21.15.137:0/263380696 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8540f5030 con 0x558004efd090 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd846ff5640 1 -- 172.21.15.137:0/263380696 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fd85c01b7f0 con 0x558004efd090 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd86c672740 10 client.16506 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.560+0000 7fd86c672740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.561+0000 7fd86c672740 20 client.16506 populate_metadata read hostname 'smithi137' 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.561+0000 7fd86c672740 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f7a240 0x558004f7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.561+0000 7fd86c672740 4 client.16506 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.561+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558004e5bd70 con 0x558004f7a240 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.561+0000 7fd862ffd640 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f7a240 0x558004f7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:21:49.567 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.561+0000 7fd846ff5640 10 client.16506 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:49.568 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.561+0000 7fd862ffd640 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f7a240 0x558004f7c630 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e35ed40d4003f06 server_cookie=d2d3d93c596a29bc in_seq=0 out_seq=0 2024-09-12T12:21:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:49 smithi189 ceph-mon[35475]: pgmap v267: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-09-12T12:21:49.634 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd846ff5640 1 -- 172.21.15.137:0/263380696 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558004e5bd70 con 0x558004f7a240 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd846ff5640 10 client.16506 handle_command_reply: tid=0 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 client.16506 shutdown 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 msgr2=0x558004f70fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f70fd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009f20 tx=0x7fd85c027690 comp rx=0 tx=0).stop 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 shutdown_connections 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558004f7a240 0x558004f7c630 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558004efd090 0x558004f70fd0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558004ef0480 0x558004f03f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 --2- 172.21.15.137:0/263380696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558004eefb60 0x558004f03a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 >> 172.21.15.137:0/263380696 conn(0x558004eec230 msgr2=0x558004efa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd846ff5640 0 client.16506 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 shutdown_connections 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 1 -- 172.21.15.137:0/263380696 wait complete. 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.633+0000 7fd86c672740 20 client.16506 trim_cache size 0 max 16384 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:21:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd84c0a2cf0 msgr2=0x7fd84c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd84c0a2cf0 0x7fd84c0a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009b80 tx=0x7fd85c004a40 comp rx=0 tx=0).stop 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864100820 msgr2=0x7fd8641971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864100820 0x7fd8641971a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd854009230 tx=0x7fd854004040 comp rx=0 tx=0).stop 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 shutdown_connections 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd84c0a2cf0 0x7fd84c0a51b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd864106830 0x7fd86419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8641011c0 0x7fd8641976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 --2- 172.21.15.137:0/1909206602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd864100820 0x7fd8641971a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.634+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 >> 172.21.15.137:0/1909206602 conn(0x7fd8640fc500 msgr2=0x7fd8640fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.635+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 shutdown_connections 2024-09-12T12:21:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:21:49.635+0000 7fd869801640 1 -- 172.21.15.137:0/1909206602 wait complete. 2024-09-12T12:21:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:49 smithi171 ceph-mon[35098]: pgmap v267: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-09-12T12:21:49.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 668730 7.68 MB/sec execute 324 sec latency 63.890 ms 2024-09-12T12:21:49.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:49 smithi137 ceph-mon[30950]: pgmap v267: 97 pgs: 97 active+clean; 198 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-09-12T12:21:50.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 671310 7.70 MB/sec execute 325 sec latency 15.972 ms 2024-09-12T12:21:51.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:51 smithi189 ceph-mon[35475]: pgmap v268: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 250 op/s 2024-09-12T12:21:51.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:51 smithi171 ceph-mon[35098]: pgmap v268: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 250 op/s 2024-09-12T12:21:51.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 672683 7.70 MB/sec execute 326 sec latency 17.429 ms 2024-09-12T12:21:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:51 smithi137 ceph-mon[30950]: pgmap v268: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 250 op/s 2024-09-12T12:21:52.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 675138 7.72 MB/sec execute 327 sec latency 12.730 ms 2024-09-12T12:21:53.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:53 smithi189 ceph-mon[35475]: pgmap v269: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-12T12:21:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:53 smithi171 ceph-mon[35098]: pgmap v269: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-12T12:21:53.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 676631 7.71 MB/sec execute 328 sec latency 14.996 ms 2024-09-12T12:21:53.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:53 smithi137 ceph-mon[30950]: pgmap v269: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-12T12:21:54.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 678747 7.73 MB/sec execute 329 sec latency 12.941 ms 2024-09-12T12:21:55.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:55 smithi189 ceph-mon[35475]: pgmap v270: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-12T12:21:55.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:55 smithi171 ceph-mon[35098]: pgmap v270: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-12T12:21:55.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 680678 7.73 MB/sec execute 330 sec latency 22.490 ms 2024-09-12T12:21:55.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:55 smithi137 ceph-mon[30950]: pgmap v270: 97 pgs: 97 active+clean; 207 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-12T12:21:56.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 682436 7.74 MB/sec execute 331 sec latency 12.354 ms 2024-09-12T12:21:57.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:57 smithi189 ceph-mon[35475]: pgmap v271: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-09-12T12:21:57.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:57 smithi171 ceph-mon[35098]: pgmap v271: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-09-12T12:21:57.742 INFO:tasks.workunit.client.0.smithi137.stdout: 1 684879 7.76 MB/sec execute 332 sec latency 31.716 ms 2024-09-12T12:21:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:57 smithi137 ceph-mon[30950]: pgmap v271: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.1 MiB/s wr, 356 op/s 2024-09-12T12:21:58.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 686554 7.76 MB/sec execute 333 sec latency 14.887 ms 2024-09-12T12:21:59.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:21:59 smithi189 ceph-mon[35475]: pgmap v272: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-12T12:21:59.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:21:59 smithi171 ceph-mon[35098]: pgmap v272: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-12T12:21:59.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 689016 7.78 MB/sec execute 334 sec latency 14.288 ms 2024-09-12T12:21:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:21:59 smithi137 ceph-mon[30950]: pgmap v272: 97 pgs: 97 active+clean; 203 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-12T12:22:00.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 689980 7.77 MB/sec execute 335 sec latency 22.572 ms 2024-09-12T12:22:01.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:01 smithi189 ceph-mon[35475]: pgmap v273: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-09-12T12:22:01.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:01 smithi171 ceph-mon[35098]: pgmap v273: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-09-12T12:22:01.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 692243 7.78 MB/sec execute 336 sec latency 22.591 ms 2024-09-12T12:22:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:01 smithi137 ceph-mon[30950]: pgmap v273: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.1 MiB/s wr, 351 op/s 2024-09-12T12:22:02.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 693224 7.77 MB/sec execute 337 sec latency 46.507 ms 2024-09-12T12:22:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 -- 172.21.15.137:0/2798696324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0101390 msgr2=0x7fa0d0105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/2798696324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0101390 0x7fa0d0105850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009860 tx=0x7fa0cc02a3d0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 -- 172.21.15.137:0/2798696324 shutdown_connections 2024-09-12T12:22:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/2798696324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d0105ea0 0x7fa0d0106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/2798696324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0101390 0x7fa0d0105850 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/2798696324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0d0100a70 0x7fa0d0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 -- 172.21.15.137:0/2798696324 >> 172.21.15.137:0/2798696324 conn(0x7fa0d0077990 msgr2=0x7fa0d00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 -- 172.21.15.137:0/2798696324 shutdown_connections 2024-09-12T12:22:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.776+0000 7fa0d80a5640 1 -- 172.21.15.137:0/2798696324 wait complete. 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.777+0000 7fa0d80a5640 1 Processor -- start 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.777+0000 7fa0d80a5640 1 -- start start 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.777+0000 7fa0d80a5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0100a70 0x7fa0d01a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.777+0000 7fa0d80a5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d0101390 0x7fa0d01a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.777+0000 7fa0d80a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0d0105ea0 0x7fa0d019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.777+0000 7fa0d80a5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01184f0 con 0x7fa0d0105ea0 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d80a5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0118370 con 0x7fa0d0100a70 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d80a5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d0118670 con 0x7fa0d0101390 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5e41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0100a70 0x7fa0d01a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5e41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0100a70 0x7fa0d01a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:36384/0 (socket says 172.21.15.137:36384) 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5e41640 1 -- 172.21.15.137:0/1239597425 learned_addr learned my addr 172.21.15.137:0/1239597425 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5640640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d0101390 0x7fa0d01a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d6642640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0d0105ea0 0x7fa0d019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5640640 1 -- 172.21.15.137:0/1239597425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0100a70 msgr2=0x7fa0d01a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5640640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0100a70 0x7fa0d01a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5640640 1 -- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0d0105ea0 msgr2=0x7fa0d019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5640640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0d0105ea0 0x7fa0d019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.778+0000 7fa0d5640640 1 -- 172.21.15.137:0/1239597425 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d019ef90 con 0x7fa0d0101390 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.779+0000 7fa0d5640640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d0101390 0x7fa0d01a4b90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc02a8f0 tx=0x7fa0cc0309a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.779+0000 7fa0c6ffd640 1 -- 172.21.15.137:0/1239597425 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc040070 con 0x7fa0d0101390 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.779+0000 7fa0c6ffd640 1 -- 172.21.15.137:0/1239597425 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0cc0055a0 con 0x7fa0d0101390 2024-09-12T12:22:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.779+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d019f220 con 0x7fa0d0101390 2024-09-12T12:22:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.779+0000 7fa0c6ffd640 1 -- 172.21.15.137:0/1239597425 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc004920 con 0x7fa0d0101390 2024-09-12T12:22:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.779+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d01ab4a0 con 0x7fa0d0101390 2024-09-12T12:22:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.783+0000 7fa0c6ffd640 1 -- 172.21.15.137:0/1239597425 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa0cc043070 con 0x7fa0d0101390 2024-09-12T12:22:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.783+0000 7fa0dae45740 1 Processor -- start 2024-09-12T12:22:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.784+0000 7fa0c6ffd640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa0bc0a2ea0 0x7fa0bc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.784+0000 7fa0c6ffd640 1 -- 172.21.15.137:0/1239597425 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8528+0+0 (secure 0 0 0) 0x7fa0cc0f0030 con 0x7fa0d0101390 2024-09-12T12:22:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.784+0000 7fa0dae45740 1 -- start start 2024-09-12T12:22:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.784+0000 7fa0d6642640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa0bc0a2ea0 0x7fa0bc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 0x55ed11260ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed1126b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed1126e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0dae45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ed111c52f0 con 0x55ed11261410 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0dae45740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ed1116d270 con 0x55ed1126bcb0 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0dae45740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ed111c12b0 con 0x55ed11260af0 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d5640640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed1126b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d5640640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed1126b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50256/0 (socket says 172.21.15.137:50256) 2024-09-12T12:22:02.785 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d5640640 1 -- 172.21.15.137:0/2938908074 learned_addr learned my addr 172.21.15.137:0/2938908074 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d6642640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa0bc0a2ea0 0x7fa0bc0a5360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800ad40 tx=0x7fa0c800a4d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d6642640 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed1126e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d5640640 1 -- 172.21.15.137:0/2938908074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 msgr2=0x55ed11260ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d5640640 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 0x55ed11260ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d5640640 1 -- 172.21.15.137:0/2938908074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 msgr2=0x55ed1126e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.785+0000 7fa0d5640640 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed1126e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.786+0000 7fa0d5640640 1 -- 172.21.15.137:0/2938908074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1116c340 con 0x55ed11261410 2024-09-12T12:22:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.787+0000 7fa0d5640640 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed1126b770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0b8a20 tx=0x7fa0cc030a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87314ca54e7aece9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.787+0000 7fa0bbfff640 1 -- 172.21.15.137:0/2938908074 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc04a040 con 0x55ed11261410 2024-09-12T12:22:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.787+0000 7fa0bbfff640 1 -- 172.21.15.137:0/2938908074 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0cc0bc5c0 con 0x55ed11261410 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.788+0000 7fa0bbfff640 1 -- 172.21.15.137:0/2938908074 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc031520 con 0x55ed11261410 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.788+0000 7fa0dae45740 1 -- 172.21.15.137:0/2938908074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 msgr2=0x55ed1126b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.788+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed1126b770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0b8a20 tx=0x7fa0cc030a10 comp rx=0 tx=0).stop 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.789+0000 7fa0dae45740 1 -- 172.21.15.137:0/2938908074 shutdown_connections 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.789+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed1126e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.789+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed1126b770 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.789+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2938908074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 0x55ed11260ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.789+0000 7fa0dae45740 1 -- 172.21.15.137:0/2938908074 >> 172.21.15.137:0/2938908074 conn(0x55ed1125d060 msgr2=0x55ed1125d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.789+0000 7fa0dae45740 1 -- 172.21.15.137:0/2938908074 shutdown_connections 2024-09-12T12:22:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.789+0000 7fa0dae45740 1 -- 172.21.15.137:0/2938908074 wait complete. 2024-09-12T12:22:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.790+0000 7fa0dae45740 1 -- start start 2024-09-12T12:22:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.790+0000 7fa0dae45740 1 Processor -- start 2024-09-12T12:22:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 0x55ed11268bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11269130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed11269670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0dae45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ed111c52f0 con 0x55ed11261410 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0d5640640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11269130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0d5640640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11269130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50270/0 (socket says 172.21.15.137:50270) 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0d5640640 1 -- 172.21.15.137:0/1385436006 learned_addr learned my addr 172.21.15.137:0/1385436006 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0dae45740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ed1114e290 con 0x55ed1126bcb0 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ed1126e640 con 0x55ed11260af0 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0d6642640 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed11269670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.791+0000 7fa0dae45740 10 client.0 _ll_register_callbacks cb 0x55ed1117f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 0x55ed11268bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0d5640640 1 -- 172.21.15.137:0/1385436006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 msgr2=0x55ed11268bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0d5640640 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 0x55ed11268bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0d5640640 1 -- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 msgr2=0x55ed11269670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0d5640640 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed11269670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0d5640640 1 -- 172.21.15.137:0/1385436006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed10ed8210 con 0x55ed11261410 2024-09-12T12:22:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0d5640640 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11269130 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0c4c50 tx=0x7fa0cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15866c680280bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.792+0000 7fa0b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:22:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.793+0000 7fa0b97fa640 1 -- 172.21.15.137:0/1385436006 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc04a040 con 0x55ed11261410 2024-09-12T12:22:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.793+0000 7fa0b97fa640 1 -- 172.21.15.137:0/1385436006 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8528+0+0 (secure 0 0 0) 0x7fa0cc0ed9c0 con 0x55ed11261410 2024-09-12T12:22:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.793+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0cc0bc5c0 con 0x55ed11261410 2024-09-12T12:22:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.793+0000 7fa0b97fa640 1 -- 172.21.15.137:0/1385436006 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa0cc0edbc0 con 0x55ed11261410 2024-09-12T12:22:02.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.793+0000 7fa0dae45740 10 client.16518 fetch_fsmap learned FSMap version 34 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.793+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ed10fa4620 con 0x55ed11261410 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.794+0000 7fa0b97fa640 1 -- 172.21.15.137:0/1385436006 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0cc0da070 con 0x55ed11261410 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.794+0000 7fa0b97fa640 1 -- 172.21.15.137:0/1385436006 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fa0cc0c4cf0 con 0x55ed11261410 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.794+0000 7fa0dae45740 10 client.16518 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.794+0000 7fa0dae45740 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.794+0000 7fa0dae45740 20 client.16518 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.794+0000 7fa0dae45740 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed1127b710 0x55ed1127db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.794+0000 7fa0dae45740 4 client.16518 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:22:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.795+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa0cc0bc5c0 con 0x55ed1127b710 2024-09-12T12:22:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.795+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed1127b710 0x55ed1127db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.796+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed1127b710 0x55ed1127db00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22b5bffd09fae03 server_cookie=8ab150a0d387d6f in_seq=0 out_seq=0 2024-09-12T12:22:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.796+0000 7fa0b97fa640 10 client.16518 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.797+0000 7fa0b97fa640 1 -- 172.21.15.137:0/1385436006 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa0cc0bc5c0 con 0x55ed1127b710 2024-09-12T12:22:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.797+0000 7fa0b97fa640 10 client.16518 handle_command_reply: tid=0 2024-09-12T12:22:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.798+0000 7fa0dae45740 1 client.16518 shutdown 2024-09-12T12:22:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.799+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 msgr2=0x55ed11269130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.799+0000 7fa0dae45740 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11269130 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0c4c50 tx=0x7fa0cc005f90 comp rx=0 tx=0).stop 2024-09-12T12:22:02.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 shutdown_connections 2024-09-12T12:22:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0dae45740 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed1126bcb0 0x55ed11269670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0dae45740 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed1127b710 0x55ed1127db00 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0dae45740 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11269130 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0dae45740 1 --2- 172.21.15.137:0/1385436006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11260af0 0x55ed11268bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 >> 172.21.15.137:0/1385436006 conn(0x55ed1125d060 msgr2=0x55ed11262dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0b97fa640 0 client.16518 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:02.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.800+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 shutdown_connections 2024-09-12T12:22:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.801+0000 7fa0dae45740 1 -- 172.21.15.137:0/1385436006 wait complete. 2024-09-12T12:22:02.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.801+0000 7fa0dae45740 20 client.16518 trim_cache size 0 max 16384 2024-09-12T12:22:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.830+0000 7fa0dae45740 1 Processor -- start 2024-09-12T12:22:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.830+0000 7fa0dae45740 1 -- start start 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.830+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed112694c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.830+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11262c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed1126bcb0 0x55ed11283270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0dae45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01b6830 con 0x55ed11261410 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0dae45740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01b40d0 con 0x55ed11260af0 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0dae45740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01b1f20 con 0x55ed1126bcb0 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed112694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5640640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11262c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed112694c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:36400/0 (socket says 172.21.15.137:36400) 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5640640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11262c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50286/0 (socket says 172.21.15.137:50286) 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2577618466 learned_addr learned my addr 172.21.15.137:0/2577618466 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d6642640 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed1126bcb0 0x55ed11283270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2577618466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed1126bcb0 msgr2=0x55ed11283270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed1126bcb0 0x55ed11283270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2577618466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 msgr2=0x55ed11262c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11262c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.831+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2577618466 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1127dfd0 con 0x55ed11260af0 2024-09-12T12:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.832+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed112694c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009d20 tx=0x7fa0c0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eda9a8b6847c8c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.832+0000 7fa0bb7fe640 1 -- 172.21.15.137:0/2577618466 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0005220 con 0x55ed11260af0 2024-09-12T12:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.832+0000 7fa0bb7fe640 1 -- 172.21.15.137:0/2577618466 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c0004c80 con 0x55ed11260af0 2024-09-12T12:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.833+0000 7fa0bb7fe640 1 -- 172.21.15.137:0/2577618466 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c001c9e0 con 0x55ed11260af0 2024-09-12T12:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.833+0000 7fa0dae45740 1 -- 172.21.15.137:0/2577618466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 msgr2=0x55ed112694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.834 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.833+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed112694c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009d20 tx=0x7fa0c0010800 comp rx=0 tx=0).stop 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 -- 172.21.15.137:0/2577618466 shutdown_connections 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed1126bcb0 0x55ed11283270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed11261410 0x55ed11262c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2577618466 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed112694c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 -- 172.21.15.137:0/2577618466 >> 172.21.15.137:0/2577618466 conn(0x55ed1125d060 msgr2=0x55ed11262550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 -- 172.21.15.137:0/2577618466 shutdown_connections 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 -- 172.21.15.137:0/2577618466 wait complete. 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.834+0000 7fa0dae45740 1 -- start start 2024-09-12T12:22:02.835 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.835+0000 7fa0dae45740 1 Processor -- start 2024-09-12T12:22:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.835+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed11265dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.835+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11261410 0x55ed11266310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed11265dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0dae45740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1126bcb0 0x55ed11266850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0dae45740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01b2590 con 0x55ed1126bcb0 2024-09-12T12:22:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed11265dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:36408/0 (socket says 172.21.15.137:36408) 2024-09-12T12:22:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0dae45740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01b3a00 con 0x55ed11260af0 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d01b2c60 con 0x55ed11261410 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2499064241 learned_addr learned my addr 172.21.15.137:0/2499064241 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0dae45740 10 client.0 _ll_register_callbacks cb 0x7fa0cc009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5640640 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11261410 0x55ed11266310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d6642640 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1126bcb0 0x55ed11266850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2499064241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11261410 msgr2=0x55ed11266310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11261410 0x55ed11266310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2499064241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1126bcb0 msgr2=0x55ed11266850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1126bcb0 0x55ed11266850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.836+0000 7fa0d5e41640 1 -- 172.21.15.137:0/2499064241 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed112e53c0 con 0x55ed11260af0 2024-09-12T12:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.837+0000 7fa0d5e41640 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed11265dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004920 tx=0x7fa0c0010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ff6c75c856c4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.837+0000 7fa0b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.838+0000 7fa0b97fa640 1 -- 172.21.15.137:0/2499064241 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0004460 con 0x55ed11260af0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.838+0000 7fa0b97fa640 1 -- 172.21.15.137:0/2499064241 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8528+0+0 (secure 0 0 0) 0x7fa0c0005020 con 0x55ed11260af0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.838+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c0004c80 con 0x55ed11260af0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.838+0000 7fa0b97fa640 1 -- 172.21.15.137:0/2499064241 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa0c0037050 con 0x55ed11260af0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.838+0000 7fa0dae45740 10 client.25898 fetch_fsmap learned FSMap version 34 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.838+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ed112e66d0 con 0x55ed11260af0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.839+0000 7fa0b97fa640 1 -- 172.21.15.137:0/2499064241 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0c002b030 con 0x55ed11260af0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.839+0000 7fa0b97fa640 1 -- 172.21.15.137:0/2499064241 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7fa0c003a020 con 0x55ed11260af0 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.839+0000 7fa0dae45740 10 client.25898 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:22:02.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.839+0000 7fa0dae45740 10 client.25898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.839+0000 7fa0dae45740 20 client.25898 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.839+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed112e6d10 0x55ed112e9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.840+0000 7fa0dae45740 4 client.25898 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:22:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.840+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa0c0004c80 con 0x55ed112e6d10 2024-09-12T12:22:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.840+0000 7fa0d5640640 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed112e6d10 0x55ed112e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.840+0000 7fa0d5640640 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed112e6d10 0x55ed112e9130 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36b1e8dd599763ee server_cookie=cc3623f4bc0fe422 in_seq=0 out_seq=0 2024-09-12T12:22:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.840+0000 7fa0b97fa640 10 client.25898 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:02.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.932+0000 7fa0b97fa640 1 -- 172.21.15.137:0/2499064241 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa0c0004c80 con 0x55ed112e6d10 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.932+0000 7fa0b97fa640 10 client.25898 handle_command_reply: tid=0 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.932+0000 7fa0dae45740 1 client.25898 shutdown 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.932+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 msgr2=0x55ed11265dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.932+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed11265dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004920 tx=0x7fa0c0010f90 comp rx=0 tx=0).stop 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 shutdown_connections 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ed112e6d10 0x55ed112e9130 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0b97fa640 0 client.25898 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1126bcb0 0x55ed11266850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ed11261410 0x55ed11266310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 --2- 172.21.15.137:0/2499064241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed11260af0 0x55ed11265dd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 >> 172.21.15.137:0/2499064241 conn(0x55ed1125d060 msgr2=0x55ed11262ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 shutdown_connections 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 1 -- 172.21.15.137:0/2499064241 wait complete. 2024-09-12T12:22:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.933+0000 7fa0dae45740 20 client.25898 trim_cache size 0 max 16384 2024-09-12T12:22:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:22:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:22:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:22:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:22:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.934+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa0bc0a2ea0 msgr2=0x7fa0bc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.934+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa0bc0a2ea0 0x7fa0bc0a5360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa0c800ad40 tx=0x7fa0c800a4d0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.934+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d0101390 msgr2=0x7fa0d01a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.934+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d0101390 0x7fa0d01a4b90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc02a8f0 tx=0x7fa0cc0309a0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.935+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 shutdown_connections 2024-09-12T12:22:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.935+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa0bc0a2ea0 0x7fa0bc0a5360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.935+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0d0105ea0 0x7fa0d019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.935+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0d0101390 0x7fa0d01a4b90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.935+0000 7fa0d80a5640 1 --2- 172.21.15.137:0/1239597425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0100a70 0x7fa0d01a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.935+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 >> 172.21.15.137:0/1239597425 conn(0x7fa0d0077990 msgr2=0x7fa0d0103800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.936+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 shutdown_connections 2024-09-12T12:22:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:02.936+0000 7fa0d80a5640 1 -- 172.21.15.137:0/1239597425 wait complete. 2024-09-12T12:22:03.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:03 smithi189 ceph-mon[35475]: pgmap v274: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-12T12:22:03.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:03 smithi189 ceph-mon[35475]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T12:22:03.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:03 smithi171 ceph-mon[35098]: pgmap v274: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-12T12:22:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:03 smithi171 ceph-mon[35098]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T12:22:03.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 694791 7.77 MB/sec execute 338 sec latency 30.706 ms 2024-09-12T12:22:03.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:03 smithi137 ceph-mon[30950]: pgmap v274: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-12T12:22:03.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:03 smithi137 ceph-mon[30950]: osdmap e93: 12 total, 12 up, 12 in 2024-09-12T12:22:04.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 696642 7.78 MB/sec execute 339 sec latency 12.089 ms 2024-09-12T12:22:05.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:05 smithi189 ceph-mon[35475]: pgmap v276: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-09-12T12:22:05.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:05 smithi171 ceph-mon[35098]: pgmap v276: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-09-12T12:22:05.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 698643 7.79 MB/sec execute 340 sec latency 14.916 ms 2024-09-12T12:22:05.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:05 smithi137 ceph-mon[30950]: pgmap v276: 97 pgs: 97 active+clean; 212 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 286 op/s 2024-09-12T12:22:06.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 700268 7.80 MB/sec execute 341 sec latency 17.955 ms 2024-09-12T12:22:07.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:07 smithi189 ceph-mon[35475]: pgmap v277: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T12:22:07.600 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:07 smithi137 ceph-mon[30950]: pgmap v277: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T12:22:07.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:07 smithi171 ceph-mon[35098]: pgmap v277: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T12:22:07.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 702952 7.81 MB/sec execute 342 sec latency 16.545 ms 2024-09-12T12:22:08.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 704565 7.82 MB/sec execute 343 sec latency 16.128 ms 2024-09-12T12:22:08.870 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:08 smithi137 ceph-mon[30950]: pgmap v278: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T12:22:08.871 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:08 smithi137 ceph-mon[30950]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T12:22:08.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:08 smithi171 ceph-mon[35098]: pgmap v278: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T12:22:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:08 smithi171 ceph-mon[35098]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T12:22:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:08 smithi189 ceph-mon[35475]: pgmap v278: 97 pgs: 97 active+clean; 206 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-12T12:22:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:08 smithi189 ceph-mon[35475]: osdmap e94: 12 total, 12 up, 12 in 2024-09-12T12:22:09.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 706914 7.84 MB/sec execute 344 sec latency 14.226 ms 2024-09-12T12:22:10.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 708377 7.83 MB/sec execute 345 sec latency 14.153 ms 2024-09-12T12:22:10.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:10 smithi171 ceph-mon[35098]: pgmap v280: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-09-12T12:22:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:10 smithi137 ceph-mon[30950]: pgmap v280: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-09-12T12:22:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:10 smithi189 ceph-mon[35475]: pgmap v280: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 364 op/s 2024-09-12T12:22:11.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 710670 7.85 MB/sec execute 346 sec latency 10.687 ms 2024-09-12T12:22:12.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 712721 7.85 MB/sec execute 347 sec latency 72.825 ms 2024-09-12T12:22:12.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:12 smithi171 ceph-mon[35098]: pgmap v281: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-09-12T12:22:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:12 smithi137 ceph-mon[30950]: pgmap v281: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-09-12T12:22:13.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:12 smithi189 ceph-mon[35475]: pgmap v281: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 318 op/s 2024-09-12T12:22:13.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 714460 7.86 MB/sec execute 348 sec latency 42.123 ms 2024-09-12T12:22:14.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 716341 7.86 MB/sec execute 349 sec latency 15.569 ms 2024-09-12T12:22:14.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:14 smithi171 ceph-mon[35098]: pgmap v282: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2024-09-12T12:22:15.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:14 smithi137 ceph-mon[30950]: pgmap v282: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2024-09-12T12:22:15.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:14 smithi189 ceph-mon[35475]: pgmap v282: 97 pgs: 97 active+clean; 215 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 291 op/s 2024-09-12T12:22:15.743 INFO:tasks.workunit.client.0.smithi137.stdout: 1 718121 7.87 MB/sec execute 350 sec latency 12.703 ms 2024-09-12T12:22:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 -- 172.21.15.137:0/1302153206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b41035a0 msgr2=0x7f47b410ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 --2- 172.21.15.137:0/1302153206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b41035a0 0x7f47b410ba80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009900 tx=0x7f47a802a480 comp rx=0 tx=0).stop 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 -- 172.21.15.137:0/1302153206 shutdown_connections 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 --2- 172.21.15.137:0/1302153206 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b410c0d0 0x7f47b410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 --2- 172.21.15.137:0/1302153206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b41035a0 0x7f47b410ba80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 --2- 172.21.15.137:0/1302153206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b4102c80 0x7f47b4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 -- 172.21.15.137:0/1302153206 >> 172.21.15.137:0/1302153206 conn(0x7f47b4077990 msgr2=0x7f47b40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 -- 172.21.15.137:0/1302153206 shutdown_connections 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 -- 172.21.15.137:0/1302153206 wait complete. 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.028+0000 7f47bbaba640 1 Processor -- start 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47bbaba640 1 -- start start 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47bbaba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b4102c80 0x7f47b4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47bbaba640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b41035a0 0x7f47b41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47bbaba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b410c0d0 0x7f47b419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47bbaba640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4111eb0 con 0x7f47b410c0d0 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47bbaba640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4111d30 con 0x7f47b41035a0 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47bbaba640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47b4112030 con 0x7f47b4102c80 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b410c0d0 0x7f47b419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47b9856640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b4102c80 0x7f47b4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b410c0d0 0x7f47b419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37452/0 (socket says 172.21.15.137:37452) 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 -- 172.21.15.137:0/391883170 learned_addr learned my addr 172.21.15.137:0/391883170 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47b9055640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b41035a0 0x7f47b41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 -- 172.21.15.137:0/391883170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b4102c80 msgr2=0x7f47b4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b4102c80 0x7f47b4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 -- 172.21.15.137:0/391883170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b41035a0 msgr2=0x7f47b41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b41035a0 0x7f47b41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 -- 172.21.15.137:0/391883170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b419d350 con 0x7f47b410c0d0 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47b9856640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b4102c80 0x7f47b4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47ba057640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b410c0d0 0x7f47b419cc20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00e970 tx=0x7f47ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.029+0000 7f47b2ffd640 1 -- 172.21.15.137:0/391883170 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac00ccc0 con 0x7f47b410c0d0 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.030+0000 7f47b2ffd640 1 -- 172.21.15.137:0/391883170 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47ac00ce60 con 0x7f47b410c0d0 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.030+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b419d640 con 0x7f47b410c0d0 2024-09-12T12:22:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.030+0000 7f47b2ffd640 1 -- 172.21.15.137:0/391883170 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac010490 con 0x7f47b410c0d0 2024-09-12T12:22:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.030+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b41a4ed0 con 0x7f47b410c0d0 2024-09-12T12:22:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.032+0000 7f47b2ffd640 1 -- 172.21.15.137:0/391883170 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f47ac005020 con 0x7f47b410c0d0 2024-09-12T12:22:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.032+0000 7f47be857740 1 Processor -- start 2024-09-12T12:22:16.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47be857740 1 -- start start 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b2ffd640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f479c0a2e20 0x7f479c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47be857740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f706bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47be857740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f706c410 0x55c2f7076770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47be857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f7076cb0 0x55c2f70790a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47be857740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c2f6fd02f0 con 0x55c2f706baf0 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47be857740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c2f6f78270 con 0x55c2f706c410 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47be857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c2f6fcc2b0 con 0x55c2f7076cb0 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b9856640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f706bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b9055640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f479c0a2e20 0x7f479c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b2ffd640 1 -- 172.21.15.137:0/391883170 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f47ac0cf810 con 0x7f47b410c0d0 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b9856640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f706bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37468/0 (socket says 172.21.15.137:37468) 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47ba057640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f7076cb0 0x55c2f70790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b9856640 1 -- 172.21.15.137:0/3386528995 learned_addr learned my addr 172.21.15.137:0/3386528995 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b9055640 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f706c410 0x55c2f7076770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47ba057640 1 -- 172.21.15.137:0/3386528995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f706c410 msgr2=0x55c2f7076770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47ba057640 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f706c410 0x55c2f7076770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47ba057640 1 -- 172.21.15.137:0/3386528995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 msgr2=0x55c2f706bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47ba057640 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f706bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47ba057640 1 -- 172.21.15.137:0/3386528995 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f6f77340 con 0x55c2f7076cb0 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b9856640 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f706bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47b9055640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f479c0a2e20 0x7f479c0a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f47b410e4f0 tx=0x7f47a8032f60 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.033+0000 7f47ba057640 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f7076cb0 0x55c2f70790a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0ccf80 tx=0x7f47ac09b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca77088b6ff714e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f479bfff640 1 -- 172.21.15.137:0/3386528995 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac09bd50 con 0x55c2f7076cb0 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f479bfff640 1 -- 172.21.15.137:0/3386528995 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47ac0a0100 con 0x55c2f7076cb0 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f479bfff640 1 -- 172.21.15.137:0/3386528995 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac04a9c0 con 0x55c2f7076cb0 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f47be857740 1 -- 172.21.15.137:0/3386528995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f7076cb0 msgr2=0x55c2f70790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f47be857740 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f7076cb0 0x55c2f70790a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f47ac0ccf80 tx=0x7f47ac09b770 comp rx=0 tx=0).stop 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f47be857740 1 -- 172.21.15.137:0/3386528995 shutdown_connections 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f47be857740 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f7076cb0 0x55c2f70790a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f47be857740 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f706c410 0x55c2f7076770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f47be857740 1 --2- 172.21.15.137:0/3386528995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f706bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.034+0000 7f47be857740 1 -- 172.21.15.137:0/3386528995 >> 172.21.15.137:0/3386528995 conn(0x55c2f7068060 msgr2=0x55c2f7068440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 -- 172.21.15.137:0/3386528995 shutdown_connections 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 -- 172.21.15.137:0/3386528995 wait complete. 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 -- start start 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 Processor -- start 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f7073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f7074110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f7074650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c2f7079880 con 0x55c2f706c410 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c2f6f59290 con 0x55c2f7076cb0 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c2f7079a00 con 0x55c2f706baf0 2024-09-12T12:22:16.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47be857740 10 client.0 _ll_register_callbacks cb 0x55c2f6f8a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47b9055640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f7074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47b9856640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f7073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47b9055640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f7074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37480/0 (socket says 172.21.15.137:37480) 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47b9856640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f7073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:43380/0 (socket says 172.21.15.137:43380) 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47b9055640 1 -- 172.21.15.137:0/3609546227 learned_addr learned my addr 172.21.15.137:0/3609546227 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.035+0000 7f47ba057640 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f7074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47b9856640 1 -- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 msgr2=0x55c2f7074650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47b9856640 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f7074650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47b9856640 1 -- 172.21.15.137:0/3609546227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 msgr2=0x55c2f7074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47b9856640 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f7074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47b9856640 1 -- 172.21.15.137:0/3609546227 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f6ce3210 con 0x55c2f706baf0 2024-09-12T12:22:16.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47b9055640 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f7074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47b9856640 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f7073bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f47a4010e60 tx=0x7f47a4013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6f6e35c99cf17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47997fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47997fa640 1 -- 172.21.15.137:0/3609546227 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4003b70 con 0x55c2f706baf0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47997fa640 1 -- 172.21.15.137:0/3609546227 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f47a4003d10 con 0x55c2f706baf0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47ac0a0100 con 0x55c2f706baf0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47997fa640 1 -- 172.21.15.137:0/3609546227 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f47a40116b0 con 0x55c2f706baf0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47be857740 10 client.25975 fetch_fsmap learned FSMap version 34 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c2f6daf620 con 0x55c2f706baf0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.036+0000 7f47997fa640 1 -- 172.21.15.137:0/3609546227 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47ac0a0100 con 0x55c2f706baf0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47997fa640 1 -- 172.21.15.137:0/3609546227 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f47a4023020 con 0x55c2f706baf0 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47be857740 10 client.25975 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:22:16.046 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47be857740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47be857740 20 client.25975 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47be857740 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f7086ca0 0x55c2f70890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47be857740 4 client.25975 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f6fdda20 con 0x55c2f7086ca0 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47b9055640 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f7086ca0 0x55c2f70890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47b9055640 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f7086ca0 0x55c2f70890b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f63789499758c55 server_cookie=1bb844a34e63dfd9 in_seq=0 out_seq=0 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.037+0000 7f47997fa640 10 client.25975 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47997fa640 1 -- 172.21.15.137:0/3609546227 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2f6fdda20 con 0x55c2f7086ca0 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47997fa640 10 client.25975 handle_command_reply: tid=0 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 client.25975 shutdown 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 msgr2=0x55c2f7073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f7073bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f47a4010e60 tx=0x7f47a4013830 comp rx=0 tx=0).stop 2024-09-12T12:22:16.047 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 shutdown_connections 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f7074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f7086ca0 0x55c2f70890b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f7074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 --2- 172.21.15.137:0/3609546227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f7073bd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47997fa640 0 client.25975 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 >> 172.21.15.137:0/3609546227 conn(0x55c2f7068060 msgr2=0x55c2f706ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 shutdown_connections 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 1 -- 172.21.15.137:0/3609546227 wait complete. 2024-09-12T12:22:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.038+0000 7f47be857740 20 client.25975 trim_cache size 0 max 16384 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.052+0000 7f47be857740 1 Processor -- start 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.052+0000 7f47be857740 1 -- start start 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47be857740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f7074480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47be857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706c410 0x55c2f706dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47be857740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f708e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47be857740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41b0120 con 0x55c2f706baf0 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47be857740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41ad9c0 con 0x55c2f7076cb0 2024-09-12T12:22:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47be857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41ab830 con 0x55c2f706c410 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f7074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f7074480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37492/0 (socket says 172.21.15.137:37492) 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 -- 172.21.15.137:0/3834029197 learned_addr learned my addr 172.21.15.137:0/3834029197 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9055640 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706c410 0x55c2f706dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47ba057640 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f708e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 -- 172.21.15.137:0/3834029197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706c410 msgr2=0x55c2f706dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706c410 0x55c2f706dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 -- 172.21.15.137:0/3834029197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 msgr2=0x55c2f708e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f708e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 -- 172.21.15.137:0/3834029197 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f706d000 con 0x55c2f706baf0 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f47b9856640 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f7074480 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f47a4006010 tx=0x7f47a4009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd787de98d46453f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f479b7fe640 1 -- 172.21.15.137:0/3834029197 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40111b0 con 0x55c2f706baf0 2024-09-12T12:22:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f479b7fe640 1 -- 172.21.15.137:0/3834029197 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a4004d00 con 0x55c2f706baf0 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.053+0000 7f479b7fe640 1 -- 172.21.15.137:0/3834029197 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a401c610 con 0x55c2f706baf0 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 -- 172.21.15.137:0/3834029197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 msgr2=0x55c2f7074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f7074480 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f47a4006010 tx=0x7f47a4009f40 comp rx=0 tx=0).stop 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 -- 172.21.15.137:0/3834029197 shutdown_connections 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f708e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706c410 0x55c2f706dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 --2- 172.21.15.137:0/3834029197 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706baf0 0x55c2f7074480 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 -- 172.21.15.137:0/3834029197 >> 172.21.15.137:0/3834029197 conn(0x55c2f7068060 msgr2=0x55c2f706d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 -- 172.21.15.137:0/3834029197 shutdown_connections 2024-09-12T12:22:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 -- 172.21.15.137:0/3834029197 wait complete. 2024-09-12T12:22:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 -- start start 2024-09-12T12:22:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.054+0000 7f47be857740 1 Processor -- start 2024-09-12T12:22:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47be857740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f707a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47be857740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f707a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47be857740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f70e99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47be857740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41abe80 con 0x55c2f706c410 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47be857740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41ad2f0 con 0x55c2f7076cb0 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47be857740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47b41ac550 con 0x55c2f706baf0 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47be857740 10 client.0 _ll_register_callbacks cb 0x7f47a40104b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9055640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f707a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f707a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f707a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:43392/0 (socket says 172.21.15.137:43392) 2024-09-12T12:22:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 -- 172.21.15.137:0/2417798180 learned_addr learned my addr 172.21.15.137:0/2417798180 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9055640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f707a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37504/0 (socket says 172.21.15.137:37504) 2024-09-12T12:22:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47ba057640 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f70e99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 -- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 msgr2=0x55c2f70e99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f70e99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 -- 172.21.15.137:0/2417798180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 msgr2=0x55c2f707a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f707a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 -- 172.21.15.137:0/2417798180 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f70ea030 con 0x55c2f706baf0 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9055640 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f707a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47b9856640 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f707a400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47a4010b20 tx=0x7f47a4010900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97f3d7b3b3dfdd71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.055+0000 7f47997fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47997fa640 1 -- 172.21.15.137:0/2417798180 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40111f0 con 0x55c2f706baf0 2024-09-12T12:22:16.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47997fa640 1 -- 172.21.15.137:0/2417798180 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8484+0+0 (secure 0 0 0) 0x7f47a40047a0 con 0x55c2f706baf0 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a4004d00 con 0x55c2f706baf0 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47997fa640 1 -- 172.21.15.137:0/2417798180 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f47a403d050 con 0x55c2f706baf0 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 10 client.25981 fetch_fsmap learned FSMap version 34 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c2f70f13c0 con 0x55c2f706baf0 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47997fa640 1 -- 172.21.15.137:0/2417798180 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a4004d00 con 0x55c2f706baf0 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47997fa640 1 -- 172.21.15.137:0/2417798180 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 34) ==== 8350+0+0 (secure 0 0 0) 0x7f47a402d030 con 0x55c2f706baf0 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 10 client.25981 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 10 client.25981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 20 client.25981 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f70f1a20 0x55c2f70f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 4 client.25981 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:22:16.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2f6fd7d70 con 0x55c2f70f1a20 2024-09-12T12:22:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.056+0000 7f47b9055640 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f70f1a20 0x55c2f70f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.057+0000 7f47997fa640 10 client.25981 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.057+0000 7f47b9055640 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f70f1a20 0x55c2f70f3e10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4590c9bf33185c2a server_cookie=ceccc0008d732642 in_seq=0 out_seq=0 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47997fa640 1 -- 172.21.15.137:0/2417798180 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2f6fd7d70 con 0x55c2f70f1a20 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47997fa640 10 client.25981 handle_command_reply: tid=0 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 client.25981 shutdown 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 msgr2=0x55c2f707a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f707a400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47a4010b20 tx=0x7f47a4010900 comp rx=0 tx=0).stop 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 shutdown_connections 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c2f7076cb0 0x55c2f70e99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c2f70f1a20 0x55c2f70f3e10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c2f706c410 0x55c2f707a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.135 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 --2- 172.21.15.137:0/2417798180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c2f706baf0 0x55c2f707a400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 >> 172.21.15.137:0/2417798180 conn(0x55c2f7068060 msgr2=0x55c2f706d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47997fa640 0 client.25981 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 shutdown_connections 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 1 -- 172.21.15.137:0/2417798180 wait complete. 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.133+0000 7f47be857740 20 client.25981 trim_cache size 0 max 16384 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:22:16.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:22:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f479c0a2e20 msgr2=0x7f479c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f479c0a2e20 0x7f479c0a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f47b410e4f0 tx=0x7f47a8032f60 comp rx=0 tx=0).stop 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b410c0d0 msgr2=0x7f47b419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b410c0d0 0x7f47b419cc20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00e970 tx=0x7f47ac00ed70 comp rx=0 tx=0).stop 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 shutdown_connections 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f479c0a2e20 0x7f479c0a52e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b410c0d0 0x7f47b419cc20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b41035a0 0x7f47b41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 --2- 172.21.15.137:0/391883170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b4102c80 0x7f47b4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 >> 172.21.15.137:0/391883170 conn(0x7f47b4077990 msgr2=0x7f47b40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.134+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 shutdown_connections 2024-09-12T12:22:16.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:16.135+0000 7f47bbaba640 1 -- 172.21.15.137:0/391883170 wait complete. 2024-09-12T12:22:16.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:22:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:22:16.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:22:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:22:16.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:22:16 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/2043842250 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649082e5c20 con 0x5649082c3400 2024-09-12T12:22:16.591 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:22:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:22:16.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 719920 7.87 MB/sec execute 351 sec latency 35.615 ms 2024-09-12T12:22:16.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:16 smithi171 ceph-mon[35098]: pgmap v283: 97 pgs: 97 active+clean; 224 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 245 op/s 2024-09-12T12:22:16.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:16 smithi171 ceph-mon[35098]: daemon mds.f restarted 2024-09-12T12:22:17.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:16 smithi137 ceph-mon[30950]: pgmap v283: 97 pgs: 97 active+clean; 224 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 245 op/s 2024-09-12T12:22:17.051 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:16 smithi137 ceph-mon[30950]: daemon mds.f restarted 2024-09-12T12:22:17.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:16 smithi189 ceph-mon[35475]: pgmap v283: 97 pgs: 97 active+clean; 224 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 245 op/s 2024-09-12T12:22:17.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:16 smithi189 ceph-mon[35475]: daemon mds.f restarted 2024-09-12T12:22:17.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 721523 7.88 MB/sec execute 352 sec latency 35.340 ms 2024-09-12T12:22:17.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:17 smithi171 ceph-mon[35098]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T12:22:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:17 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/630156999,v1:172.21.15.189:6835/630156999] up:boot 2024-09-12T12:22:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:17 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:22:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:17 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:22:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:17 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:22:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:17 smithi137 ceph-mon[30950]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T12:22:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:17 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/630156999,v1:172.21.15.189:6835/630156999] up:boot 2024-09-12T12:22:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:17 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:22:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:17 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:22:18.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:17 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:22:18.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:17 smithi189 ceph-mon[35475]: osdmap e95: 12 total, 12 up, 12 in 2024-09-12T12:22:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:17 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/630156999,v1:172.21.15.189:6835/630156999] up:boot 2024-09-12T12:22:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:17 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:22:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:17 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:22:18.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:17 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:22:18.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 724192 7.90 MB/sec execute 353 sec latency 28.093 ms 2024-09-12T12:22:18.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:18 smithi171 ceph-mon[35098]: pgmap v285: 97 pgs: 97 active+clean; 224 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-12T12:22:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:18 smithi137 ceph-mon[30950]: pgmap v285: 97 pgs: 97 active+clean; 224 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-12T12:22:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:18 smithi189 ceph-mon[35475]: pgmap v285: 97 pgs: 97 active+clean; 224 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-12T12:22:19.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 725583 7.89 MB/sec execute 354 sec latency 23.190 ms 2024-09-12T12:22:20.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 727912 7.91 MB/sec execute 355 sec latency 14.793 ms 2024-09-12T12:22:20.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:20 smithi171 ceph-mon[35098]: pgmap v286: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:20 smithi137 ceph-mon[30950]: pgmap v286: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:20 smithi189 ceph-mon[35475]: pgmap v286: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:21.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 729335 7.91 MB/sec execute 356 sec latency 17.664 ms 2024-09-12T12:22:22.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 731436 7.92 MB/sec execute 357 sec latency 35.655 ms 2024-09-12T12:22:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:22 smithi171 ceph-mon[35098]: pgmap v287: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:22 smithi137 ceph-mon[30950]: pgmap v287: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:22 smithi189 ceph-mon[35475]: pgmap v287: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:23.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 732419 7.91 MB/sec execute 358 sec latency 114.437 ms 2024-09-12T12:22:23.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:23 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:22:24.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:23 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:22:24.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:23 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:22:24.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 735197 7.93 MB/sec execute 359 sec latency 15.386 ms 2024-09-12T12:22:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:24 smithi171 ceph-mon[35098]: pgmap v288: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:24 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:22:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:24 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:22:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:24 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:22:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:24 smithi137 ceph-mon[30950]: pgmap v288: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:24 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:22:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:24 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:22:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:24 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:22:25.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:24 smithi189 ceph-mon[35475]: pgmap v288: 97 pgs: 97 active+clean; 215 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 331 op/s 2024-09-12T12:22:25.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:24 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:22:25.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:24 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:22:25.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:24 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:22:25.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 736530 7.93 MB/sec execute 360 sec latency 26.148 ms 2024-09-12T12:22:26.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 738818 7.94 MB/sec execute 361 sec latency 12.629 ms 2024-09-12T12:22:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:26 smithi171 ceph-mon[35098]: pgmap v289: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-12T12:22:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:26 smithi137 ceph-mon[30950]: pgmap v289: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-12T12:22:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:26 smithi189 ceph-mon[35475]: pgmap v289: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 322 op/s 2024-09-12T12:22:27.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 740145 7.94 MB/sec execute 362 sec latency 42.501 ms 2024-09-12T12:22:28.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 742554 7.95 MB/sec execute 363 sec latency 16.286 ms 2024-09-12T12:22:28.892 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:28 smithi189 ceph-mon[35475]: pgmap v290: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 296 op/s 2024-09-12T12:22:28.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:28 smithi171 ceph-mon[35098]: pgmap v290: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 296 op/s 2024-09-12T12:22:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:28 smithi137 ceph-mon[30950]: pgmap v290: 97 pgs: 97 active+clean; 222 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 296 op/s 2024-09-12T12:22:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 -- 172.21.15.137:0/446448038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 msgr2=0x7f6e64107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 --2- 172.21.15.137:0/446448038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 0x7f6e64107f50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009ca0 tx=0x7f6e6002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 -- 172.21.15.137:0/446448038 shutdown_connections 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 --2- 172.21.15.137:0/446448038 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 0x7f6e6410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 --2- 172.21.15.137:0/446448038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e64101ad0 0x7f6e64101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 --2- 172.21.15.137:0/446448038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 0x7f6e64107f50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 -- 172.21.15.137:0/446448038 >> 172.21.15.137:0/446448038 conn(0x7f6e640fd7d0 msgr2=0x7f6e640ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 -- 172.21.15.137:0/446448038 shutdown_connections 2024-09-12T12:22:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.230+0000 7f6e6c711640 1 -- 172.21.15.137:0/446448038 wait complete. 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 Processor -- start 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 -- start start 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e64101ad0 0x7f6e64198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 0x7f6e64198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 0x7f6e6419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64111ee0 con 0x7f6e64102470 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64111d60 con 0x7f6e64107b70 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e6c711640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64112060 con 0x7f6e64101ad0 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e69cac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 0x7f6e64198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e69cac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 0x7f6e64198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58574/0 (socket says 172.21.15.137:58574) 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.231+0000 7f6e69cac640 1 -- 172.21.15.137:0/1991509893 learned_addr learned my addr 172.21.15.137:0/1991509893 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e64101ad0 0x7f6e64198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e6acae640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 0x7f6e6419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e69cac640 1 -- 172.21.15.137:0/1991509893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e64101ad0 msgr2=0x7f6e64198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e69cac640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e64101ad0 0x7f6e64198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e69cac640 1 -- 172.21.15.137:0/1991509893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 msgr2=0x7f6e6419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e69cac640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 0x7f6e6419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e69cac640 1 -- 172.21.15.137:0/1991509893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6419d310 con 0x7f6e64102470 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e64101ad0 0x7f6e64198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e69cac640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 0x7f6e64198860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400ea60 tx=0x7f6e5400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e5b7fe640 1 -- 172.21.15.137:0/1991509893 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400cde0 con 0x7f6e64102470 2024-09-12T12:22:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6419d600 con 0x7f6e64102470 2024-09-12T12:22:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e5b7fe640 1 -- 172.21.15.137:0/1991509893 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e540035f0 con 0x7f6e64102470 2024-09-12T12:22:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e5b7fe640 1 -- 172.21.15.137:0/1991509893 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54010670 con 0x7f6e64102470 2024-09-12T12:22:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.232+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e64074420 con 0x7f6e64102470 2024-09-12T12:22:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.236+0000 7f6e6f4ab740 1 Processor -- start 2024-09-12T12:22:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.236+0000 7f6e5b7fe640 1 -- 172.21.15.137:0/1991509893 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6e540037a0 con 0x7f6e64102470 2024-09-12T12:22:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.237+0000 7f6e5b7fe640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6e500a2ea0 0x7f6e500a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.238+0000 7f6e5b7fe640 1 -- 172.21.15.137:0/1991509893 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f6e540cf7f0 con 0x7f6e64102470 2024-09-12T12:22:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6f4ab740 1 -- start start 2024-09-12T12:22:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6acae640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6e500a2ea0 0x7f6e500a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563b4d11e2f0 con 0x563b4d1b9af0 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563b4d0c6270 con 0x563b4d1ba410 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b4d11a2b0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6acae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6acae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:35822/0 (socket says 172.21.15.137:35822) 2024-09-12T12:22:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e6acae640 1 -- 172.21.15.137:0/1920973147 learned_addr learned my addr 172.21.15.137:0/1920973147 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.239+0000 7f6e69cac640 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e6acae640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6e500a2ea0 0x7f6e500a5360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009fd0 tx=0x7f6e5c015850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e6acae640 1 -- 172.21.15.137:0/1920973147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 msgr2=0x563b4d1c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e6acae640 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e6acae640 1 -- 172.21.15.137:0/1920973147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 msgr2=0x563b4d1b9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e6acae640 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e6acae640 1 -- 172.21.15.137:0/1920973147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4d0c5340 con 0x563b4d1c4cb0 2024-09-12T12:22:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e6acae640 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c70a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c011740 tx=0x7f6e5c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2253aa7ebc95e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.240+0000 7f6e58ff9640 1 -- 172.21.15.137:0/1920973147 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c015230 con 0x563b4d1c4cb0 2024-09-12T12:22:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.241+0000 7f6e58ff9640 1 -- 172.21.15.137:0/1920973147 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e5c0035f0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.241+0000 7f6e58ff9640 1 -- 172.21.15.137:0/1920973147 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00f750 con 0x563b4d1c4cb0 2024-09-12T12:22:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/1920973147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 msgr2=0x563b4d1c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c70a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c011740 tx=0x7f6e5c01f000 comp rx=0 tx=0).stop 2024-09-12T12:22:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/1920973147 shutdown_connections 2024-09-12T12:22:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c70a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/1920973147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1b9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/1920973147 >> 172.21.15.137:0/1920973147 conn(0x563b4d1b6060 msgr2=0x563b4d1b6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/1920973147 shutdown_connections 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.242+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/1920973147 wait complete. 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 -- start start 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 Processor -- start 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563b4d1d8830 con 0x563b4d1b9af0 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563b4d0a7290 con 0x563b4d1ba410 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563b4d1d89b0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6f4ab740 10 client.0 _ll_register_callbacks cb 0x563b4d0d8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e69cac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e69cac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:58704/0 (socket says 172.21.15.137:58704) 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e6acae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.243+0000 7f6e69cac640 1 -- 172.21.15.137:0/2337465981 learned_addr learned my addr 172.21.15.137:0/2337465981 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e69cac640 1 -- 172.21.15.137:0/2337465981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 msgr2=0x563b4d1c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e69cac640 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e69cac640 1 -- 172.21.15.137:0/2337465981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 msgr2=0x563b4d1c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e69cac640 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e69cac640 1 -- 172.21.15.137:0/2337465981 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4ce31210 con 0x563b4d1ba410 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e69cac640 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c2180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400d010 tx=0x7f6e5409c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cb94224da83c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.244+0000 7f6e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.245+0000 7f6e4dffb640 1 -- 172.21.15.137:0/2337465981 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e540a4970 con 0x563b4d1ba410 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.245+0000 7f6e4dffb640 1 -- 172.21.15.137:0/2337465981 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f6e54098370 con 0x563b4d1ba410 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.245+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5c0035f0 con 0x563b4d1ba410 2024-09-12T12:22:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.245+0000 7f6e4dffb640 1 -- 172.21.15.137:0/2337465981 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e5407e020 con 0x563b4d1ba410 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.245+0000 7f6e6f4ab740 10 client.25934 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.245+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563b4cefd620 con 0x563b4d1ba410 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.245+0000 7f6e4dffb640 1 -- 172.21.15.137:0/2337465981 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e540b9070 con 0x563b4d1ba410 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.246+0000 7f6e4dffb640 1 -- 172.21.15.137:0/2337465981 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e5409b150 con 0x563b4d1ba410 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.246+0000 7f6e6f4ab740 10 client.25934 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.246+0000 7f6e6f4ab740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.246+0000 7f6e6f4ab740 20 client.25934 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.246+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d1ee4b0 0x563b4d1f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.246+0000 7f6e6f4ab740 4 client.25934 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.246+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6e5c0035f0 con 0x563b4d1ee4b0 2024-09-12T12:22:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.247+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d1ee4b0 0x563b4d1f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.247+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d1ee4b0 0x563b4d1f08a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed333897c6ff90 server_cookie=cc346aa9d9bfb87 in_seq=0 out_seq=0 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.247+0000 7f6e4dffb640 10 client.25934 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e4dffb640 1 -- 172.21.15.137:0/2337465981 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6e5c0035f0 con 0x563b4d1ee4b0 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e4dffb640 10 client.25934 handle_command_reply: tid=0 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 client.25934 shutdown 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 msgr2=0x563b4d1c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c2180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400d010 tx=0x7f6e5409c4c0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 shutdown_connections 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1c2180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d1ee4b0 0x563b4d1f08a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/2337465981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 >> 172.21.15.137:0/2337465981 conn(0x563b4d1b6060 msgr2=0x563b4d1bbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e4dffb640 0 client.25934 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.248+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 shutdown_connections 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.249+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/2337465981 wait complete. 2024-09-12T12:22:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.249+0000 7f6e6f4ab740 20 client.25934 trim_cache size 0 max 16384 2024-09-12T12:22:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.263+0000 7f6e6f4ab740 1 Processor -- start 2024-09-12T12:22:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6f4ab740 1 -- start start 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1f6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6e641a5700 con 0x563b4d1b9af0 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64072500 con 0x563b4d1ba410 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64070350 con 0x563b4d1c4cb0 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6a4ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.271 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e69cac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:35842/0 (socket says 172.21.15.137:35842) 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6a4ad640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58580/0 (socket says 172.21.15.137:58580) 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e69cac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1f6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:58710/0 (socket says 172.21.15.137:58710) 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 -- 172.21.15.137:0/343173232 learned_addr learned my addr 172.21.15.137:0/343173232 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 -- 172.21.15.137:0/343173232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 msgr2=0x563b4d1f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 -- 172.21.15.137:0/343173232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 msgr2=0x563b4d1c3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 -- 172.21.15.137:0/343173232 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4d1b71d0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.264+0000 7f6e6acae640 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1f6650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006010 tx=0x7f6e5c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d5c89fb175c5771 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e4ffff640 1 -- 172.21.15.137:0/343173232 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c009a90 con 0x563b4d1c4cb0 2024-09-12T12:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e4ffff640 1 -- 172.21.15.137:0/343173232 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e5c030070 con 0x563b4d1c4cb0 2024-09-12T12:22:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e4ffff640 1 -- 172.21.15.137:0/343173232 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c003a60 con 0x563b4d1c4cb0 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/343173232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 msgr2=0x563b4d1f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1f6650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006010 tx=0x7f6e5c009a20 comp rx=0 tx=0).stop 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/343173232 shutdown_connections 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1c4cb0 0x563b4d1f6650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1ba410 0x563b4d1f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/343173232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1b9af0 0x563b4d1c3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/343173232 >> 172.21.15.137:0/343173232 conn(0x563b4d1b6060 msgr2=0x563b4d1c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/343173232 shutdown_connections 2024-09-12T12:22:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.265+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/343173232 wait complete. 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 -- start start 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 Processor -- start 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1b9af0 0x563b4d1e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1ba410 0x563b4d1e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1c4cb0 0x563b4d1e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640709c0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64071e30 con 0x563b4d1b9af0 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e64071090 con 0x563b4d1ba410 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6f4ab740 10 client.0 _ll_register_callbacks cb 0x7f6e5409bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1c4cb0 0x563b4d1e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1c4cb0 0x563b4d1e34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58584/0 (socket says 172.21.15.137:58584) 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6a4ad640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1b9af0 0x563b4d1e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 -- 172.21.15.137:0/4199611253 learned_addr learned my addr 172.21.15.137:0/4199611253 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e69cac640 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1ba410 0x563b4d1e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 -- 172.21.15.137:0/4199611253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1ba410 msgr2=0x563b4d1e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1ba410 0x563b4d1e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 -- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1b9af0 msgr2=0x563b4d1e66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1b9af0 0x563b4d1e66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6acae640 1 -- 172.21.15.137:0/4199611253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4d1b6cf0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.266+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1b9af0 0x563b4d1e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6acae640 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1c4cb0 0x563b4d1e34a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c015d50 tx=0x7f6e5c009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52759a460429298 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e4dffb640 1 -- 172.21.15.137:0/4199611253 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c039060 con 0x563b4d1c4cb0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5c030070 con 0x563b4d1c4cb0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e4dffb640 1 -- 172.21.15.137:0/4199611253 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f6e5c0052d0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e4dffb640 1 -- 172.21.15.137:0/4199611253 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6e5c009a90 con 0x563b4d1c4cb0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 10 client.16560 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563b4d2599e0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e4dffb640 1 -- 172.21.15.137:0/4199611253 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e5c030070 con 0x563b4d1c4cb0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e4dffb640 1 -- 172.21.15.137:0/4199611253 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6e5c00f8f0 con 0x563b4d1c4cb0 2024-09-12T12:22:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 10 client.16560 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 20 client.16560 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d259e60 0x563b4d25c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 4 client.16560 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.267+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b4d12ba20 con 0x563b4d259e60 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.268+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d259e60 0x563b4d25c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.268+0000 7f6e4dffb640 10 client.16560 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.268+0000 7f6e6a4ad640 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d259e60 0x563b4d25c250 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f3c2c73523d358 server_cookie=32e3905d28a80961 in_seq=0 out_seq=0 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.333+0000 7f6e4dffb640 1 -- 172.21.15.137:0/4199611253 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b4d12ba20 con 0x563b4d259e60 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.333+0000 7f6e4dffb640 10 client.16560 handle_command_reply: tid=0 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.333+0000 7f6e6f4ab740 1 client.16560 shutdown 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1c4cb0 msgr2=0x563b4d1e34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1c4cb0 0x563b4d1e34a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c015d50 tx=0x7f6e5c009dd0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 shutdown_connections 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563b4d259e60 0x563b4d25c250 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563b4d1c4cb0 0x563b4d1e34a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b4d1ba410 0x563b4d1e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e4dffb640 0 client.16560 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 --2- 172.21.15.137:0/4199611253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563b4d1b9af0 0x563b4d1e66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 >> 172.21.15.137:0/4199611253 conn(0x563b4d1b6060 msgr2=0x563b4d1b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 shutdown_connections 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 1 -- 172.21.15.137:0/4199611253 wait complete. 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.334+0000 7f6e6f4ab740 20 client.16560 trim_cache size 0 max 16384 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:22:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:22:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:22:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6e500a2ea0 msgr2=0x7f6e500a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6e500a2ea0 0x7f6e500a5360 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009fd0 tx=0x7f6e5c015850 comp rx=0 tx=0).stop 2024-09-12T12:22:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 msgr2=0x7f6e64198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 0x7f6e64198860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400ea60 tx=0x7f6e5400ef20 comp rx=0 tx=0).stop 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 shutdown_connections 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6e500a2ea0 0x7f6e500a5360 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6e64107b70 0x7f6e6419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e64102470 0x7f6e64198860 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 --2- 172.21.15.137:0/1991509893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e64101ad0 0x7f6e64198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 >> 172.21.15.137:0/1991509893 conn(0x7f6e640fd7d0 msgr2=0x7f6e640fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 shutdown_connections 2024-09-12T12:22:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:29.335+0000 7f6e6c711640 1 -- 172.21.15.137:0/1991509893 wait complete. 2024-09-12T12:22:29.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 744608 7.95 MB/sec execute 364 sec latency 26.708 ms 2024-09-12T12:22:29.909 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:29 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:22:29.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:29 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:22:30.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:29 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:22:30.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 746416 7.96 MB/sec execute 365 sec latency 21.661 ms 2024-09-12T12:22:30.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:30 smithi171 ceph-mon[35098]: pgmap v291: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 423 op/s 2024-09-12T12:22:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:30 smithi137 ceph-mon[30950]: pgmap v291: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 423 op/s 2024-09-12T12:22:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:30 smithi189 ceph-mon[35475]: pgmap v291: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 423 op/s 2024-09-12T12:22:31.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 748278 7.96 MB/sec execute 366 sec latency 62.932 ms 2024-09-12T12:22:32.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 750000 7.97 MB/sec execute 367 sec latency 19.922 ms 2024-09-12T12:22:32.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:32 smithi171 ceph-mon[35098]: pgmap v292: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-12T12:22:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:32 smithi137 ceph-mon[30950]: pgmap v292: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-12T12:22:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:32 smithi189 ceph-mon[35475]: pgmap v292: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-12T12:22:33.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 752706 8.00 MB/sec execute 368 sec latency 39.031 ms 2024-09-12T12:22:34.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 753989 7.99 MB/sec execute 369 sec latency 21.468 ms 2024-09-12T12:22:34.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:34 smithi171 ceph-mon[35098]: pgmap v293: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-12T12:22:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:34 smithi137 ceph-mon[30950]: pgmap v293: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-12T12:22:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:34 smithi189 ceph-mon[35475]: pgmap v293: 97 pgs: 97 active+clean; 218 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 264 op/s 2024-09-12T12:22:35.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 756585 8.01 MB/sec execute 370 sec latency 15.349 ms 2024-09-12T12:22:36.744 INFO:tasks.workunit.client.0.smithi137.stdout: 1 757976 8.00 MB/sec execute 371 sec latency 24.930 ms 2024-09-12T12:22:36.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:36 smithi171 ceph-mon[35098]: pgmap v294: 97 pgs: 97 active+clean; 225 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 369 op/s 2024-09-12T12:22:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:36 smithi137 ceph-mon[30950]: pgmap v294: 97 pgs: 97 active+clean; 225 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 369 op/s 2024-09-12T12:22:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:36 smithi189 ceph-mon[35475]: pgmap v294: 97 pgs: 97 active+clean; 225 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 369 op/s 2024-09-12T12:22:37.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 760151 8.02 MB/sec execute 372 sec latency 44.304 ms 2024-09-12T12:22:38.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 761570 8.01 MB/sec execute 373 sec latency 32.706 ms 2024-09-12T12:22:38.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:38 smithi171 ceph-mon[35098]: pgmap v295: 97 pgs: 97 active+clean; 225 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2024-09-12T12:22:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:38 smithi137 ceph-mon[30950]: pgmap v295: 97 pgs: 97 active+clean; 225 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2024-09-12T12:22:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:38 smithi189 ceph-mon[35475]: pgmap v295: 97 pgs: 97 active+clean; 225 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 260 op/s 2024-09-12T12:22:39.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 763865 8.03 MB/sec execute 374 sec latency 11.709 ms 2024-09-12T12:22:40.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 766175 8.04 MB/sec execute 375 sec latency 19.156 ms 2024-09-12T12:22:40.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:40 smithi171 ceph-mon[35098]: pgmap v296: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 373 op/s 2024-09-12T12:22:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:40 smithi137 ceph-mon[30950]: pgmap v296: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 373 op/s 2024-09-12T12:22:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:40 smithi189 ceph-mon[35475]: pgmap v296: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 373 op/s 2024-09-12T12:22:41.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 767679 8.04 MB/sec execute 376 sec latency 22.522 ms 2024-09-12T12:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 -- 172.21.15.137:0/1915575560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 msgr2=0x7fea9c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 --2- 172.21.15.137:0/1915575560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c100e10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fea90009930 tx=0x7fea9002a420 comp rx=0 tx=0).stop 2024-09-12T12:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 -- 172.21.15.137:0/1915575560 shutdown_connections 2024-09-12T12:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 --2- 172.21.15.137:0/1915575560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9c101350 0x7fea9c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 --2- 172.21.15.137:0/1915575560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c100e10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.444 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 --2- 172.21.15.137:0/1915575560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea9c1069c0 0x7fea9c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 -- 172.21.15.137:0/1915575560 >> 172.21.15.137:0/1915575560 conn(0x7fea9c077990 msgr2=0x7fea9c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 -- 172.21.15.137:0/1915575560 shutdown_connections 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.430+0000 7feaa3142640 1 -- 172.21.15.137:0/1915575560 wait complete. 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 Processor -- start 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 -- start start 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea9c101350 0x7fea9c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9c1069c0 0x7fea9c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c110e00 con 0x7fea9c1009b0 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c110c80 con 0x7fea9c101350 2024-09-12T12:22:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa3142640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c110f80 con 0x7fea9c1069c0 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa0ede640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa0ede640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55486/0 (socket says 172.21.15.137:55486) 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa0ede640 1 -- 172.21.15.137:0/4061047626 learned_addr learned my addr 172.21.15.137:0/4061047626 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7fea9bfff640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea9c101350 0x7fea9c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.431+0000 7feaa16df640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9c1069c0 0x7fea9c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa0ede640 1 -- 172.21.15.137:0/4061047626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9c1069c0 msgr2=0x7fea9c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa0ede640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9c1069c0 0x7fea9c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa0ede640 1 -- 172.21.15.137:0/4061047626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea9c101350 msgr2=0x7fea9c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa0ede640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea9c101350 0x7fea9c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa0ede640 1 -- 172.21.15.137:0/4061047626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c19d320 con 0x7fea9c1009b0 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa0ede640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c198330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00e9f0 tx=0x7fea8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7fea99ffb640 1 -- 172.21.15.137:0/4061047626 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00cd80 con 0x7fea9c1009b0 2024-09-12T12:22:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7fea99ffb640 1 -- 172.21.15.137:0/4061047626 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c010910 con 0x7fea9c1009b0 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7fea99ffb640 1 -- 172.21.15.137:0/4061047626 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c018990 con 0x7fea9c1009b0 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c19d610 con 0x7fea9c1009b0 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.432+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1a4ea0 con 0x7fea9c1009b0 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7fea99ffb640 1 -- 172.21.15.137:0/4061047626 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fea8c010ac0 con 0x7fea9c1009b0 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 Processor -- start 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 -- start start 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c3bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c46770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5604d1ba02f0 con 0x5604d1c3baf0 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5604d1b48270 con 0x5604d1c3c410 2024-09-12T12:22:42.447 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.435+0000 7feaa5edb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5604d1b9c2b0 con 0x5604d1c46cb0 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7feaa16df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c3bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7feaa0ede640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c46770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea99ffb640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea840a2e20 0x7fea840a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:36178/0 (socket says 172.21.15.137:36178) 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 -- 172.21.15.137:0/2758253272 learned_addr learned my addr 172.21.15.137:0/2758253272 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7feaa0ede640 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c46770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:33342/0 (socket says 172.21.15.137:33342) 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7feaa16df640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea840a2e20 0x7fea840a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 -- 172.21.15.137:0/2758253272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 msgr2=0x5604d1c46770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c46770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 -- 172.21.15.137:0/2758253272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 msgr2=0x5604d1c3bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c3bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea99ffb640 1 -- 172.21.15.137:0/4061047626 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7fea8c0cf630 con 0x7fea9c1009b0 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 -- 172.21.15.137:0/2758253272 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d1b47340 con 0x5604d1c46cb0 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7feaa16df640 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c3bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea9bfff640 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c490a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a940 tx=0x7fea90005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8ddfc90e86cf5ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7feaa16df640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea840a2e20 0x7fea840a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea94005fd0 tx=0x7fea94004ab0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea82ffd640 1 -- 172.21.15.137:0/2758253272 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90040070 con 0x5604d1c46cb0 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea82ffd640 1 -- 172.21.15.137:0/2758253272 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea900318f0 con 0x5604d1c46cb0 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.436+0000 7fea82ffd640 1 -- 172.21.15.137:0/2758253272 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90035040 con 0x5604d1c46cb0 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 -- 172.21.15.137:0/2758253272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 msgr2=0x5604d1c490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c490a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a940 tx=0x7fea90005c00 comp rx=0 tx=0).stop 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 -- 172.21.15.137:0/2758253272 shutdown_connections 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c490a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c46770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 --2- 172.21.15.137:0/2758253272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c3bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 -- 172.21.15.137:0/2758253272 >> 172.21.15.137:0/2758253272 conn(0x5604d1c38060 msgr2=0x5604d1c38440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 -- 172.21.15.137:0/2758253272 shutdown_connections 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.437+0000 7feaa5edb740 1 -- 172.21.15.137:0/2758253272 wait complete. 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 -- start start 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 Processor -- start 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3baf0 0x5604d1c43b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3c410 0x5604d1c44070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5604d1c49880 con 0x5604d1c3c410 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5604d1b29290 con 0x5604d1c3baf0 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5604d1c49a00 con 0x5604d1c46cb0 2024-09-12T12:22:42.450 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa5edb740 10 client.0 _ll_register_callbacks cb 0x5604d1b5a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3c410 0x5604d1c44070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3c410 0x5604d1c44070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55502/0 (socket says 172.21.15.137:55502) 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 -- 172.21.15.137:0/39188701 learned_addr learned my addr 172.21.15.137:0/39188701 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa16df640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa0ede640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3baf0 0x5604d1c43b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 -- 172.21.15.137:0/39188701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 msgr2=0x5604d1c445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 -- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3baf0 msgr2=0x5604d1c43b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3baf0 0x5604d1c43b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7fea9bfff640 1 -- 172.21.15.137:0/39188701 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d18b3210 con 0x5604d1c3c410 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.438+0000 7feaa0ede640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3baf0 0x5604d1c43b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7fea9bfff640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3c410 0x5604d1c44070 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fea90000c00 tx=0x7fea900052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d83cd75fa6a89d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7fea7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7fea7bfff640 1 -- 172.21.15.137:0/39188701 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90040030 con 0x5604d1c3c410 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea900318f0 con 0x5604d1c3c410 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7fea7bfff640 1 -- 172.21.15.137:0/39188701 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7fea90004260 con 0x5604d1c3c410 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7fea7bfff640 1 -- 172.21.15.137:0/39188701 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea90056050 con 0x5604d1c3c410 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 10 client.16572 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5604d197f620 con 0x5604d1c3c410 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7fea7bfff640 1 -- 172.21.15.137:0/39188701 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea900318f0 con 0x5604d1c3c410 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7fea7bfff640 1 -- 172.21.15.137:0/39188701 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fea90003710 con 0x5604d1c3c410 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 10 client.16572 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 20 client.16572 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:42.452 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1c56a10 0x5604d1c58e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 4 client.16572 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.439+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604d1bada20 con 0x5604d1c56a10 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.440+0000 7feaa0ede640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1c56a10 0x5604d1c58e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.440+0000 7feaa0ede640 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1c56a10 0x5604d1c58e00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee60ffeb8b03433a server_cookie=218a3a046da576db in_seq=0 out_seq=0 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.440+0000 7fea7bfff640 10 client.16572 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7fea7bfff640 1 -- 172.21.15.137:0/39188701 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604d1bada20 con 0x5604d1c56a10 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7fea7bfff640 10 client.16572 handle_command_reply: tid=0 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 client.16572 shutdown 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3c410 msgr2=0x5604d1c44070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3c410 0x5604d1c44070 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fea90000c00 tx=0x7fea900052d0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 shutdown_connections 2024-09-12T12:22:42.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1c56a10 0x5604d1c58e00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3c410 0x5604d1c44070 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 --2- 172.21.15.137:0/39188701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3baf0 0x5604d1c43b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7fea7bfff640 0 client.16572 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 >> 172.21.15.137:0/39188701 conn(0x5604d1c38060 msgr2=0x5604d1c3ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 shutdown_connections 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 1 -- 172.21.15.137:0/39188701 wait complete. 2024-09-12T12:22:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.441+0000 7feaa5edb740 20 client.16572 trim_cache size 0 max 16384 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 Processor -- start 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 -- start start 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c5e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c1affd0 con 0x5604d1c3baf0 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c1ad840 con 0x5604d1c3c410 2024-09-12T12:22:42.462 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa5edb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c1ab690 con 0x5604d1c46cb0 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55508/0 (socket says 172.21.15.137:55508) 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 -- 172.21.15.137:0/3150434340 learned_addr learned my addr 172.21.15.137:0/3150434340 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa16df640 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c5e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7fea9bfff640 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 -- 172.21.15.137:0/3150434340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 msgr2=0x5604d1c5e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c5e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 -- 172.21.15.137:0/3150434340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 msgr2=0x5604d1c370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 -- 172.21.15.137:0/3150434340 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604d1c37e90 con 0x5604d1c3baf0 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7feaa0ede640 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c443e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fea8c09bb80 tx=0x7fea8c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5932a3dab2a654e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:42.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7fea827fc640 1 -- 172.21.15.137:0/3150434340 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c099ef0 con 0x5604d1c3baf0 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7fea827fc640 1 -- 172.21.15.137:0/3150434340 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c0ccff0 con 0x5604d1c3baf0 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.457+0000 7fea827fc640 1 -- 172.21.15.137:0/3150434340 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0d1af0 con 0x5604d1c3baf0 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 -- 172.21.15.137:0/3150434340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 msgr2=0x5604d1c443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c443e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fea8c09bb80 tx=0x7fea8c018420 comp rx=0 tx=0).stop 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 -- 172.21.15.137:0/3150434340 shutdown_connections 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c46cb0 0x5604d1c5e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 --2- 172.21.15.137:0/3150434340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c3baf0 0x5604d1c443e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 -- 172.21.15.137:0/3150434340 >> 172.21.15.137:0/3150434340 conn(0x5604d1c38060 msgr2=0x5604d1c369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 -- 172.21.15.137:0/3150434340 shutdown_connections 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 -- 172.21.15.137:0/3150434340 wait complete. 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 -- start start 2024-09-12T12:22:42.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.458+0000 7feaa5edb740 1 Processor -- start 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c3baf0 0x5604d1c4efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c4f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa5edb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c46cb0 0x5604d1c405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa5edb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c1abd00 con 0x5604d1c46cb0 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa5edb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c1ad170 con 0x5604d1c3c410 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa5edb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c1ac3d0 con 0x5604d1c3baf0 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa5edb740 10 client.0 _ll_register_callbacks cb 0x7fea8c09b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa16df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c46cb0 0x5604d1c405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c3baf0 0x5604d1c4efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c4f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c3baf0 0x5604d1c4efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:36214/0 (socket says 172.21.15.137:36214) 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa16df640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c46cb0 0x5604d1c405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55524/0 (socket says 172.21.15.137:55524) 2024-09-12T12:22:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 -- 172.21.15.137:0/40343088 learned_addr learned my addr 172.21.15.137:0/40343088 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 -- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 msgr2=0x5604d1c4f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c4f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 -- 172.21.15.137:0/40343088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c46cb0 msgr2=0x5604d1c405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c46cb0 0x5604d1c405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa0ede640 1 -- 172.21.15.137:0/40343088 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604d1c40db0 con 0x5604d1c3baf0 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7feaa16df640 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c46cb0 0x5604d1c405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.459+0000 7fea9bfff640 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c4f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa0ede640 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c3baf0 0x5604d1c4efd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0a4280 tx=0x7fea8c09ffb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f925b714519a3b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7fea7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7fea7bfff640 1 -- 172.21.15.137:0/40343088 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0b8020 con 0x5604d1c3baf0 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c0ccff0 con 0x5604d1c3baf0 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7fea7bfff640 1 -- 172.21.15.137:0/40343088 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7fea8c0ba2c0 con 0x5604d1c3baf0 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7fea7bfff640 1 -- 172.21.15.137:0/40343088 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea8c0ba4c0 con 0x5604d1c3baf0 2024-09-12T12:22:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 10 client.26023 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5604d1c4e920 con 0x5604d1c3baf0 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7fea7bfff640 1 -- 172.21.15.137:0/40343088 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8c0ccff0 con 0x5604d1c3baf0 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7fea7bfff640 1 -- 172.21.15.137:0/40343088 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fea8c010430 con 0x5604d1c3baf0 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 10 client.26023 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 10 client.26023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 20 client.26023 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1cc16a0 0x5604d1cc3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 4 client.26023 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604d1ba7d70 con 0x5604d1cc16a0 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.460+0000 7fea9bfff640 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1cc16a0 0x5604d1cc3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.461+0000 7fea7bfff640 10 client.26023 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.461+0000 7fea9bfff640 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1cc16a0 0x5604d1cc3ac0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c98b7948a16bf78 server_cookie=d87e7f55cfbd3f6b in_seq=0 out_seq=0 2024-09-12T12:22:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7fea7bfff640 1 -- 172.21.15.137:0/40343088 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604d1ba7d70 con 0x5604d1cc16a0 2024-09-12T12:22:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7fea7bfff640 10 client.26023 handle_command_reply: tid=0 2024-09-12T12:22:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 client.26023 shutdown 2024-09-12T12:22:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c3baf0 msgr2=0x5604d1c4efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c3baf0 0x5604d1c4efd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0a4280 tx=0x7fea8c09ffb0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 shutdown_connections 2024-09-12T12:22:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5604d1cc16a0 0x5604d1cc3ac0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5604d1c46cb0 0x5604d1c405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5604d1c3c410 0x5604d1c4f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 --2- 172.21.15.137:0/40343088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5604d1c3baf0 0x5604d1c4efd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 >> 172.21.15.137:0/40343088 conn(0x5604d1c38060 msgr2=0x5604d1c371c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7fea7bfff640 0 client.26023 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.529+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 shutdown_connections 2024-09-12T12:22:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.530+0000 7feaa5edb740 1 -- 172.21.15.137:0/40343088 wait complete. 2024-09-12T12:22:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.530+0000 7feaa5edb740 20 client.26023 trim_cache size 0 max 16384 2024-09-12T12:22:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:22:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:22:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:22:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:22:42.532 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea840a2e20 msgr2=0x7fea840a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea840a2e20 0x7fea840a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea94005fd0 tx=0x7fea94004ab0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 msgr2=0x7fea9c198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c198330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00e9f0 tx=0x7fea8c00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 shutdown_connections 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea840a2e20 0x7fea840a52e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9c1069c0 0x7fea9c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea9c101350 0x7fea9c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 --2- 172.21.15.137:0/4061047626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea9c1009b0 0x7fea9c198330 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.532+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 >> 172.21.15.137:0/4061047626 conn(0x7fea9c077990 msgr2=0x7fea9c069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.533+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 shutdown_connections 2024-09-12T12:22:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:42.533+0000 7feaa3142640 1 -- 172.21.15.137:0/4061047626 wait complete. 2024-09-12T12:22:42.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 769292 8.03 MB/sec execute 377 sec latency 37.006 ms 2024-09-12T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:42 smithi171 ceph-mon[35098]: pgmap v297: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T12:22:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:42 smithi137 ceph-mon[30950]: pgmap v297: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T12:22:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:42 smithi189 ceph-mon[35475]: pgmap v297: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T12:22:43.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 770942 8.04 MB/sec execute 378 sec latency 27.547 ms 2024-09-12T12:22:44.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 773070 8.04 MB/sec execute 379 sec latency 15.604 ms 2024-09-12T12:22:44.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:44 smithi171 ceph-mon[35098]: pgmap v298: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T12:22:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:44 smithi137 ceph-mon[30950]: pgmap v298: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T12:22:45.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:44 smithi189 ceph-mon[35475]: pgmap v298: 97 pgs: 97 active+clean; 233 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-12T12:22:45.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 774725 8.05 MB/sec execute 380 sec latency 31.563 ms 2024-09-12T12:22:46.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 777422 8.07 MB/sec execute 381 sec latency 18.877 ms 2024-09-12T12:22:46.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:46 smithi171 ceph-mon[35098]: pgmap v299: 97 pgs: 97 active+clean; 228 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 375 op/s 2024-09-12T12:22:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:46 smithi137 ceph-mon[30950]: pgmap v299: 97 pgs: 97 active+clean; 228 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 375 op/s 2024-09-12T12:22:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:46 smithi189 ceph-mon[35475]: pgmap v299: 97 pgs: 97 active+clean; 228 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 375 op/s 2024-09-12T12:22:47.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 778569 8.06 MB/sec execute 382 sec latency 19.240 ms 2024-09-12T12:22:48.755 INFO:tasks.workunit.client.0.smithi137.stdout: 1 781207 8.08 MB/sec execute 383 sec latency 44.277 ms 2024-09-12T12:22:48.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:48 smithi171 ceph-mon[35098]: pgmap v300: 97 pgs: 97 active+clean; 228 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-12T12:22:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:48 smithi137 ceph-mon[30950]: pgmap v300: 97 pgs: 97 active+clean; 228 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-12T12:22:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:48 smithi189 ceph-mon[35475]: pgmap v300: 97 pgs: 97 active+clean; 228 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-12T12:22:49.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 782587 8.08 MB/sec execute 384 sec latency 25.660 ms 2024-09-12T12:22:50.624 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T12:22:50.624 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T12:22:50.625 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:22:50.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 784754 8.09 MB/sec execute 385 sec latency 13.945 ms 2024-09-12T12:22:50.889 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:22:50.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:50 smithi171 ceph-mon[35098]: pgmap v301: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-09-12T12:22:51.014 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:50 smithi137 ceph-mon[30950]: pgmap v301: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-09-12T12:22:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:50 smithi189 ceph-mon[35475]: pgmap v301: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.462+0000 7f257994b640 1 -- 172.21.15.137:0/1223894951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 msgr2=0x7f257410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.462+0000 7f257994b640 1 --2- 172.21.15.137:0/1223894951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 0x7f257410bb60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2568009900 tx=0x7f256802a480 comp rx=0 tx=0).stop 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 -- 172.21.15.137:0/1223894951 shutdown_connections 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 --2- 172.21.15.137:0/1223894951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f257410c180 0x7f257410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 --2- 172.21.15.137:0/1223894951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 0x7f257410bb60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 --2- 172.21.15.137:0/1223894951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2574102d50 0x7f2574103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 -- 172.21.15.137:0/1223894951 >> 172.21.15.137:0/1223894951 conn(0x7f25740fc7d0 msgr2=0x7f25740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:51.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 -- 172.21.15.137:0/1223894951 shutdown_connections 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 -- 172.21.15.137:0/1223894951 wait complete. 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 Processor -- start 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.463+0000 7f257994b640 1 -- start start 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f257994b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2574102d50 0x7f2574198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f257994b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 0x7f2574198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f257994b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f257410c180 0x7f257419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f257994b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2574111f80 con 0x7f2574103670 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f257994b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2574111e00 con 0x7f257410c180 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f257994b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2574112100 con 0x7f2574102d50 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f2572ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2574102d50 0x7f2574198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f25737fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f257410c180 0x7f257419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f2572ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2574102d50 0x7f2574198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33334/0 (socket says 172.21.15.137:33334) 2024-09-12T12:22:51.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f2572ffd640 1 -- 172.21.15.137:0/1060840071 learned_addr learned my addr 172.21.15.137:0/1060840071 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f25727fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 0x7f2574198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f25737fe640 1 -- 172.21.15.137:0/1060840071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2574102d50 msgr2=0x7f2574198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f25737fe640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2574102d50 0x7f2574198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f25737fe640 1 -- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 msgr2=0x7f2574198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f25737fe640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 0x7f2574198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f25737fe640 1 -- 172.21.15.137:0/1060840071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257419d4b0 con 0x7f257410c180 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.464+0000 7f2572ffd640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2574102d50 0x7f2574198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.465+0000 7f25737fe640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f257410c180 0x7f257419cdd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f256c009fd0 tx=0x7f256c00dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.465+0000 7f2578949640 1 -- 172.21.15.137:0/1060840071 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c00bda0 con 0x7f257410c180 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.465+0000 7f2578949640 1 -- 172.21.15.137:0/1060840071 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f256c005020 con 0x7f257410c180 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.465+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257419d7a0 con 0x7f257410c180 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.465+0000 7f2578949640 1 -- 172.21.15.137:0/1060840071 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c0106c0 con 0x7f257410c180 2024-09-12T12:22:51.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.465+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25741008e0 con 0x7f257410c180 2024-09-12T12:22:51.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.466+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2574103130 con 0x7f257410c180 2024-09-12T12:22:51.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.468+0000 7f2578949640 1 -- 172.21.15.137:0/1060840071 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f256c005540 con 0x7f257410c180 2024-09-12T12:22:51.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.468+0000 7f2578949640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f255c0a2d70 0x7f255c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:51.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.468+0000 7f2572ffd640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f255c0a2d70 0x7f255c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:51.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.468+0000 7f2578949640 1 -- 172.21.15.137:0/1060840071 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f256c0cfa40 con 0x7f257410c180 2024-09-12T12:22:51.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.469+0000 7f2572ffd640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f255c0a2d70 0x7f255c0a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2564009c00 tx=0x7f2564005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:51.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.471+0000 7f2578949640 1 -- 172.21.15.137:0/1060840071 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f256c09c690 con 0x7f257410c180 2024-09-12T12:22:51.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.708+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2574199860 con 0x7f257410c180 2024-09-12T12:22:51.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.709+0000 7f2578949640 1 -- 172.21.15.137:0/1060840071 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11936 (secure 0 0 0) 0x7f2574199860 con 0x7f257410c180 2024-09-12T12:22:51.709 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:22:51.709 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":37,"btime":"2024-09-12T12:22:28:627072+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:22:28.158799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:22:51.709 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 37 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f255c0a2d70 msgr2=0x7f255c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f255c0a2d70 0x7f255c0a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2564009c00 tx=0x7f2564005f90 comp rx=0 tx=0).stop 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f257410c180 msgr2=0x7f257419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f257410c180 0x7f257419cdd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f256c009fd0 tx=0x7f256c00dea0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 shutdown_connections 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f255c0a2d70 0x7f255c0a5230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f257410c180 0x7f257419cdd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2574103670 0x7f2574198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 --2- 172.21.15.137:0/1060840071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2574102d50 0x7f2574198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 >> 172.21.15.137:0/1060840071 conn(0x7f25740fc7d0 msgr2=0x7f25740fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 shutdown_connections 2024-09-12T12:22:51.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:51.711+0000 7f257994b640 1 -- 172.21.15.137:0/1060840071 wait complete. 2024-09-12T12:22:51.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 785888 8.08 MB/sec execute 386 sec latency 29.226 ms 2024-09-12T12:22:52.478 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T12:22:52.478 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T12:22:52.743 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:22:52.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788467 8.10 MB/sec execute 387 sec latency 16.863 ms 2024-09-12T12:22:52.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:52 smithi137 ceph-mon[30950]: pgmap v302: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-12T12:22:52.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:52 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1060840071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:22:52.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:52 smithi171 ceph-mon[35098]: pgmap v302: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-12T12:22:52.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:52 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1060840071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:22:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:52 smithi189 ceph-mon[35475]: pgmap v302: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-12T12:22:53.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:52 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1060840071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:22:53.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f0455ffb640 1 -- 172.21.15.137:0/1882521703 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450030d30 con 0x7f045811b140 2024-09-12T12:22:53.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 -- 172.21.15.137:0/1882521703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045811b140 msgr2=0x7f045811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 --2- 172.21.15.137:0/1882521703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045811b140 0x7f045811d530 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0450009ca0 tx=0x7f045002a8b0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 -- 172.21.15.137:0/1882521703 shutdown_connections 2024-09-12T12:22:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 --2- 172.21.15.137:0/1882521703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045811b140 0x7f045811d530 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 --2- 172.21.15.137:0/1882521703 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0458109830 0x7f0458109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 --2- 172.21.15.137:0/1882521703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0458109450 0x7f0458110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 -- 172.21.15.137:0/1882521703 >> 172.21.15.137:0/1882521703 conn(0x7f045806e140 msgr2=0x7f045806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.281+0000 7f045d848640 1 -- 172.21.15.137:0/1882521703 shutdown_connections 2024-09-12T12:22:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.282+0000 7f045d848640 1 -- 172.21.15.137:0/1882521703 wait complete. 2024-09-12T12:22:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.282+0000 7f045d848640 1 Processor -- start 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.282+0000 7f045d848640 1 -- start start 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f045d848640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0458109450 0x7f04581a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f045d848640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0458109830 0x7f04581a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f045d848640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045811b140 0x7f04581aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f045d848640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0458120df0 con 0x7f0458109830 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f045d848640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0458120c70 con 0x7f045811b140 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f045d848640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0458120f70 con 0x7f0458109450 2024-09-12T12:22:53.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0458109830 0x7f04581a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0458109450 0x7f04581a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045811b140 0x7f04581aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0458109450 0x7f04581a71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33352/0 (socket says 172.21.15.137:33352) 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045811b140 0x7f04581aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:42170/0 (socket says 172.21.15.137:42170) 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f0456ffd640 1 -- 172.21.15.137:0/2911951617 learned_addr learned my addr 172.21.15.137:0/2911951617 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 -- 172.21.15.137:0/2911951617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0458109450 msgr2=0x7f04581a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0458109450 0x7f04581a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 -- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0458109830 msgr2=0x7f04581a76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0458109830 0x7f04581a76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 -- 172.21.15.137:0/2911951617 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581ac150 con 0x7f045811b140 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f0456ffd640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0458109450 0x7f04581a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f04577fe640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045811b140 0x7f04581aba70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0450009ca0 tx=0x7f0450030b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.283+0000 7f045c846640 1 -- 172.21.15.137:0/2911951617 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450003b80 con 0x7f045811b140 2024-09-12T12:22:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.284+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04581ac3e0 con 0x7f045811b140 2024-09-12T12:22:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.284+0000 7f045c846640 1 -- 172.21.15.137:0/2911951617 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0450040070 con 0x7f045811b140 2024-09-12T12:22:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.284+0000 7f045c846640 1 -- 172.21.15.137:0/2911951617 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450004380 con 0x7f045811b140 2024-09-12T12:22:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.284+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581b3d20 con 0x7f045811b140 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.286+0000 7f045c846640 1 -- 172.21.15.137:0/2911951617 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0450004520 con 0x7f045811b140 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.286+0000 7f045fc21740 1 Processor -- start 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.286+0000 7f045fc21740 1 -- start start 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045c846640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04400a2d70 0x7f04400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045fc21740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045fc21740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045fc21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045fc21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a69f1f160 con 0x559a69fc5a30 2024-09-12T12:22:53.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045fc21740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559a69ec6720 con 0x559a69fba850 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045fc21740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a69f1b120 con 0x559a69fbb170 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04400a2d70 0x7f04400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f045c846640 1 -- 172.21.15.137:0/2911951617 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f04500f0240 con 0x7f045811b140 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33360/0 (socket says 172.21.15.137:33360) 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 -- 172.21.15.137:0/2199119980 learned_addr learned my addr 172.21.15.137:0/2199119980 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f0456ffd640 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04577fe640 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 -- 172.21.15.137:0/2199119980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 msgr2=0x559a69fbac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 -- 172.21.15.137:0/2199119980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 msgr2=0x559a69fc7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 -- 172.21.15.137:0/2199119980 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a69c90ba0 con 0x559a69fbb170 2024-09-12T12:22:53.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04577fe640 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04400a2d70 0x7f04400a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f044c013950 tx=0x7f044c009af0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.287+0000 7f04567fc640 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc54f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f044c009720 tx=0x7f044c01d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd1056b48355bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f043dffb640 1 -- 172.21.15.137:0/2199119980 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00fe50 con 0x559a69fbb170 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f043dffb640 1 -- 172.21.15.137:0/2199119980 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044c009890 con 0x559a69fbb170 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f043dffb640 1 -- 172.21.15.137:0/2199119980 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00ea10 con 0x559a69fbb170 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 -- 172.21.15.137:0/2199119980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 msgr2=0x559a69fc54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc54f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f044c009720 tx=0x7f044c01d000 comp rx=0 tx=0).stop 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 -- 172.21.15.137:0/2199119980 shutdown_connections 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc54f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 --2- 172.21.15.137:0/2199119980 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 -- 172.21.15.137:0/2199119980 >> 172.21.15.137:0/2199119980 conn(0x559a69fb6de0 msgr2=0x559a69fb71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.288+0000 7f045fc21740 1 -- 172.21.15.137:0/2199119980 shutdown_connections 2024-09-12T12:22:53.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.289+0000 7f045fc21740 1 -- 172.21.15.137:0/2199119980 wait complete. 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.289+0000 7f045fc21740 1 -- start start 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.289+0000 7f045fc21740 1 Processor -- start 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.289+0000 7f045fc21740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fc2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.289+0000 7f045fc21740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.289+0000 7f045fc21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f045fc21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559a69fc8600 con 0x559a69fc5a30 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f045fc21740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559a69ea7900 con 0x559a69fba850 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f045fc21740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559a69fc8780 con 0x559a69fbb170 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04577fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33362/0 (socket says 172.21.15.137:33362) 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 -- 172.21.15.137:0/3220802620 learned_addr learned my addr 172.21.15.137:0/3220802620 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:53.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f045fc21740 10 client.0 _ll_register_callbacks cb 0x559a69ea5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f0456ffd640 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fc2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 -- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 msgr2=0x559a69fc2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fc2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 -- 172.21.15.137:0/3220802620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 msgr2=0x559a69fc3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 -- 172.21.15.137:0/3220802620 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a69ee7c50 con 0x559a69fbb170 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f04567fc640 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc2e40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f044c00f920 tx=0x7f044c00fe50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b170b2b7f60ca1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f0436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f0436ffd640 1 -- 172.21.15.137:0/3220802620 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c026040 con 0x559a69fbb170 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f044c009890 con 0x559a69fbb170 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.290+0000 7f0436ffd640 1 -- 172.21.15.137:0/3220802620 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f044c01b070 con 0x559a69fbb170 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f0436ffd640 1 -- 172.21.15.137:0/3220802620 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f044c005570 con 0x559a69fbb170 2024-09-12T12:22:53.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 10 client.26041 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559a69fd4250 con 0x559a69fbb170 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f0436ffd640 1 -- 172.21.15.137:0/3220802620 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044c009890 con 0x559a69fbb170 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f0436ffd640 1 -- 172.21.15.137:0/3220802620 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f044c029070 con 0x559a69fbb170 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 10 client.26041 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 10 client.26041 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 20 client.26041 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a69fd55a0 0x559a69fd7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 4 client.26041 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a69f2c890 con 0x559a69fd55a0 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.291+0000 7f0456ffd640 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a69fd55a0 0x559a69fd7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.292+0000 7f0456ffd640 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a69fd55a0 0x559a69fd7990 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67148cf8501559aa server_cookie=6aee57755fa81e65 in_seq=0 out_seq=0 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.292+0000 7f0436ffd640 10 client.26041 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:53.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.292+0000 7f0436ffd640 1 -- 172.21.15.137:0/3220802620 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a69f2c890 con 0x559a69fd55a0 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.292+0000 7f0436ffd640 10 client.26041 handle_command_reply: tid=0 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.292+0000 7f045fc21740 1 client.26041 shutdown 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 msgr2=0x559a69fc2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc2e40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f044c00f920 tx=0x7f044c00fe50 comp rx=0 tx=0).stop 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 shutdown_connections 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a69fd55a0 0x559a69fd7990 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fc5a30 0x559a69fc3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fbb170 0x559a69fc2e40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 --2- 172.21.15.137:0/3220802620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fc2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 >> 172.21.15.137:0/3220802620 conn(0x559a69fb6de0 msgr2=0x559a69fbcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f0436ffd640 0 client.26041 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 shutdown_connections 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 1 -- 172.21.15.137:0/3220802620 wait complete. 2024-09-12T12:22:53.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.293+0000 7f045fc21740 20 client.26041 trim_cache size 0 max 16384 2024-09-12T12:22:53.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 Processor -- start 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 -- start start 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fc4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fc5a30 0x559a69fdd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04581bedd0 con 0x559a69fbb170 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04581bc640 con 0x559a69fba850 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.311+0000 7f045fc21740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04581ba4b0 con 0x559a69fc5a30 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fc4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fc4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40638/0 (socket says 172.21.15.137:40638) 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 -- 172.21.15.137:0/3240498996 learned_addr learned my addr 172.21.15.137:0/3240498996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f0456ffd640 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04577fe640 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fc5a30 0x559a69fdd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 -- 172.21.15.137:0/3240498996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fc5a30 msgr2=0x559a69fdd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fc5a30 0x559a69fdd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 -- 172.21.15.137:0/3240498996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 msgr2=0x559a69fbd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 -- 172.21.15.137:0/3240498996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a69fbafa0 con 0x559a69fbb170 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f0456ffd640 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f04567fc640 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fc4870 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f044c00eb80 tx=0x7f044c005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e24f53c202f46c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f043d7fa640 1 -- 172.21.15.137:0/3240498996 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c00e780 con 0x559a69fbb170 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f043d7fa640 1 -- 172.21.15.137:0/3240498996 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044c0097a0 con 0x559a69fbb170 2024-09-12T12:22:53.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.312+0000 7f043d7fa640 1 -- 172.21.15.137:0/3240498996 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c013170 con 0x559a69fbb170 2024-09-12T12:22:53.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 -- 172.21.15.137:0/3240498996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 msgr2=0x559a69fc4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fc4870 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f044c00eb80 tx=0x7f044c005360 comp rx=0 tx=0).stop 2024-09-12T12:22:53.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 -- 172.21.15.137:0/3240498996 shutdown_connections 2024-09-12T12:22:53.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fc5a30 0x559a69fdd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fc4870 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 --2- 172.21.15.137:0/3240498996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fba850 0x559a69fbd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 -- 172.21.15.137:0/3240498996 >> 172.21.15.137:0/3240498996 conn(0x559a69fb6de0 msgr2=0x559a69fc4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 -- 172.21.15.137:0/3240498996 shutdown_connections 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 -- 172.21.15.137:0/3240498996 wait complete. 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.313+0000 7f045fc21740 1 -- start start 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 1 Processor -- start 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fba850 0x559a69fcdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fc5a30 0x559a69fc90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f04581bab00 con 0x559a69fbb170 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04581bbf70 con 0x559a69fc5a30 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04581bb1d0 con 0x559a69fba850 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f045fc21740 10 client.0 _ll_register_callbacks cb 0x7f0448009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f04567fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f04577fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fc5a30 0x559a69fc90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fba850 0x559a69fcdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f04577fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fc5a30 0x559a69fc90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:42206/0 (socket says 172.21.15.137:42206) 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f04577fe640 1 -- 172.21.15.137:0/1456951101 learned_addr learned my addr 172.21.15.137:0/1456951101 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fba850 0x559a69fcdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33386/0 (socket says 172.21.15.137:33386) 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f04567fc640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40648/0 (socket says 172.21.15.137:40648) 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f0456ffd640 1 -- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fc5a30 msgr2=0x559a69fc90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f0456ffd640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fc5a30 0x559a69fc90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f0456ffd640 1 -- 172.21.15.137:0/1456951101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 msgr2=0x559a69fce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f0456ffd640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f0456ffd640 1 -- 172.21.15.137:0/1456951101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a69fc98d0 con 0x559a69fba850 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f04567fc640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.314+0000 7f04577fe640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fc5a30 0x559a69fc90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f0456ffd640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fba850 0x559a69fcdce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0448005d00 tx=0x7f04480035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f1008d9ef2ae100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:53.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f0436ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f0436ffd640 1 -- 172.21.15.137:0/1456951101 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448027650 con 0x559a69fba850 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f0436ffd640 1 -- 172.21.15.137:0/1456951101 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f04480277f0 con 0x559a69fba850 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f044c0097a0 con 0x559a69fba850 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f0436ffd640 1 -- 172.21.15.137:0/1456951101 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f04480279f0 con 0x559a69fba850 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f045fc21740 10 client.26047 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559a6a03fc20 con 0x559a69fba850 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f0436ffd640 1 -- 172.21.15.137:0/1456951101 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0448027b90 con 0x559a69fba850 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f0436ffd640 1 -- 172.21.15.137:0/1456951101 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f0448012450 con 0x559a69fba850 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f045fc21740 10 client.26047 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f045fc21740 10 client.26047 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.315+0000 7f045fc21740 20 client.26047 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.316+0000 7f045fc21740 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a6a040310 0x559a6a042700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.316+0000 7f045fc21740 4 client.26047 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T12:22:53.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.316+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x7f044c0097a0 con 0x559a6a040310 2024-09-12T12:22:53.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.316+0000 7f04567fc640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a6a040310 0x559a6a042700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:53.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.316+0000 7f0436ffd640 10 client.26047 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:53.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.316+0000 7f04567fc640 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a6a040310 0x559a6a042700 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e664bebc26df31 server_cookie=5e60f77666aa9329 in_seq=0 out_seq=0 2024-09-12T12:22:53.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f0436ffd640 1 -- 172.21.15.137:0/1456951101 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f044c0097a0 con 0x559a6a040310 2024-09-12T12:22:53.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f0436ffd640 10 client.26047 handle_command_reply: tid=0 2024-09-12T12:22:53.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 client.26047 shutdown 2024-09-12T12:22:53.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fba850 msgr2=0x559a69fcdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fba850 0x559a69fcdce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0448005d00 tx=0x7f04480035f0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 shutdown_connections 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559a69fc5a30 0x559a69fc90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559a6a040310 0x559a6a042700 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559a69fbb170 0x559a69fce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 --2- 172.21.15.137:0/1456951101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a69fba850 0x559a69fcdce0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f0436ffd640 0 client.26047 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.379+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 >> 172.21.15.137:0/1456951101 conn(0x559a69fb6de0 msgr2=0x559a69fc30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 shutdown_connections 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045fc21740 1 -- 172.21.15.137:0/1456951101 wait complete. 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045fc21740 20 client.26047 trim_cache size 0 max 16384 2024-09-12T12:22:53.380 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 37, 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 38, 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "9aaa6358": { 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 38, 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:22:53.381 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 142.6, 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:22:53.382 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:22:53.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04400a2d70 msgr2=0x7f04400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04400a2d70 0x7f04400a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f044c013950 tx=0x7f044c009af0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045811b140 msgr2=0x7f04581aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:53.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045811b140 0x7f04581aba70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0450009ca0 tx=0x7f0450030b60 comp rx=0 tx=0).stop 2024-09-12T12:22:53.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 shutdown_connections 2024-09-12T12:22:53.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04400a2d70 0x7f04400a5230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045811b140 0x7f04581aba70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0458109830 0x7f04581a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 --2- 172.21.15.137:0/2911951617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0458109450 0x7f04581a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:53.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 >> 172.21.15.137:0/2911951617 conn(0x7f045806e140 msgr2=0x7f045811c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:53.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 shutdown_connections 2024-09-12T12:22:53.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:22:53.380+0000 7f045d848640 1 -- 172.21.15.137:0/2911951617 wait complete. 2024-09-12T12:22:53.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 8.08 MB/sec execute 388 sec latency 402.201 ms 2024-09-12T12:22:54.101 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9aaa6358', quiesce took 3.5 seconds. Will release after: 19.05 2024-09-12T12:22:54.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 8.06 MB/sec execute 389 sec latency 1402.288 ms 2024-09-12T12:22:54.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:54 smithi171 ceph-mon[35098]: pgmap v303: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-12T12:22:55.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:54 smithi137 ceph-mon[30950]: pgmap v303: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-12T12:22:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:54 smithi189 ceph-mon[35475]: pgmap v303: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 266 op/s 2024-09-12T12:22:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 -- 172.21.15.137:0/251138228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c01035e0 msgr2=0x7f56c010bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 --2- 172.21.15.137:0/251138228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c01035e0 0x7f56c010bac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009900 tx=0x7f56b402a480 comp rx=0 tx=0).stop 2024-09-12T12:22:55.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 -- 172.21.15.137:0/251138228 shutdown_connections 2024-09-12T12:22:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 --2- 172.21.15.137:0/251138228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56c010c110 0x7f56c010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 --2- 172.21.15.137:0/251138228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c01035e0 0x7f56c010bac0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 --2- 172.21.15.137:0/251138228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0102cc0 0x7f56c01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 -- 172.21.15.137:0/251138228 >> 172.21.15.137:0/251138228 conn(0x7f56c00fc6b0 msgr2=0x7f56c00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 -- 172.21.15.137:0/251138228 shutdown_connections 2024-09-12T12:22:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.636+0000 7f56c66ac640 1 -- 172.21.15.137:0/251138228 wait complete. 2024-09-12T12:22:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 Processor -- start 2024-09-12T12:22:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 -- start start 2024-09-12T12:22:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0102cc0 0x7f56c01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56c01035e0 0x7f56c0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c010c110 0x7f56c019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0111ef0 con 0x7f56c010c110 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0111d70 con 0x7f56c0102cc0 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.637+0000 7f56c66ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0112070 con 0x7f56c01035e0 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c010c110 0x7f56c019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c010c110 0x7f56c019cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40672/0 (socket says 172.21.15.137:40672) 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 -- 172.21.15.137:0/1848039660 learned_addr learned my addr 172.21.15.137:0/1848039660 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56c01035e0 0x7f56c0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56bffff640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0102cc0 0x7f56c01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 -- 172.21.15.137:0/1848039660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56c01035e0 msgr2=0x7f56c0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56c01035e0 0x7f56c0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 -- 172.21.15.137:0/1848039660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0102cc0 msgr2=0x7f56c01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0102cc0 0x7f56c01982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 -- 172.21.15.137:0/1848039660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c019d2b0 con 0x7f56c010c110 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c4c49640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c010c110 0x7f56c019cb80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f56b800eaa0 tx=0x7f56b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56bd7fa640 1 -- 172.21.15.137:0/1848039660 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b800cdb0 con 0x7f56c010c110 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56bd7fa640 1 -- 172.21.15.137:0/1848039660 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b8005020 con 0x7f56c010c110 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c019d5a0 con 0x7f56c010c110 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56bd7fa640 1 -- 172.21.15.137:0/1848039660 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b80116b0 con 0x7f56c010c110 2024-09-12T12:22:55.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.638+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01a4e30 con 0x7f56c010c110 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56bd7fa640 1 -- 172.21.15.137:0/1848039660 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f56b8005560 con 0x7f56c010c110 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 Processor -- start 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 -- start start 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x565172288ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x565172293770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5651721ed2f0 con 0x565172293cb0 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565172195270 con 0x565172288af0 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.641+0000 7f56c944c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5651721e92b0 con 0x565172289410 2024-09-12T12:22:55.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40676/0 (socket says 172.21.15.137:40676) 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 -- 172.21.15.137:0/1145706932 learned_addr learned my addr 172.21.15.137:0/1145706932 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bd7fa640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56a80a2da0 0x7f56a80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x565172288ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bffff640 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x565172293770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56c4c49640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56a80a2da0 0x7f56a80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 -- 172.21.15.137:0/1145706932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 msgr2=0x565172293770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x565172293770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bd7fa640 1 -- 172.21.15.137:0/1848039660 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f56b80cf720 con 0x7f56c010c110 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 -- 172.21.15.137:0/1145706932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 msgr2=0x565172288ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x565172288ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 -- 172.21.15.137:0/1145706932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565172194340 con 0x565172293cb0 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bffff640 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x565172293770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722960a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f56b402a9a0 tx=0x7f56b40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba48e567a6795456 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56c4c49640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56a80a2da0 0x7f56a80a5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56b809bbc0 tx=0x7f56b8086760 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.642+0000 7f56a67fc640 1 -- 172.21.15.137:0/1145706932 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4035040 con 0x565172293cb0 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56a67fc640 1 -- 172.21.15.137:0/1145706932 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4009520 con 0x565172293cb0 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56a67fc640 1 -- 172.21.15.137:0/1145706932 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4003d30 con 0x565172293cb0 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 -- 172.21.15.137:0/1145706932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 msgr2=0x5651722960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722960a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f56b402a9a0 tx=0x7f56b40056d0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 -- 172.21.15.137:0/1145706932 shutdown_connections 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722960a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x565172293770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 --2- 172.21.15.137:0/1145706932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x565172288ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 -- 172.21.15.137:0/1145706932 >> 172.21.15.137:0/1145706932 conn(0x565172285060 msgr2=0x565172285440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 -- 172.21.15.137:0/1145706932 shutdown_connections 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.643+0000 7f56c944c740 1 -- 172.21.15.137:0/1145706932 wait complete. 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 -- start start 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 Processor -- start 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172290bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 0x565172291110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 0x565172291650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x565172296880 con 0x565172288af0 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565172176290 con 0x565172289410 2024-09-12T12:22:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x565172296a00 con 0x565172293cb0 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56c944c740 10 client.0 _ll_register_callbacks cb 0x5651721a7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172290bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172290bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40690/0 (socket says 172.21.15.137:40690) 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56bf7fe640 1 -- 172.21.15.137:0/4163734155 learned_addr learned my addr 172.21.15.137:0/4163734155 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.644+0000 7f56bffff640 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 0x565172291110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 0x565172291650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56bf7fe640 1 -- 172.21.15.137:0/4163734155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 msgr2=0x565172291650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 0x565172291650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56bf7fe640 1 -- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 msgr2=0x565172291110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 0x565172291110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56bf7fe640 1 -- 172.21.15.137:0/4163734155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565171f00210 con 0x565172288af0 2024-09-12T12:22:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172290bd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f56b4000c00 tx=0x7f56b4003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a3205a7ab977c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f569f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f569f7fe640 1 -- 172.21.15.137:0/4163734155 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4035040 con 0x565172288af0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f569f7fe640 1 -- 172.21.15.137:0/4163734155 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f56b4004130 con 0x565172288af0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b4009520 con 0x565172288af0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f569f7fe640 1 -- 172.21.15.137:0/4163734155 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f56b405a030 con 0x565172288af0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56c944c740 10 client.16611 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565171fcc620 con 0x565172288af0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.645+0000 7f569f7fe640 1 -- 172.21.15.137:0/4163734155 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4009520 con 0x565172288af0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f569f7fe640 1 -- 172.21.15.137:0/4163734155 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f56b4048070 con 0x565172288af0 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56c944c740 10 client.16611 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56c944c740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56c944c740 20 client.16611 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56c944c740 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5651722a3ad0 0x5651722a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56c944c740 4 client.16611 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651721faa20 con 0x5651722a3ad0 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56bffff640 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5651722a3ad0 0x5651722a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f569f7fe640 10 client.16611 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.646+0000 7f56bffff640 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5651722a3ad0 0x5651722a5ec0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402f5f6acb865cb8 server_cookie=4ea62d35c52d0af in_seq=0 out_seq=0 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f569f7fe640 1 -- 172.21.15.137:0/4163734155 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651721faa20 con 0x5651722a3ad0 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f569f7fe640 10 client.16611 handle_command_reply: tid=0 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 client.16611 shutdown 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 msgr2=0x565172290bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172290bd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f56b4000c00 tx=0x7f56b4003a30 comp rx=0 tx=0).stop 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 shutdown_connections 2024-09-12T12:22:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 0x565172291650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 0x565172291110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5651722a3ad0 0x5651722a5ec0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 --2- 172.21.15.137:0/4163734155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172290bd0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f569f7fe640 0 client.16611 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 >> 172.21.15.137:0/4163734155 conn(0x565172285060 msgr2=0x56517228ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 shutdown_connections 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 1 -- 172.21.15.137:0/4163734155 wait complete. 2024-09-12T12:22:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.647+0000 7f56c944c740 20 client.16611 trim_cache size 0 max 16384 2024-09-12T12:22:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 Processor -- start 2024-09-12T12:22:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 -- start start 2024-09-12T12:22:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172291480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 0x5651722a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 0x5651722a9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01aff60 con 0x565172288af0 2024-09-12T12:22:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01ad7d0 con 0x565172289410 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56c944c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01ab620 con 0x565172293cb0 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172291480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172291480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40700/0 (socket says 172.21.15.137:40700) 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 -- 172.21.15.137:0/883863205 learned_addr learned my addr 172.21.15.137:0/883863205 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 -- 172.21.15.137:0/883863205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 msgr2=0x5651722a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/883863205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 0x5651722a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 -- 172.21.15.137:0/883863205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 msgr2=0x5651722a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/883863205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 0x5651722a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 -- 172.21.15.137:0/883863205 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565172289240 con 0x565172288af0 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/883863205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172291480 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f56b40038c0 tx=0x7f56b402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490be19231de3bba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56a5ffb640 1 -- 172.21.15.137:0/883863205 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4035040 con 0x565172288af0 2024-09-12T12:22:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.662+0000 7f56a5ffb640 1 -- 172.21.15.137:0/883863205 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4051030 con 0x565172288af0 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56a5ffb640 1 -- 172.21.15.137:0/883863205 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4003f30 con 0x565172288af0 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 -- 172.21.15.137:0/883863205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 msgr2=0x565172291480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 --2- 172.21.15.137:0/883863205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172291480 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f56b40038c0 tx=0x7f56b402a480 comp rx=0 tx=0).stop 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 -- 172.21.15.137:0/883863205 shutdown_connections 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 --2- 172.21.15.137:0/883863205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172293cb0 0x5651722a9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 --2- 172.21.15.137:0/883863205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172289410 0x5651722a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 --2- 172.21.15.137:0/883863205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172288af0 0x565172291480 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 -- 172.21.15.137:0/883863205 >> 172.21.15.137:0/883863205 conn(0x565172285060 msgr2=0x565172292c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 -- 172.21.15.137:0/883863205 shutdown_connections 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 -- 172.21.15.137:0/883863205 wait complete. 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.663+0000 7f56c944c740 1 -- start start 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 1 Processor -- start 2024-09-12T12:22:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x5651722a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x5651722a6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722a6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01abc90 con 0x565172293cb0 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01ad100 con 0x565172288af0 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56c01ac360 con 0x565172289410 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56c944c740 10 client.0 _ll_register_callbacks cb 0x7f56b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722a6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722a6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40702/0 (socket says 172.21.15.137:40702) 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 -- 172.21.15.137:0/2698802342 learned_addr learned my addr 172.21.15.137:0/2698802342 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:22:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bffff640 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x5651722a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x5651722a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 -- 172.21.15.137:0/2698802342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 msgr2=0x5651722a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x5651722a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 -- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 msgr2=0x5651722a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x5651722a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.664+0000 7f56bf7fe640 1 -- 172.21.15.137:0/2698802342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651722a7460 con 0x565172293cb0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56bf7fe640 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722a6d70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f56b4003890 tx=0x7f56b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b2b5584c771c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f569f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f569f7fe640 1 -- 172.21.15.137:0/2698802342 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4035040 con 0x565172293cb0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f569f7fe640 1 -- 172.21.15.137:0/2698802342 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8440+0+0 (secure 0 0 0) 0x7f56b4032e30 con 0x565172293cb0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b4051030 con 0x565172293cb0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f569f7fe640 1 -- 172.21.15.137:0/2698802342 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f56b40038f0 con 0x565172293cb0 2024-09-12T12:22:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 10 client.16617 fetch_fsmap learned FSMap version 37 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56517230e2c0 con 0x565172293cb0 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f569f7fe640 1 -- 172.21.15.137:0/2698802342 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4051030 con 0x565172293cb0 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f569f7fe640 1 -- 172.21.15.137:0/2698802342 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f56b4048040 con 0x565172293cb0 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 10 client.16617 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 20 client.16617 populate_metadata read hostname 'smithi137' 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56517230e6d0 0x565172310af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 4 client.16617 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.665+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651721f4d70 con 0x56517230e6d0 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.666+0000 7f56bffff640 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56517230e6d0 0x565172310af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.666+0000 7f569f7fe640 10 client.16617 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.666+0000 7f56bffff640 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56517230e6d0 0x565172310af0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93fcac031311a535 server_cookie=5dfb371fb7d4b640 in_seq=0 out_seq=0 2024-09-12T12:22:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f569f7fe640 1 -- 172.21.15.137:0/2698802342 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651721f4d70 con 0x56517230e6d0 2024-09-12T12:22:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f569f7fe640 10 client.16617 handle_command_reply: tid=0 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 client.16617 shutdown 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 msgr2=0x5651722a6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722a6d70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f56b4003890 tx=0x7f56b40057f0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 shutdown_connections 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56517230e6d0 0x565172310af0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565172293cb0 0x5651722a6d70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x565172289410 0x5651722a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 --2- 172.21.15.137:0/2698802342 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565172288af0 0x5651722a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 >> 172.21.15.137:0/2698802342 conn(0x565172285060 msgr2=0x56517229ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.733+0000 7f569f7fe640 0 client.16617 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.734+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 shutdown_connections 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.734+0000 7f56c944c740 1 -- 172.21.15.137:0/2698802342 wait complete. 2024-09-12T12:22:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.734+0000 7f56c944c740 20 client.16617 trim_cache size 0 max 16384 2024-09-12T12:22:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:22:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:22:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:22:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.734+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56a80a2da0 msgr2=0x7f56a80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.734+0000 7f56c66ac640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56a80a2da0 0x7f56a80a5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f56b809bbc0 tx=0x7f56b8086760 comp rx=0 tx=0).stop 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.734+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c010c110 msgr2=0x7f56c019cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.734+0000 7f56c66ac640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c010c110 0x7f56c019cb80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f56b800eaa0 tx=0x7f56b800ef60 comp rx=0 tx=0).stop 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 shutdown_connections 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f56a80a2da0 0x7f56a80a5260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56c010c110 0x7f56c019cb80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f56c01035e0 0x7f56c0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 --2- 172.21.15.137:0/1848039660 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56c0102cc0 0x7f56c01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:22:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 >> 172.21.15.137:0/1848039660 conn(0x7f56c00fc6b0 msgr2=0x7f56c00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:22:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 shutdown_connections 2024-09-12T12:22:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:22:55.735+0000 7f56c66ac640 1 -- 172.21.15.137:0/1848039660 wait complete. 2024-09-12T12:22:55.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 8.04 MB/sec execute 390 sec latency 2402.338 ms 2024-09-12T12:22:56.745 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 8.02 MB/sec execute 391 sec latency 3402.416 ms 2024-09-12T12:22:56.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:56 smithi171 ceph-mon[35098]: pgmap v304: 97 pgs: 97 active+clean; 240 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 357 op/s 2024-09-12T12:22:57.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:56 smithi137 ceph-mon[30950]: pgmap v304: 97 pgs: 97 active+clean; 240 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 357 op/s 2024-09-12T12:22:57.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:56 smithi189 ceph-mon[35475]: pgmap v304: 97 pgs: 97 active+clean; 240 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 357 op/s 2024-09-12T12:22:57.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 8.00 MB/sec execute 392 sec latency 4402.503 ms 2024-09-12T12:22:58.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.98 MB/sec execute 393 sec latency 5402.603 ms 2024-09-12T12:22:58.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:22:58 smithi171 ceph-mon[35098]: pgmap v305: 97 pgs: 97 active+clean; 240 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2024-09-12T12:22:59.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:22:58 smithi137 ceph-mon[30950]: pgmap v305: 97 pgs: 97 active+clean; 240 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2024-09-12T12:22:59.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:22:58 smithi189 ceph-mon[35475]: pgmap v305: 97 pgs: 97 active+clean; 240 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2024-09-12T12:22:59.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.96 MB/sec execute 394 sec latency 6402.703 ms 2024-09-12T12:23:00.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.94 MB/sec execute 395 sec latency 7402.814 ms 2024-09-12T12:23:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:00 smithi171 ceph-mon[35098]: pgmap v306: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-12T12:23:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:00 smithi137 ceph-mon[30950]: pgmap v306: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-12T12:23:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:00 smithi189 ceph-mon[35475]: pgmap v306: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.4 MiB/s wr, 253 op/s 2024-09-12T12:23:01.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.92 MB/sec execute 396 sec latency 8402.904 ms 2024-09-12T12:23:02.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.90 MB/sec execute 397 sec latency 9403.009 ms 2024-09-12T12:23:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:02 smithi171 ceph-mon[35098]: pgmap v307: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-12T12:23:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:02 smithi137 ceph-mon[30950]: pgmap v307: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-12T12:23:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:02 smithi189 ceph-mon[35475]: pgmap v307: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-12T12:23:03.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.88 MB/sec execute 398 sec latency 10403.103 ms 2024-09-12T12:23:04.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.86 MB/sec execute 399 sec latency 11403.215 ms 2024-09-12T12:23:04.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:04 smithi171 ceph-mon[35098]: pgmap v308: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-12T12:23:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:04 smithi137 ceph-mon[30950]: pgmap v308: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-12T12:23:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:04 smithi189 ceph-mon[35475]: pgmap v308: 97 pgs: 97 active+clean; 235 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-12T12:23:05.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.84 MB/sec execute 400 sec latency 12403.318 ms 2024-09-12T12:23:06.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.82 MB/sec execute 401 sec latency 13403.383 ms 2024-09-12T12:23:06.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:06 smithi171 ceph-mon[35098]: pgmap v309: 97 pgs: 97 active+clean; 234 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-09-12T12:23:07.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:06 smithi137 ceph-mon[30950]: pgmap v309: 97 pgs: 97 active+clean; 234 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-09-12T12:23:07.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:06 smithi189 ceph-mon[35475]: pgmap v309: 97 pgs: 97 active+clean; 234 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-09-12T12:23:07.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.80 MB/sec execute 402 sec latency 14403.424 ms 2024-09-12T12:23:08.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.78 MB/sec execute 403 sec latency 15403.512 ms 2024-09-12T12:23:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 -- 172.21.15.137:0/737180295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 msgr2=0x7f8f1010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 --2- 172.21.15.137:0/737180295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f1010d8b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8f08009cd0 tx=0x7f8f0802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 -- 172.21.15.137:0/737180295 shutdown_connections 2024-09-12T12:23:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 --2- 172.21.15.137:0/737180295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f1010d8b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 --2- 172.21.15.137:0/737180295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f101009e0 0x7f8f10100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 --2- 172.21.15.137:0/737180295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f101069f0 0x7f8f10106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 -- 172.21.15.137:0/737180295 >> 172.21.15.137:0/737180295 conn(0x7f8f100fc6c0 msgr2=0x7f8f100feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 -- 172.21.15.137:0/737180295 shutdown_connections 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.824+0000 7f8f1596d640 1 -- 172.21.15.137:0/737180295 wait complete. 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 Processor -- start 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 -- start start 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f101009e0 0x7f8f101972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f101977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f101069f0 0x7f8f10197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10110e30 con 0x7f8f10101380 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10110cb0 con 0x7f8f101069f0 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f1596d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f10110fb0 con 0x7f8f101009e0 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f067fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f101977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f101009e0 0x7f8f101972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f067fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f101977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44060/0 (socket says 172.21.15.137:44060) 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f067fc640 1 -- 172.21.15.137:0/1114236378 learned_addr learned my addr 172.21.15.137:0/1114236378 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.825+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f101069f0 0x7f8f10197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f067fc640 1 -- 172.21.15.137:0/1114236378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f101009e0 msgr2=0x7f8f101972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f101009e0 0x7f8f101972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f067fc640 1 -- 172.21.15.137:0/1114236378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f101069f0 msgr2=0x7f8f10197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f101069f0 0x7f8f10197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f067fc640 1 -- 172.21.15.137:0/1114236378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f101a6b10 con 0x7f8f10101380 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f101977e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d770 tx=0x7f8ef800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f1496b640 1 -- 172.21.15.137:0/1114236378 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8017070 con 0x7f8f10101380 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f1496b640 1 -- 172.21.15.137:0/1114236378 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef8005500 con 0x7f8f10101380 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f101a6e00 con 0x7f8f10101380 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f1496b640 1 -- 172.21.15.137:0/1114236378 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8004e00 con 0x7f8f10101380 2024-09-12T12:23:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.826+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f101a7270 con 0x7f8f10101380 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.828+0000 7f8f1496b640 1 -- 172.21.15.137:0/1114236378 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8ef8004fa0 con 0x7f8f10101380 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.828+0000 7f8f187e5740 1 Processor -- start 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f187e5740 1 -- start start 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f1496b640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8efc0a2da0 0x7f8efc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97f9af0 0x5605b97f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b9804770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f187e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5605b975e2f0 con 0x5605b97f9af0 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f187e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5605b9706270 con 0x5605b9804cb0 2024-09-12T12:23:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f187e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605b975a2b0 con 0x5605b97fa410 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8efc0a2da0 0x7f8efc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f1496b640 1 -- 172.21.15.137:0/1114236378 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f8ef80cee10 con 0x7f8f10101380 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97f9af0 0x5605b97f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97f9af0 0x5605b97f9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44066/0 (socket says 172.21.15.137:44066) 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 -- 172.21.15.137:0/1926129346 learned_addr learned my addr 172.21.15.137:0/1926129346 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b9804770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 -- 172.21.15.137:0/1926129346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 msgr2=0x5605b9804770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b9804770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 -- 172.21.15.137:0/1926129346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 msgr2=0x5605b98070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 -- 172.21.15.137:0/1926129346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b9705340 con 0x5605b97f9af0 2024-09-12T12:23:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b9804770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8efc0a2da0 0x7f8efc0a5260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006010 tx=0x7f8f080316c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f0effd640 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97f9af0 0x5605b97f9ed0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000bc90 tx=0x7f8f000107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f0a83693a7ddc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.829+0000 7f8f05ffb640 1 -- 172.21.15.137:0/1926129346 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0001c480 con 0x5605b97f9af0 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f05ffb640 1 -- 172.21.15.137:0/1926129346 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f0001cb00 con 0x5605b97f9af0 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f05ffb640 1 -- 172.21.15.137:0/1926129346 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f00011ba0 con 0x5605b97f9af0 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 -- 172.21.15.137:0/1926129346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97f9af0 msgr2=0x5605b97f9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97f9af0 0x5605b97f9ed0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8f0000bc90 tx=0x7f8f000107a0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 -- 172.21.15.137:0/1926129346 shutdown_connections 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b9804770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1926129346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97f9af0 0x5605b97f9ed0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 -- 172.21.15.137:0/1926129346 >> 172.21.15.137:0/1926129346 conn(0x5605b97f6060 msgr2=0x5605b97f6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 -- 172.21.15.137:0/1926129346 shutdown_connections 2024-09-12T12:23:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 -- 172.21.15.137:0/1926129346 wait complete. 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.830+0000 7f8f187e5740 1 -- start start 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 1 Processor -- start 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b9801b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b9802090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5605b9807880 con 0x5605b97fa410 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5605b96e7290 con 0x5605b9804cb0 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5605b9807a00 con 0x5605b97f9af0 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f187e5740 10 client.0 _ll_register_callbacks cb 0x5605b9718140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b9802090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b9801b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b9802090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44078/0 (socket says 172.21.15.137:44078) 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 -- 172.21.15.137:0/1063356871 learned_addr learned my addr 172.21.15.137:0/1063356871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 -- 172.21.15.137:0/1063356871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 msgr2=0x5605b9801b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b9801b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 -- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 msgr2=0x5605b98025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f067fc640 1 -- 172.21.15.137:0/1063356871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b9471210 con 0x5605b97fa410 2024-09-12T12:23:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.831+0000 7f8f0effd640 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b9801b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f067fc640 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b9802090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d740 tx=0x7f8ef80057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c183681a2e4b676 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8eeeffd640 1 -- 172.21.15.137:0/1063356871 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8020040 con 0x5605b97fa410 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8eeeffd640 1 -- 172.21.15.137:0/1063356871 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f8ef8097a10 con 0x5605b97fa410 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f0001cb00 con 0x5605b97fa410 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8eeeffd640 1 -- 172.21.15.137:0/1063356871 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8ef80b42c0 con 0x5605b97fa410 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 10 client.16629 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5605b953d620 con 0x5605b97fa410 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8eeeffd640 1 -- 172.21.15.137:0/1063356871 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f0001cb00 con 0x5605b97fa410 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8eeeffd640 1 -- 172.21.15.137:0/1063356871 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef80cc7a0 con 0x5605b97fa410 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 10 client.16629 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:08.841 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 20 client.16629 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b9814be0 0x5605b9817030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 4 client.16629 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605b976ba20 con 0x5605b9814be0 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.832+0000 7f8f0effd640 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b9814be0 0x5605b9817030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.833+0000 7f8eeeffd640 10 client.16629 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.833+0000 7f8f0effd640 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b9814be0 0x5605b9817030 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9c9979e7d03774 server_cookie=e34dd9368aab688c in_seq=0 out_seq=0 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.833+0000 7f8eeeffd640 1 -- 172.21.15.137:0/1063356871 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605b976ba20 con 0x5605b9814be0 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.833+0000 7f8eeeffd640 10 client.16629 handle_command_reply: tid=0 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.833+0000 7f8f187e5740 1 client.16629 shutdown 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 msgr2=0x5605b9802090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b9802090 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d740 tx=0x7f8ef80057e0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 shutdown_connections 2024-09-12T12:23:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b98025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b9814be0 0x5605b9817030 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b9802090 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8eeeffd640 0 client.16629 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 --2- 172.21.15.137:0/1063356871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b9801b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 >> 172.21.15.137:0/1063356871 conn(0x5605b97f6060 msgr2=0x5605b97fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 shutdown_connections 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 1 -- 172.21.15.137:0/1063356871 wait complete. 2024-09-12T12:23:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.834+0000 7f8f187e5740 20 client.16629 trim_cache size 0 max 16384 2024-09-12T12:23:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 Processor -- start 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 -- start start 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b97f9af0 0x5605b9802400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b981c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b9804cb0 0x5605b981ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101b23d0 con 0x5605b9804cb0 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101afc40 con 0x5605b97f9af0 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f187e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101ada90 con 0x5605b97fa410 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b9804cb0 0x5605b981ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b9804cb0 0x5605b981ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44080/0 (socket says 172.21.15.137:44080) 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 -- 172.21.15.137:0/638636288 learned_addr learned my addr 172.21.15.137:0/638636288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f067fc640 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b981c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0effd640 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b97f9af0 0x5605b9802400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 -- 172.21.15.137:0/638636288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 msgr2=0x5605b981c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b981c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 -- 172.21.15.137:0/638636288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b97f9af0 msgr2=0x5605b9802400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b97f9af0 0x5605b9802400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 -- 172.21.15.137:0/638636288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605b97f5e90 con 0x5605b9804cb0 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0effd640 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b97f9af0 0x5605b9802400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f067fc640 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b981c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b9804cb0 0x5605b981ced0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802ae00 tx=0x7f8f08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6d33818eece830 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.848+0000 7f8f057fa640 1 -- 172.21.15.137:0/638636288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08004710 con 0x5605b9804cb0 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f057fa640 1 -- 172.21.15.137:0/638636288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f08036070 con 0x5605b9804cb0 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f057fa640 1 -- 172.21.15.137:0/638636288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f0803d680 con 0x5605b9804cb0 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 -- 172.21.15.137:0/638636288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b9804cb0 msgr2=0x5605b981ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b9804cb0 0x5605b981ced0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8f0802ae00 tx=0x7f8f08004a40 comp rx=0 tx=0).stop 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 -- 172.21.15.137:0/638636288 shutdown_connections 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b9804cb0 0x5605b981ced0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97fa410 0x5605b981c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 --2- 172.21.15.137:0/638636288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b97f9af0 0x5605b9802400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 -- 172.21.15.137:0/638636288 >> 172.21.15.137:0/638636288 conn(0x5605b97f6060 msgr2=0x5605b9803bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 -- 172.21.15.137:0/638636288 shutdown_connections 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 -- 172.21.15.137:0/638636288 wait complete. 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.849+0000 7f8f187e5740 1 -- start start 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 1 Processor -- start 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b980d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b97fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b97fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101ae100 con 0x5605b97fa410 2024-09-12T12:23:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101af570 con 0x5605b9804cb0 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8f101ae7d0 con 0x5605b97f9af0 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f067fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b97fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f187e5740 10 client.0 _ll_register_callbacks cb 0x7f8ef809ad00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f0effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b980d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f067fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b97fe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44084/0 (socket says 172.21.15.137:44084) 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f067fc640 1 -- 172.21.15.137:0/867743835 learned_addr learned my addr 172.21.15.137:0/867743835 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f0f7fe640 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b97fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f0effd640 1 -- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 msgr2=0x5605b97fef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f0effd640 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b97fef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f0effd640 1 -- 172.21.15.137:0/867743835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 msgr2=0x5605b97fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f0effd640 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b97fe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.850+0000 7f8f0effd640 1 -- 172.21.15.137:0/867743835 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605b97ff720 con 0x5605b97f9af0 2024-09-12T12:23:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f0effd640 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b980d4d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f00011740 tx=0x7f8f00005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a486da5490bcf5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8eeeffd640 1 -- 172.21.15.137:0/867743835 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f000114b0 con 0x5605b97f9af0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8eeeffd640 1 -- 172.21.15.137:0/867743835 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f8f000050d0 con 0x5605b97f9af0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f08036070 con 0x5605b97f9af0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8eeeffd640 1 -- 172.21.15.137:0/867743835 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8f000052d0 con 0x5605b97f9af0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 10 client.26074 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5605b987f9c0 con 0x5605b97f9af0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8eeeffd640 1 -- 172.21.15.137:0/867743835 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f08036070 con 0x5605b97f9af0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8eeeffd640 1 -- 172.21.15.137:0/867743835 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f8f00003820 con 0x5605b97f9af0 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 10 client.26074 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 20 client.26074 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b987ffa0 0x5605b9882390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 4 client.26074 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:23:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605b9765d70 con 0x5605b987ffa0 2024-09-12T12:23:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.851+0000 7f8f067fc640 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b987ffa0 0x5605b9882390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.852+0000 7f8eeeffd640 10 client.26074 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.852+0000 7f8f067fc640 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b987ffa0 0x5605b9882390 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=119c575d17e62b99 server_cookie=709603054eea6bd2 in_seq=0 out_seq=0 2024-09-12T12:23:08.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8eeeffd640 1 -- 172.21.15.137:0/867743835 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605b9765d70 con 0x5605b987ffa0 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8eeeffd640 10 client.26074 handle_command_reply: tid=0 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 client.26074 shutdown 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 msgr2=0x5605b980d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b980d4d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f00011740 tx=0x7f8f00005850 comp rx=0 tx=0).stop 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 shutdown_connections 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5605b9804cb0 0x5605b97fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5605b987ffa0 0x5605b9882390 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5605b97fa410 0x5605b97fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 --2- 172.21.15.137:0/867743835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5605b97f9af0 0x5605b980d4d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8eeeffd640 0 client.26074 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 >> 172.21.15.137:0/867743835 conn(0x5605b97f6060 msgr2=0x5605b97f49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 shutdown_connections 2024-09-12T12:23:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 1 -- 172.21.15.137:0/867743835 wait complete. 2024-09-12T12:23:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.894+0000 7f8f187e5740 20 client.26074 trim_cache size 0 max 16384 2024-09-12T12:23:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:23:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:23:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:23:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:23:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8efc0a2da0 msgr2=0x7f8efc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8efc0a2da0 0x7f8efc0a5260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006010 tx=0x7f8f080316c0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 msgr2=0x7f8f101977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f101977e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d770 tx=0x7f8ef800dc30 comp rx=0 tx=0).stop 2024-09-12T12:23:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 shutdown_connections 2024-09-12T12:23:08.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8efc0a2da0 0x7f8efc0a5260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f101069f0 0x7f8f10197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8f10101380 0x7f8f101977e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 --2- 172.21.15.137:0/1114236378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f101009e0 0x7f8f101972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 >> 172.21.15.137:0/1114236378 conn(0x7f8f100fc6c0 msgr2=0x7f8f100fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 shutdown_connections 2024-09-12T12:23:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:08.895+0000 7f8f1596d640 1 -- 172.21.15.137:0/1114236378 wait complete. 2024-09-12T12:23:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:08 smithi171 ceph-mon[35098]: pgmap v310: 97 pgs: 97 active+clean; 234 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-12T12:23:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:08 smithi171 ceph-mon[35098]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T12:23:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:08 smithi137 ceph-mon[30950]: pgmap v310: 97 pgs: 97 active+clean; 234 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-12T12:23:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:08 smithi137 ceph-mon[30950]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T12:23:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:08 smithi189 ceph-mon[35475]: pgmap v310: 97 pgs: 97 active+clean; 234 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-12T12:23:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:08 smithi189 ceph-mon[35475]: osdmap e96: 12 total, 12 up, 12 in 2024-09-12T12:23:09.746 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.76 MB/sec execute 404 sec latency 16403.564 ms 2024-09-12T12:23:10.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.74 MB/sec execute 405 sec latency 17403.623 ms 2024-09-12T12:23:10.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:10 smithi171 ceph-mon[35098]: pgmap v312: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-09-12T12:23:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:10 smithi137 ceph-mon[30950]: pgmap v312: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-09-12T12:23:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:10 smithi189 ceph-mon[35475]: pgmap v312: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-09-12T12:23:11.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.72 MB/sec execute 406 sec latency 18403.709 ms 2024-09-12T12:23:12.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.71 MB/sec execute 407 sec latency 19403.812 ms 2024-09-12T12:23:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:12 smithi137 ceph-mon[30950]: pgmap v313: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-09-12T12:23:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:12 smithi137 ceph-mon[30950]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T12:23:13.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:12 smithi189 ceph-mon[35475]: pgmap v313: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-09-12T12:23:13.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:12 smithi189 ceph-mon[35475]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T12:23:13.152 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:23:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:12 smithi171 ceph-mon[35098]: pgmap v313: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 1.7 MiB/s wr, 103 op/s 2024-09-12T12:23:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:12 smithi171 ceph-mon[35098]: osdmap e97: 12 total, 12 up, 12 in 2024-09-12T12:23:13.421 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:23:13.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.69 MB/sec execute 408 sec latency 20403.878 ms 2024-09-12T12:23:13.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 -- 172.21.15.137:0/3670038191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f705410c180 msgr2=0x7f705410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:13.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 --2- 172.21.15.137:0/3670038191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f705410c180 0x7f705410e610 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f704c009ca0 tx=0x7f704c02a950 comp rx=0 tx=0).stop 2024-09-12T12:23:13.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 -- 172.21.15.137:0/3670038191 shutdown_connections 2024-09-12T12:23:13.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 --2- 172.21.15.137:0/3670038191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f705410c180 0x7f705410e610 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 --2- 172.21.15.137:0/3670038191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054103700 0x7f705410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 --2- 172.21.15.137:0/3670038191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7054102de0 0x7f70541031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 -- 172.21.15.137:0/3670038191 >> 172.21.15.137:0/3670038191 conn(0x7f70540fc7a0 msgr2=0x7f70540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 -- 172.21.15.137:0/3670038191 shutdown_connections 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 -- 172.21.15.137:0/3670038191 wait complete. 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 Processor -- start 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.924+0000 7f705ade5640 1 -- start start 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f705ade5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054102de0 0x7f7054100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f705ade5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054103700 0x7f7054100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f705ade5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f705410c180 0x7f70541014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f705ade5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7054111f50 con 0x7f7054103700 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f705ade5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7054111dd0 con 0x7f705410c180 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f705ade5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f70541120d0 con 0x7f7054102de0 2024-09-12T12:23:13.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054103700 0x7f7054100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054103700 0x7f7054100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60018/0 (socket says 172.21.15.137:60018) 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 -- 172.21.15.137:0/1575800560 learned_addr learned my addr 172.21.15.137:0/1575800560 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7058b81640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054102de0 0x7f7054100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7059382640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f705410c180 0x7f70541014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 -- 172.21.15.137:0/1575800560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054102de0 msgr2=0x7f7054100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054102de0 0x7f7054100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 -- 172.21.15.137:0/1575800560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f705410c180 msgr2=0x7f70541014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f705410c180 0x7f70541014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 -- 172.21.15.137:0/1575800560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70541a68d0 con 0x7f7054103700 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7059382640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f705410c180 0x7f70541014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7058b81640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054102de0 0x7f7054100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:13.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7053fff640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054103700 0x7f7054100f80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f704800d770 tx=0x7f704800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7051ffb640 1 -- 172.21.15.137:0/1575800560 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048017070 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.925+0000 7f7051ffb640 1 -- 172.21.15.137:0/1575800560 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7048005500 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.926+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70541a6bc0 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.926+0000 7f7051ffb640 1 -- 172.21.15.137:0/1575800560 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048004e00 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.926+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541a70b0 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.927+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70541031c0 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.928+0000 7f7051ffb640 1 -- 172.21.15.137:0/1575800560 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7048004fa0 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.928+0000 7f7051ffb640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f703c0a2cf0 0x7f703c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.928+0000 7f7058b81640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f703c0a2cf0 0x7f703c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.928+0000 7f7051ffb640 1 -- 172.21.15.137:0/1575800560 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f70480cee00 con 0x7f7054103700 2024-09-12T12:23:13.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.929+0000 7f7058b81640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f703c0a2cf0 0x7f703c0a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7044005d00 tx=0x7f704400b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:13.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:13.933+0000 7f7051ffb640 1 -- 172.21.15.137:0/1575800560 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f704809ba40 con 0x7f7054103700 2024-09-12T12:23:14.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.136+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f705402d7c0 con 0x7f7054103700 2024-09-12T12:23:14.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.137+0000 7f7051ffb640 1 -- 172.21.15.137:0/1575800560 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11936 (secure 0 0 0) 0x7f704809b340 con 0x7f7054103700 2024-09-12T12:23:14.138 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:23:14.138 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":37,"btime":"2024-09-12T12:22:28:627072+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:22:28.158799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:23:14.138 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 37 2024-09-12T12:23:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f703c0a2cf0 msgr2=0x7f703c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f703c0a2cf0 0x7f703c0a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7044005d00 tx=0x7f704400b410 comp rx=0 tx=0).stop 2024-09-12T12:23:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054103700 msgr2=0x7f7054100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054103700 0x7f7054100f80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f704800d770 tx=0x7f704800dc30 comp rx=0 tx=0).stop 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 shutdown_connections 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f703c0a2cf0 0x7f703c0a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f705410c180 0x7f70541014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7054103700 0x7f7054100f80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 --2- 172.21.15.137:0/1575800560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7054102de0 0x7f7054100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 >> 172.21.15.137:0/1575800560 conn(0x7f70540fc7a0 msgr2=0x7f70540fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 shutdown_connections 2024-09-12T12:23:14.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:14.140+0000 7f705ade5640 1 -- 172.21.15.137:0/1575800560 wait complete. 2024-09-12T12:23:14.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.67 MB/sec execute 409 sec latency 21404.194 ms 2024-09-12T12:23:14.874 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:14 smithi137 ceph-mon[30950]: pgmap v315: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-12T12:23:14.874 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:14 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1575800560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:14.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 9aaa6358 --release --await' 2024-09-12T12:23:14.900 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 9aaa6358 --release --await 2024-09-12T12:23:15.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:14 smithi189 ceph-mon[35475]: pgmap v315: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-12T12:23:15.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:14 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1575800560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:15.146 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:23:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:14 smithi171 ceph-mon[35098]: pgmap v315: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-12T12:23:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:14 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1575800560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:15.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 788824 7.65 MB/sec execute 410 sec latency 22404.281 ms 2024-09-12T12:23:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.033+0000 7f347edb1640 1 -- 172.21.15.137:0/314290689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478100e20 msgr2=0x7f3478101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.033+0000 7f347edb1640 1 --2- 172.21.15.137:0/314290689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478100e20 0x7f3478101200 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3468009ca0 tx=0x7f346802a950 comp rx=0 tx=0).stop 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- 172.21.15.137:0/314290689 shutdown_connections 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 --2- 172.21.15.137:0/314290689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478102ef0 0x7f347810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 --2- 172.21.15.137:0/314290689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3478102570 0x7f34781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 --2- 172.21.15.137:0/314290689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478100e20 0x7f3478101200 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- 172.21.15.137:0/314290689 >> 172.21.15.137:0/314290689 conn(0x7f34780fc7d0 msgr2=0x7f34780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- 172.21.15.137:0/314290689 shutdown_connections 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- 172.21.15.137:0/314290689 wait complete. 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 Processor -- start 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- start start 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100e20 0x7f3478198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478102570 0x7f3478198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3478102ef0 0x7f347819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3478112060 con 0x7f3478102570 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3478111ee0 con 0x7f3478102ef0 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.034+0000 7f347edb1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34781121e0 con 0x7f3478100e20 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478102570 0x7f3478198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478102570 0x7f3478198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60026/0 (socket says 172.21.15.137:60026) 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 -- 172.21.15.137:0/3894533200 learned_addr learned my addr 172.21.15.137:0/3894533200 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f347cb4d640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100e20 0x7f3478198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f347d34e640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3478102ef0 0x7f347819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 -- 172.21.15.137:0/3894533200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100e20 msgr2=0x7f3478198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100e20 0x7f3478198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 -- 172.21.15.137:0/3894533200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3478102ef0 msgr2=0x7f347819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3478102ef0 0x7f347819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 -- 172.21.15.137:0/3894533200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347819d4b0 con 0x7f3478102570 2024-09-12T12:23:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f347cb4d640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100e20 0x7f3478198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3477fff640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478102570 0x7f3478198a40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f346000c920 tx=0x7f346000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3475ffb640 1 -- 172.21.15.137:0/3894533200 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460007c70 con 0x7f3478102570 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347819d7a0 con 0x7f3478102570 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3475ffb640 1 -- 172.21.15.137:0/3894533200 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460007e10 con 0x7f3478102570 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f3475ffb640 1 -- 172.21.15.137:0/3894533200 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460018890 con 0x7f3478102570 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.035+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34781a5080 con 0x7f3478102570 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.037+0000 7f3475ffb640 1 -- 172.21.15.137:0/3894533200 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3460018a70 con 0x7f3478102570 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 Processor -- start 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f3475ffb640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34640a2cf0 0x7f34640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 -- start start 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f347d34e640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34640a2cf0 0x7f34640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f3475ffb640 1 -- 172.21.15.137:0/3894533200 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f34600cf630 con 0x7f3478102570 2024-09-12T12:23:16.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7060 0x5626c10f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7980 0x5626c1101d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1104630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5626c105c620 con 0x5626c1102240 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5626c1003e50 con 0x5626c10f7980 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f348118a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626c10585e0 con 0x5626c10f7060 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f347d34e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f347d34e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1104630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60040/0 (socket says 172.21.15.137:60040) 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.038+0000 7f347d34e640 1 -- 172.21.15.137:0/1575720106 learned_addr learned my addr 172.21.15.137:0/1575720106 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347cb4d640 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7060 0x5626c10f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f3477fff640 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7980 0x5626c1101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347d34e640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34640a2cf0 0x7f34640a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3470013450 tx=0x7f3470013850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347d34e640 1 -- 172.21.15.137:0/1575720106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7060 msgr2=0x5626c10f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347d34e640 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7060 0x5626c10f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347d34e640 1 -- 172.21.15.137:0/1575720106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7980 msgr2=0x5626c1101d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347d34e640 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7980 0x5626c1101d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347d34e640 1 -- 172.21.15.137:0/1575720106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c0ff4d40 con 0x5626c1102240 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f347d34e640 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1104630 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3470013d50 tx=0x7f347001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4760060044d50bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f346effd640 1 -- 172.21.15.137:0/1575720106 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470022b40 con 0x5626c1102240 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f346effd640 1 -- 172.21.15.137:0/1575720106 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3470022ce0 con 0x5626c1102240 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.039+0000 7f346effd640 1 -- 172.21.15.137:0/1575720106 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347000ee60 con 0x5626c1102240 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 -- 172.21.15.137:0/1575720106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 msgr2=0x5626c1104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1104630 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3470013d50 tx=0x7f347001b000 comp rx=0 tx=0).stop 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 -- 172.21.15.137:0/1575720106 shutdown_connections 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1104630 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7980 0x5626c1101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 --2- 172.21.15.137:0/1575720106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7060 0x5626c10f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 -- 172.21.15.137:0/1575720106 >> 172.21.15.137:0/1575720106 conn(0x5626c10f35f0 msgr2=0x5626c10f39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 -- 172.21.15.137:0/1575720106 shutdown_connections 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 -- 172.21.15.137:0/1575720106 wait complete. 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.040+0000 7f348118a740 1 -- start start 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 1 Processor -- start 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c10f7060 0x5626c10ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c10ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c1102240 0x5626c10ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5626c1104e10 con 0x5626c10f7060 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5626c0fe4f10 con 0x5626c1102240 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5626c1104f90 con 0x5626c10f7980 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f348118a740 10 client.0 _ll_register_callbacks cb 0x5626c1008db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c10f7060 0x5626c10ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c10f7060 0x5626c10ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60052/0 (socket says 172.21.15.137:60052) 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 -- 172.21.15.137:0/3461372191 learned_addr learned my addr 172.21.15.137:0/3461372191 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347d34e640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c1102240 0x5626c10ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f3477fff640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c10ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 -- 172.21.15.137:0/3461372191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 msgr2=0x5626c10ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c10ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 -- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c1102240 msgr2=0x5626c10ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c1102240 0x5626c10ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347cb4d640 1 -- 172.21.15.137:0/3461372191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c0d6b580 con 0x5626c10f7060 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.041+0000 7f347d34e640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c1102240 0x5626c10ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f347cb4d640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c10f7060 0x5626c10ff100 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f346802f5d0 tx=0x7f346802f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312acca39d17877e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f3457fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f3457fff640 1 -- 172.21.15.137:0/3461372191 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468034040 con 0x5626c10f7060 2024-09-12T12:23:16.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f3457fff640 1 -- 172.21.15.137:0/3461372191 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f3468003b80 con 0x5626c10f7060 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3470022ce0 con 0x5626c10f7060 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f3457fff640 1 -- 172.21.15.137:0/3461372191 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f346802f720 con 0x5626c10f7060 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 10 client.16653 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5626c0e35960 con 0x5626c10f7060 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f3457fff640 1 -- 172.21.15.137:0/3461372191 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3470022ce0 con 0x5626c10f7060 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f3457fff640 1 -- 172.21.15.137:0/3461372191 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f346804b020 con 0x5626c10f7060 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 10 client.16653 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 10 client.16653 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 20 client.16653 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c1111d80 0x5626c1114200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 4 client.16653 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626c1069b90 con 0x5626c1111d80 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.042+0000 7f3477fff640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c1111d80 0x5626c1114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.043+0000 7f3477fff640 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c1111d80 0x5626c1114200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152621df205b3ece server_cookie=78fc028bee2824d0 in_seq=0 out_seq=0 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.043+0000 7f3457fff640 10 client.16653 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f3457fff640 1 -- 172.21.15.137:0/3461372191 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626c1069b90 con 0x5626c1111d80 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f3457fff640 10 client.16653 handle_command_reply: tid=0 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 client.16653 shutdown 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c10f7060 msgr2=0x5626c10ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c10f7060 0x5626c10ff100 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f346802f5d0 tx=0x7f346802f600 comp rx=0 tx=0).stop 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 shutdown_connections 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c1102240 0x5626c10ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c10ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c1111d80 0x5626c1114200 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 --2- 172.21.15.137:0/3461372191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c10f7060 0x5626c10ff100 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f3457fff640 0 client.16653 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:16.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 >> 172.21.15.137:0/3461372191 conn(0x5626c10f35f0 msgr2=0x5626c10f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 shutdown_connections 2024-09-12T12:23:16.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 1 -- 172.21.15.137:0/3461372191 wait complete. 2024-09-12T12:23:16.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.044+0000 7f348118a740 20 client.16653 trim_cache size 0 max 16384 2024-09-12T12:23:16.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 Processor -- start 2024-09-12T12:23:16.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 -- start start 2024-09-12T12:23:16.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c10f2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1119b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c111a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34781adec0 con 0x5626c1102240 2024-09-12T12:23:16.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34781ab730 con 0x5626c10f7060 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f348118a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34781a95f0 con 0x5626c10f7980 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c111a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c111a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60060/0 (socket says 172.21.15.137:60060) 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 -- 172.21.15.137:0/3137839757 learned_addr learned my addr 172.21.15.137:0/3137839757 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347cb4d640 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c10f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f3477fff640 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1119b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 -- 172.21.15.137:0/3137839757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 msgr2=0x5626c1119b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1119b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 -- 172.21.15.137:0/3137839757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 msgr2=0x5626c10f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c10f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.061+0000 7f347d34e640 1 -- 172.21.15.137:0/3137839757 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626c10f3090 con 0x5626c1102240 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f347d34e640 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c111a060 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f347000d4b0 tx=0x7f347000d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edba435b885310ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f346e7fc640 1 -- 172.21.15.137:0/3137839757 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470039070 con 0x5626c1102240 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f346e7fc640 1 -- 172.21.15.137:0/3137839757 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3470022e20 con 0x5626c1102240 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f346e7fc640 1 -- 172.21.15.137:0/3137839757 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470004880 con 0x5626c1102240 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f348118a740 1 -- 172.21.15.137:0/3137839757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 msgr2=0x5626c111a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f348118a740 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c111a060 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f347000d4b0 tx=0x7f347000d610 comp rx=0 tx=0).stop 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f348118a740 1 -- 172.21.15.137:0/3137839757 shutdown_connections 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f348118a740 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c111a060 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f348118a740 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1119b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f348118a740 1 --2- 172.21.15.137:0/3137839757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c10f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.062+0000 7f348118a740 1 -- 172.21.15.137:0/3137839757 >> 172.21.15.137:0/3137839757 conn(0x5626c10f35f0 msgr2=0x5626c10f1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.063+0000 7f348118a740 1 -- 172.21.15.137:0/3137839757 shutdown_connections 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.063+0000 7f348118a740 1 -- 172.21.15.137:0/3137839757 wait complete. 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.063+0000 7f348118a740 1 -- start start 2024-09-12T12:23:16.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.063+0000 7f348118a740 1 Processor -- start 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.063+0000 7f348118a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c110a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.063+0000 7f348118a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1106b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.063+0000 7f348118a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1107070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f348118a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f34781a9c40 con 0x5626c1102240 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f348118a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34781ab060 con 0x5626c10f7060 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f348118a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34781aa2c0 con 0x5626c10f7980 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60064/0 (socket says 172.21.15.137:60064) 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 -- 172.21.15.137:0/4119996279 learned_addr learned my addr 172.21.15.137:0/4119996279 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f348118a740 10 client.0 _ll_register_callbacks cb 0x7f3468009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347cb4d640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c110a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f3477fff640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1106b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 -- 172.21.15.137:0/4119996279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 msgr2=0x5626c1106b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1106b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 -- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 msgr2=0x5626c110a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c110a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 -- 172.21.15.137:0/4119996279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626c11076b0 con 0x5626c1102240 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f3477fff640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f347d34e640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1107070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3470037040 tx=0x7f34700115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b38b793a4568cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f3457fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f3457fff640 1 -- 172.21.15.137:0/4119996279 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470042060 con 0x5626c1102240 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3470022e20 con 0x5626c1102240 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f3457fff640 1 -- 172.21.15.137:0/4119996279 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f347003d410 con 0x5626c1102240 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f3457fff640 1 -- 172.21.15.137:0/4119996279 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3470039070 con 0x5626c1102240 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f348118a740 10 client.16659 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:16.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.064+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5626c117d950 con 0x5626c1102240 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f3457fff640 1 -- 172.21.15.137:0/4119996279 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3470022e20 con 0x5626c1102240 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f3457fff640 1 -- 172.21.15.137:0/4119996279 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f347002b3f0 con 0x5626c1102240 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f348118a740 10 client.16659 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f348118a740 10 client.16659 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f348118a740 20 client.16659 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f348118a740 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c117ddc0 0x5626c11801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f348118a740 4 client.16659 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9aaa6358", "release": true, "await": true}] 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9aaa6358", "release": true, "await": true}) -- 0x5626c10640a0 con 0x5626c117ddc0 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f347cb4d640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c117ddc0 0x5626c11801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f347cb4d640 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c117ddc0 0x5626c11801b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d84247d06e0dfd6 server_cookie=7e9e065df6f1bb4b in_seq=0 out_seq=0 2024-09-12T12:23:16.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.065+0000 7f3457fff640 10 client.16659 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f3457fff640 1 -- 172.21.15.137:0/4119996279 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5626c10640a0 con 0x5626c117ddc0 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f3457fff640 10 client.16659 handle_command_reply: tid=0 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 client.16659 shutdown 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 msgr2=0x5626c1107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1107070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3470037040 tx=0x7f34700115a0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 shutdown_connections 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5626c117ddc0 0x5626c11801b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626c1102240 0x5626c1107070 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626c10f7980 0x5626c1106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 --2- 172.21.15.137:0/4119996279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5626c10f7060 0x5626c110a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 >> 172.21.15.137:0/4119996279 conn(0x5626c10f35f0 msgr2=0x5626c10f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f3457fff640 0 client.16659 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 shutdown_connections 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 1 -- 172.21.15.137:0/4119996279 wait complete. 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.074+0000 7f348118a740 20 client.16659 trim_cache size 0 max 16384 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:23:16.076 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 37, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 40, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "9aaa6358": { 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 40, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 142.6, 2024-09-12T12:23:16.077 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:16.078 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34640a2cf0 msgr2=0x7f34640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34640a2cf0 0x7f34640a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3470013450 tx=0x7f3470013850 comp rx=0 tx=0).stop 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478102570 msgr2=0x7f3478198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478102570 0x7f3478198a40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f346000c920 tx=0x7f346000cde0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 shutdown_connections 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f34640a2cf0 0x7f34640a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3478102ef0 0x7f347819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478102570 0x7f3478198a40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 --2- 172.21.15.137:0/3894533200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100e20 0x7f3478198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:16.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 >> 172.21.15.137:0/3894533200 conn(0x7f34780fc7d0 msgr2=0x7f3478105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:16.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 shutdown_connections 2024-09-12T12:23:16.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:16.075+0000 7f347edb1640 1 -- 172.21.15.137:0/3894533200 wait complete. 2024-09-12T12:23:16.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 790090 7.64 MB/sec execute 411 sec latency 22734.288 ms 2024-09-12T12:23:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:16 smithi137 ceph-mon[30950]: pgmap v316: 97 pgs: 97 active+clean; 230 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-09-12T12:23:17.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:16 smithi189 ceph-mon[35475]: pgmap v316: 97 pgs: 97 active+clean; 230 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-09-12T12:23:17.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9aaa6358', total seconds elapsed: 26.5 2024-09-12T12:23:17.165 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T12:23:17.165 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:23:17.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:16 smithi171 ceph-mon[35098]: pgmap v316: 97 pgs: 97 active+clean; 230 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.3 MiB/s wr, 93 op/s 2024-09-12T12:23:17.436 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:23:17.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 792218 7.66 MB/sec execute 412 sec latency 14.330 ms 2024-09-12T12:23:17.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.937+0000 7fad3a282640 1 -- 172.21.15.137:0/4240282685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 msgr2=0x7fad3410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.937+0000 7fad3a282640 1 --2- 172.21.15.137:0/4240282685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 0x7fad3410ec40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009cd0 tx=0x7fad2c02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 -- 172.21.15.137:0/4240282685 shutdown_connections 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 --2- 172.21.15.137:0/4240282685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 0x7fad3410ec40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 --2- 172.21.15.137:0/4240282685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad341036e0 0x7fad34107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 --2- 172.21.15.137:0/4240282685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad34102d30 0x7fad34103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 -- 172.21.15.137:0/4240282685 >> 172.21.15.137:0/4240282685 conn(0x7fad340fc7d0 msgr2=0x7fad340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 -- 172.21.15.137:0/4240282685 shutdown_connections 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 -- 172.21.15.137:0/4240282685 wait complete. 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 Processor -- start 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 -- start start 2024-09-12T12:23:17.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad34102d30 0x7fad34198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.938+0000 7fad3a282640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad341036e0 0x7fad34198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3a282640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 0x7fad3419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3a282640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad341120a0 con 0x7fad341082a0 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3a282640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fad34111f20 con 0x7fad341036e0 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3a282640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad34112220 con 0x7fad34102d30 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad33fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad34102d30 0x7fad34198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3881f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 0x7fad3419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad33fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad34102d30 0x7fad34198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:57060/0 (socket says 172.21.15.137:57060) 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad33fff640 1 -- 172.21.15.137:0/805358552 learned_addr learned my addr 172.21.15.137:0/805358552 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad337fe640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad341036e0 0x7fad34198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3881f640 1 -- 172.21.15.137:0/805358552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad34102d30 msgr2=0x7fad34198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3881f640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad34102d30 0x7fad34198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3881f640 1 -- 172.21.15.137:0/805358552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad341036e0 msgr2=0x7fad34198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3881f640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad341036e0 0x7fad34198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3881f640 1 -- 172.21.15.137:0/805358552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3419d4f0 con 0x7fad341082a0 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3881f640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 0x7fad3419ce10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fad2c02ae00 tx=0x7fad2c031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad317fa640 1 -- 172.21.15.137:0/805358552 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c004040 con 0x7fad341082a0 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3419d780 con 0x7fad341082a0 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad317fa640 1 -- 172.21.15.137:0/805358552 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad2c0046c0 con 0x7fad341082a0 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad317fa640 1 -- 172.21.15.137:0/805358552 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c030e40 con 0x7fad341082a0 2024-09-12T12:23:17.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.939+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad341a50c0 con 0x7fad341082a0 2024-09-12T12:23:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.941+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad34103110 con 0x7fad341082a0 2024-09-12T12:23:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.942+0000 7fad317fa640 1 -- 172.21.15.137:0/805358552 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fad2c0305a0 con 0x7fad341082a0 2024-09-12T12:23:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.942+0000 7fad317fa640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fad1c0a2cf0 0x7fad1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.942+0000 7fad33fff640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fad1c0a2cf0 0x7fad1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.942+0000 7fad317fa640 1 -- 172.21.15.137:0/805358552 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fad2c02e030 con 0x7fad341082a0 2024-09-12T12:23:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.943+0000 7fad33fff640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fad1c0a2cf0 0x7fad1c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad24009c90 tx=0x7fad24005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:17.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:17.947+0000 7fad317fa640 1 -- 172.21.15.137:0/805358552 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad2c04b060 con 0x7fad341082a0 2024-09-12T12:23:18.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.165+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad341a5290 con 0x7fad341082a0 2024-09-12T12:23:18.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.166+0000 7fad317fa640 1 -- 172.21.15.137:0/805358552 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11936 (secure 0 0 0) 0x7fad2c0bd6c0 con 0x7fad341082a0 2024-09-12T12:23:18.167 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:23:18.167 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":37,"btime":"2024-09-12T12:22:28:627072+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:22:28.158799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:23:18.167 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 37 2024-09-12T12:23:18.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fad1c0a2cf0 msgr2=0x7fad1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fad1c0a2cf0 0x7fad1c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fad24009c90 tx=0x7fad24005f90 comp rx=0 tx=0).stop 2024-09-12T12:23:18.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 msgr2=0x7fad3419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:18.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 0x7fad3419ce10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fad2c02ae00 tx=0x7fad2c031e60 comp rx=0 tx=0).stop 2024-09-12T12:23:18.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 shutdown_connections 2024-09-12T12:23:18.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fad1c0a2cf0 0x7fad1c0a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad341082a0 0x7fad3419ce10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad341036e0 0x7fad34198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 --2- 172.21.15.137:0/805358552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad34102d30 0x7fad34198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:18.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 >> 172.21.15.137:0/805358552 conn(0x7fad340fc7d0 msgr2=0x7fad34105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:18.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.168+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 shutdown_connections 2024-09-12T12:23:18.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:18.169+0000 7fad3a282640 1 -- 172.21.15.137:0/805358552 wait complete. 2024-09-12T12:23:18.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 793767 7.65 MB/sec execute 413 sec latency 35.924 ms 2024-09-12T12:23:18.926 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T12:23:18.926 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T12:23:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:18 smithi137 ceph-mon[30950]: pgmap v317: 97 pgs: 97 active+clean; 230 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-09-12T12:23:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:18 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/805358552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:18 smithi189 ceph-mon[35475]: pgmap v317: 97 pgs: 97 active+clean; 230 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-09-12T12:23:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:18 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/805358552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:19.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:18 smithi171 ceph-mon[35098]: pgmap v317: 97 pgs: 97 active+clean; 230 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-09-12T12:23:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:18 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/805358552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:19.205 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:23:19.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795827 7.67 MB/sec execute 414 sec latency 19.497 ms 2024-09-12T12:23:19.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 -- 172.21.15.137:0/2317907032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 msgr2=0x7f262410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 --2- 172.21.15.137:0/2317907032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 0x7f262410da40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f261c009ca0 tx=0x7f261c02a950 comp rx=0 tx=0).stop 2024-09-12T12:23:19.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 -- 172.21.15.137:0/2317907032 shutdown_connections 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 --2- 172.21.15.137:0/2317907032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 0x7f262410da40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 --2- 172.21.15.137:0/2317907032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2624102570 0x7f26241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 --2- 172.21.15.137:0/2317907032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 0x7f2624101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 -- 172.21.15.137:0/2317907032 >> 172.21.15.137:0/2317907032 conn(0x7f26240fc7d0 msgr2=0x7f26240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 -- 172.21.15.137:0/2317907032 shutdown_connections 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.793+0000 7f262aa0e640 1 -- 172.21.15.137:0/2317907032 wait complete. 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 Processor -- start 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 -- start start 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 0x7f26240fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2624102570 0x7f26240fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 0x7f26240fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2624110f80 con 0x7f2624102ef0 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2624110e00 con 0x7f2624100e20 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f262aa0e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2624111100 con 0x7f2624102570 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2628fab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 0x7f26240fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2623fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 0x7f26240fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f26237fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2624102570 0x7f26240fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2623fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 0x7f26240fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39418/0 (socket says 172.21.15.137:39418) 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2623fff640 1 -- 172.21.15.137:0/475200667 learned_addr learned my addr 172.21.15.137:0/475200667 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2628fab640 1 -- 172.21.15.137:0/475200667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2624102570 msgr2=0x7f26240fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2628fab640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2624102570 0x7f26240fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2628fab640 1 -- 172.21.15.137:0/475200667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 msgr2=0x7f26240fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2628fab640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 0x7f26240fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2628fab640 1 -- 172.21.15.137:0/475200667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26241a6970 con 0x7f2624102ef0 2024-09-12T12:23:19.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f2623fff640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 0x7f26240fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.794+0000 7f26237fe640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2624102570 0x7f26240fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.795+0000 7f2628fab640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 0x7f26240fd700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f261c005cc0 tx=0x7f261c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.795+0000 7f26217fa640 1 -- 172.21.15.137:0/475200667 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c034040 con 0x7f2624102ef0 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.795+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26241a6c00 con 0x7f2624102ef0 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.795+0000 7f26217fa640 1 -- 172.21.15.137:0/475200667 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261c030920 con 0x7f2624102ef0 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.795+0000 7f26217fa640 1 -- 172.21.15.137:0/475200667 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c0045b0 con 0x7f2624102ef0 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.795+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26241a7130 con 0x7f2624102ef0 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.797+0000 7f26217fa640 1 -- 172.21.15.137:0/475200667 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f261c037020 con 0x7f2624102ef0 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.797+0000 7f262cde7740 1 Processor -- start 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.797+0000 7f262cde7740 1 -- start start 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f262cde7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf42dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f262cde7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42e170 0x55c7cf4384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f262cde7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf43ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f262cde7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c7cf392160 con 0x55c7cf438a30 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f262cde7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c7cf339720 con 0x55c7cf42e170 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f262cde7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7cf38e120 con 0x55c7cf42d850 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf42dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26217fa640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26100a2cf0 0x7f26100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf42dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:40042/0 (socket says 172.21.15.137:40042) 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 -- 172.21.15.137:0/3262722174 learned_addr learned my addr 172.21.15.137:0/3262722174 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f2623fff640 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42e170 0x55c7cf4384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf43ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f2628fab640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26100a2cf0 0x7f26100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 -- 172.21.15.137:0/3262722174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42e170 msgr2=0x55c7cf4384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42e170 0x55c7cf4384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 -- 172.21.15.137:0/3262722174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 msgr2=0x55c7cf43ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26217fa640 1 -- 172.21.15.137:0/475200667 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f261c0f00d0 con 0x7f2624102ef0 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf43ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 -- 172.21.15.137:0/3262722174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7cf103ba0 con 0x55c7cf42d850 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f2623fff640 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42e170 0x55c7cf4384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f26237fe640 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf42dc30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f260c0058c0 tx=0x7f260c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15dc3e9a85b3803c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.798+0000 7f2628fab640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26100a2cf0 0x7f26100a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f261c0bc650 tx=0x7f261c004040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f261a7fc640 1 -- 172.21.15.137:0/3262722174 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c00fc20 con 0x55c7cf42d850 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f261a7fc640 1 -- 172.21.15.137:0/3262722174 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260c00fdc0 con 0x55c7cf42d850 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f261a7fc640 1 -- 172.21.15.137:0/3262722174 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c003b00 con 0x55c7cf42d850 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 -- 172.21.15.137:0/3262722174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 msgr2=0x55c7cf42dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf42dc30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f260c0058c0 tx=0x7f260c005850 comp rx=0 tx=0).stop 2024-09-12T12:23:19.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 -- 172.21.15.137:0/3262722174 shutdown_connections 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf43ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42e170 0x55c7cf4384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 --2- 172.21.15.137:0/3262722174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf42dc30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 -- 172.21.15.137:0/3262722174 >> 172.21.15.137:0/3262722174 conn(0x55c7cf429de0 msgr2=0x55c7cf42a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 -- 172.21.15.137:0/3262722174 shutdown_connections 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.799+0000 7f262cde7740 1 -- 172.21.15.137:0/3262722174 wait complete. 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 -- start start 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 Processor -- start 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42d850 0x55c7cf435850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf4362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c7cf43b600 con 0x55c7cf438a30 2024-09-12T12:23:19.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c7cf31a900 con 0x55c7cf42d850 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7cf43b780 con 0x55c7cf42e170 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f262cde7740 10 client.0 _ll_register_callbacks cb 0x55c7cf318780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f26237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf4362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f2623fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f26237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf4362d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49634/0 (socket says 172.21.15.137:49634) 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f2623fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:40056/0 (socket says 172.21.15.137:40056) 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f26237fe640 1 -- 172.21.15.137:0/2066085529 learned_addr learned my addr 172.21.15.137:0/2066085529 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.800+0000 7f26237fe640 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42d850 0x55c7cf435850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f2623fff640 1 -- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42d850 msgr2=0x55c7cf435850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f2623fff640 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42d850 0x55c7cf435850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f2623fff640 1 -- 172.21.15.137:0/2066085529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 msgr2=0x55c7cf4362d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f2623fff640 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf4362d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f2623fff640 1 -- 172.21.15.137:0/2066085529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7cf35ac50 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f2623fff640 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55c7cf42ac40 tx=0x7f261400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=479df54c08301c07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f26037fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f26037fe640 1 -- 172.21.15.137:0/2066085529 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400fe60 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f260c00fdc0 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f26037fe640 1 -- 172.21.15.137:0/2066085529 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f2614004020 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f26037fe640 1 -- 172.21.15.137:0/2066085529 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2614004220 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f262cde7740 10 client.26110 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c7cf4472f0 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.801+0000 7f26037fe640 1 -- 172.21.15.137:0/2066085529 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260c00fdc0 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f26037fe640 1 -- 172.21.15.137:0/2066085529 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f261401c470 con 0x55c7cf42e170 2024-09-12T12:23:19.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f262cde7740 10 client.26110 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f262cde7740 10 client.26110 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f262cde7740 20 client.26110 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f262cde7740 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf448800 0x55c7cf44ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f262cde7740 4 client.26110 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7cf39f890 con 0x55c7cf448800 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f26237fe640 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf448800 0x55c7cf44ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f26037fe640 10 client.26110 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.802+0000 7f26237fe640 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf448800 0x55c7cf44ac20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24305193afb6b78e server_cookie=dee3f376937dba85 in_seq=0 out_seq=0 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f26037fe640 1 -- 172.21.15.137:0/2066085529 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7cf39f890 con 0x55c7cf448800 2024-09-12T12:23:19.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f26037fe640 10 client.26110 handle_command_reply: tid=0 2024-09-12T12:23:19.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 client.26110 shutdown 2024-09-12T12:23:19.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 msgr2=0x55c7cf435d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x55c7cf42ac40 tx=0x7f261400f790 comp rx=0 tx=0).stop 2024-09-12T12:23:19.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 shutdown_connections 2024-09-12T12:23:19.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf448800 0x55c7cf44ac20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf438a30 0x55c7cf4362d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f26037fe640 0 client.26110 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 --2- 172.21.15.137:0/2066085529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf42d850 0x55c7cf435850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 >> 172.21.15.137:0/2066085529 conn(0x55c7cf429de0 msgr2=0x55c7cf42fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 shutdown_connections 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 1 -- 172.21.15.137:0/2066085529 wait complete. 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.803+0000 7f262cde7740 20 client.26110 trim_cache size 0 max 16384 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 Processor -- start 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 -- start start 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf437720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42e170 0x55c7cf424d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26241b2260 con 0x55c7cf42e170 2024-09-12T12:23:19.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26241afad0 con 0x55c7cf438a30 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f262cde7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26241ad940 con 0x55c7cf42d850 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42e170 0x55c7cf424d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f26237fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf437720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42e170 0x55c7cf424d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49648/0 (socket says 172.21.15.137:49648) 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 -- 172.21.15.137:0/524128084 learned_addr learned my addr 172.21.15.137:0/524128084 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f26237fe640 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 -- 172.21.15.137:0/524128084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 msgr2=0x55c7cf437720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf437720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 -- 172.21.15.137:0/524128084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 msgr2=0x55c7cf436120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f2623fff640 1 -- 172.21.15.137:0/524128084 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7cf433da0 con 0x55c7cf42e170 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.820+0000 7f26237fe640 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:19.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f2623fff640 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42e170 0x55c7cf424d30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f26140097f0 tx=0x7f2614012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dce7465233fb132 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f2619ffb640 1 -- 172.21.15.137:0/524128084 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614029030 con 0x55c7cf42e170 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f2619ffb640 1 -- 172.21.15.137:0/524128084 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261401c470 con 0x55c7cf42e170 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f2619ffb640 1 -- 172.21.15.137:0/524128084 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614010590 con 0x55c7cf42e170 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f262cde7740 1 -- 172.21.15.137:0/524128084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42e170 msgr2=0x55c7cf424d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f262cde7740 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42e170 0x55c7cf424d30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f26140097f0 tx=0x7f2614012880 comp rx=0 tx=0).stop 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f262cde7740 1 -- 172.21.15.137:0/524128084 shutdown_connections 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f262cde7740 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f262cde7740 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42e170 0x55c7cf424d30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.821+0000 7f262cde7740 1 --2- 172.21.15.137:0/524128084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42d850 0x55c7cf437720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 -- 172.21.15.137:0/524128084 >> 172.21.15.137:0/524128084 conn(0x55c7cf429de0 msgr2=0x55c7cf434810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 -- 172.21.15.137:0/524128084 shutdown_connections 2024-09-12T12:23:19.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 -- 172.21.15.137:0/524128084 wait complete. 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 -- start start 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 Processor -- start 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42d850 0x55c7cf4357c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26241adf90 con 0x55c7cf42d850 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26241af400 con 0x55c7cf438a30 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26241ae660 con 0x55c7cf42e170 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.822+0000 7f262cde7740 10 client.0 _ll_register_callbacks cb 0x7f2614009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42d850 0x55c7cf4357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f2623fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42d850 0x55c7cf4357c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49660/0 (socket says 172.21.15.137:49660) 2024-09-12T12:23:19.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26237fe640 1 -- 172.21.15.137:0/533979352 learned_addr learned my addr 172.21.15.137:0/533979352 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26237fe640 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f2623fff640 1 -- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 msgr2=0x55c7cf436240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f2623fff640 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f2623fff640 1 -- 172.21.15.137:0/533979352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42d850 msgr2=0x55c7cf4357c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f2623fff640 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42d850 0x55c7cf4357c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f2623fff640 1 -- 172.21.15.137:0/533979352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7cf43ef20 con 0x55c7cf42e170 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26237fe640 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42d850 0x55c7cf4357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f2623fff640 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f2614012970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfebb022d546494b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26037fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26037fe640 1 -- 172.21.15.137:0/533979352 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614032070 con 0x55c7cf42e170 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f26037fe640 1 -- 172.21.15.137:0/533979352 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f2614010430 con 0x55c7cf42e170 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.823+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f261401c470 con 0x55c7cf42e170 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f26037fe640 1 -- 172.21.15.137:0/533979352 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f261403e050 con 0x55c7cf42e170 2024-09-12T12:23:19.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 10 client.26116 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c7cf4b3730 con 0x55c7cf42e170 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f26037fe640 1 -- 172.21.15.137:0/533979352 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261401c470 con 0x55c7cf42e170 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f26037fe640 1 -- 172.21.15.137:0/533979352 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f261402e7b0 con 0x55c7cf42e170 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 10 client.26116 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 10 client.26116 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 20 client.26116 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf4b3db0 0x55c7cf4b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 4 client.26116 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55c7cf399be0 con 0x55c7cf4b3db0 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.824+0000 7f26237fe640 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf4b3db0 0x55c7cf4b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.825+0000 7f26237fe640 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf4b3db0 0x55c7cf4b61d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432a554293def837 server_cookie=8cdf819817ee99fb in_seq=0 out_seq=0 2024-09-12T12:23:19.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.825+0000 7f26037fe640 10 client.26116 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:19.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f26037fe640 1 -- 172.21.15.137:0/533979352 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c7cf399be0 con 0x55c7cf4b3db0 2024-09-12T12:23:19.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f26037fe640 10 client.26116 handle_command_reply: tid=0 2024-09-12T12:23:19.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 client.26116 shutdown 2024-09-12T12:23:19.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 msgr2=0x55c7cf435d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2614000c00 tx=0x7f2614012970 comp rx=0 tx=0).stop 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 shutdown_connections 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c7cf438a30 0x55c7cf436240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7cf42e170 0x55c7cf435d00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c7cf4b3db0 0x55c7cf4b61d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 --2- 172.21.15.137:0/533979352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c7cf42d850 0x55c7cf4357c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f26037fe640 0 client.26116 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 >> 172.21.15.137:0/533979352 conn(0x55c7cf429de0 msgr2=0x55c7cf42f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 shutdown_connections 2024-09-12T12:23:19.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 1 -- 172.21.15.137:0/533979352 wait complete. 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.896+0000 7f262cde7740 20 client.26116 trim_cache size 0 max 16384 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 37, 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 42, 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stdout: "304f6872": { 2024-09-12T12:23:19.899 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 42, 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 142.6, 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:23:19.900 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:19.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:19.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:19.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:19.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:19.901 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:23:19.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26100a2cf0 msgr2=0x7f26100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26100a2cf0 0x7f26100a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f261c0bc650 tx=0x7f261c004040 comp rx=0 tx=0).stop 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 msgr2=0x7f26240fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 0x7f26240fd700 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f261c005cc0 tx=0x7f261c02f3a0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 shutdown_connections 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26100a2cf0 0x7f26100a51b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2624102ef0 0x7f26240fd700 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2624102570 0x7f26240fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 --2- 172.21.15.137:0/475200667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2624100e20 0x7f26240fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:19.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 >> 172.21.15.137:0/475200667 conn(0x7f26240fc7d0 msgr2=0x7f262406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:19.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 shutdown_connections 2024-09-12T12:23:19.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:19.897+0000 7f262aa0e640 1 -- 172.21.15.137:0/475200667 wait complete. 2024-09-12T12:23:20.625 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '304f6872', quiesce took 3.5 seconds. Will release after: 19.05 2024-09-12T12:23:20.747 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.65 MB/sec execute 415 sec latency 880.921 ms 2024-09-12T12:23:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:20 smithi137 ceph-mon[30950]: pgmap v318: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-09-12T12:23:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:20 smithi189 ceph-mon[35475]: pgmap v318: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-09-12T12:23:21.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:20 smithi171 ceph-mon[35098]: pgmap v318: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-09-12T12:23:21.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.63 MB/sec execute 416 sec latency 1881.027 ms 2024-09-12T12:23:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 -- 172.21.15.137:0/3327029591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c1035c0 msgr2=0x7fd77c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 --2- 172.21.15.137:0/3327029591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c1035c0 0x7fd77c10baa0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009900 tx=0x7fd76c02a410 comp rx=0 tx=0).stop 2024-09-12T12:23:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 -- 172.21.15.137:0/3327029591 shutdown_connections 2024-09-12T12:23:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 --2- 172.21.15.137:0/3327029591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd77c10c0d0 0x7fd77c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 --2- 172.21.15.137:0/3327029591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c1035c0 0x7fd77c10baa0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 --2- 172.21.15.137:0/3327029591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd77c102ca0 0x7fd77c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 -- 172.21.15.137:0/3327029591 >> 172.21.15.137:0/3327029591 conn(0x7fd77c0fc6b0 msgr2=0x7fd77c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 -- 172.21.15.137:0/3327029591 shutdown_connections 2024-09-12T12:23:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.989+0000 7fd782a67640 1 -- 172.21.15.137:0/3327029591 wait complete. 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 Processor -- start 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 -- start start 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd77c102ca0 0x7fd77c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd77c1035c0 0x7fd77c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c10c0d0 0x7fd77c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c111ed0 con 0x7fd77c10c0d0 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c111d50 con 0x7fd77c102ca0 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd782a67640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c112050 con 0x7fd77c1035c0 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd781004640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c10c0d0 0x7fd77c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd781004640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c10c0d0 0x7fd77c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49674/0 (socket says 172.21.15.137:49674) 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.990+0000 7fd781004640 1 -- 172.21.15.137:0/1130290844 learned_addr learned my addr 172.21.15.137:0/1130290844 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd77bfff640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd77c1035c0 0x7fd77c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd780803640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd77c102ca0 0x7fd77c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd781004640 1 -- 172.21.15.137:0/1130290844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd77c1035c0 msgr2=0x7fd77c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd781004640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd77c1035c0 0x7fd77c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd781004640 1 -- 172.21.15.137:0/1130290844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd77c102ca0 msgr2=0x7fd77c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd781004640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd77c102ca0 0x7fd77c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd781004640 1 -- 172.21.15.137:0/1130290844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c19d350 con 0x7fd77c10c0d0 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd781004640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c10c0d0 0x7fd77c19cc20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd77400eaa0 tx=0x7fd77400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd779ffb640 1 -- 172.21.15.137:0/1130290844 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77400cdb0 con 0x7fd77c10c0d0 2024-09-12T12:23:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c19d640 con 0x7fd77c10c0d0 2024-09-12T12:23:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd779ffb640 1 -- 172.21.15.137:0/1130290844 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd774005020 con 0x7fd77c10c0d0 2024-09-12T12:23:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd779ffb640 1 -- 172.21.15.137:0/1130290844 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7740116b0 con 0x7fd77c10c0d0 2024-09-12T12:23:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.991+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c1a4ed0 con 0x7fd77c10c0d0 2024-09-12T12:23:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.993+0000 7fd779ffb640 1 -- 172.21.15.137:0/1130290844 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd774005560 con 0x7fd77c10c0d0 2024-09-12T12:23:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 Processor -- start 2024-09-12T12:23:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 -- start start 2024-09-12T12:23:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x560299585ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x560299590770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd779ffb640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7680a2da0 0x7fd7680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x5602995930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5602994ea2f0 con 0x560299590cb0 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560299492270 con 0x560299585af0 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd7857ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5602994e62b0 con 0x560299586410 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd781004640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x5602995930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd781004640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x5602995930a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49690/0 (socket says 172.21.15.137:49690) 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd781004640 1 -- 172.21.15.137:0/169859518 learned_addr learned my addr 172.21.15.137:0/169859518 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd780803640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7680a2da0 0x7fd7680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd77bfff640 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x560299590770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.994+0000 7fd779ffb640 1 -- 172.21.15.137:0/1130290844 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fd7740cf6a0 con 0x7fd77c10c0d0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd77bfff640 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x560299585ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd781004640 1 -- 172.21.15.137:0/169859518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 msgr2=0x560299590770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd781004640 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x560299590770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd781004640 1 -- 172.21.15.137:0/169859518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 msgr2=0x560299585ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd781004640 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x560299585ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd781004640 1 -- 172.21.15.137:0/169859518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560299491340 con 0x560299590cb0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd77bfff640 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x560299590770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd781004640 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x5602995930a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd7740cce10 tx=0x7fd77406b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb76fdcff46371b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd780803640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7680a2da0 0x7fd7680a5260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd764005fd0 tx=0x7fd764005e20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd772ffd640 1 -- 172.21.15.137:0/169859518 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77401f040 con 0x560299590cb0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd772ffd640 1 -- 172.21.15.137:0/169859518 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77409c280 con 0x560299590cb0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.995+0000 7fd772ffd640 1 -- 172.21.15.137:0/169859518 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7740a47f0 con 0x560299590cb0 2024-09-12T12:23:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 -- 172.21.15.137:0/169859518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 msgr2=0x5602995930a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x5602995930a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd7740cce10 tx=0x7fd77406b9e0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 -- 172.21.15.137:0/169859518 shutdown_connections 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x5602995930a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x560299590770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 --2- 172.21.15.137:0/169859518 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x560299585ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 -- 172.21.15.137:0/169859518 >> 172.21.15.137:0/169859518 conn(0x560299582060 msgr2=0x560299582440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 -- 172.21.15.137:0/169859518 shutdown_connections 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 -- 172.21.15.137:0/169859518 wait complete. 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 -- start start 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.996+0000 7fd7857ff740 1 Processor -- start 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x56029958dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x56029958e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x56029958e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560299593880 con 0x560299590cb0 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560299473290 con 0x560299585af0 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560299593a00 con 0x560299586410 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 10 client.0 _ll_register_callbacks cb 0x5602994a4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x56029958e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x56029958e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49700/0 (socket says 172.21.15.137:49700) 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 -- 172.21.15.137:0/55480555 learned_addr learned my addr 172.21.15.137:0/55480555 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd780803640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x56029958dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd77bfff640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x56029958e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 -- 172.21.15.137:0/55480555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 msgr2=0x56029958e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x56029958e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 -- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 msgr2=0x56029958dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x56029958dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 -- 172.21.15.137:0/55480555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602991fd210 con 0x560299590cb0 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd77bfff640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x56029958e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd780803640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x56029958dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd781004640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x56029958e6a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x5602995869b0 tx=0x7fd7740a0070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218fa343b41161bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd75bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd75bfff640 1 -- 172.21.15.137:0/55480555 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77401f050 con 0x560299590cb0 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd75bfff640 1 -- 172.21.15.137:0/55480555 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fd77409fcd0 con 0x560299590cb0 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77409c280 con 0x560299590cb0 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.997+0000 7fd75bfff640 1 -- 172.21.15.137:0/55480555 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd774065300 con 0x560299590cb0 2024-09-12T12:23:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 10 client.16695 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5602992c9620 con 0x560299590cb0 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd75bfff640 1 -- 172.21.15.137:0/55480555 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77409c280 con 0x560299590cb0 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd75bfff640 1 -- 172.21.15.137:0/55480555 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd77400f040 con 0x560299590cb0 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 10 client.16695 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 20 client.16695 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5602995a0b00 0x5602995a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 4 client.16695 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602994f7a20 con 0x5602995a0b00 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.998+0000 7fd780803640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5602995a0b00 0x5602995a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd780803640 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5602995a0b00 0x5602995a2ef0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79c29f13c9aafcec server_cookie=b5239dc6160ab43a in_seq=0 out_seq=0 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd75bfff640 10 client.16695 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd75bfff640 1 -- 172.21.15.137:0/55480555 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602994f7a20 con 0x5602995a0b00 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd75bfff640 10 client.16695 handle_command_reply: tid=0 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 client.16695 shutdown 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 msgr2=0x56029958e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x56029958e6a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x5602995869b0 tx=0x7fd7740a0070 comp rx=0 tx=0).stop 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 shutdown_connections 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5602995a0b00 0x5602995a2ef0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299590cb0 0x56029958e6a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299586410 0x56029958e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd75bfff640 0 client.16695 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 --2- 172.21.15.137:0/55480555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x56029958dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 >> 172.21.15.137:0/55480555 conn(0x560299582060 msgr2=0x560299587dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 shutdown_connections 2024-09-12T12:23:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:21.999+0000 7fd7857ff740 1 -- 172.21.15.137:0/55480555 wait complete. 2024-09-12T12:23:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.000+0000 7fd7857ff740 20 client.16695 trim_cache size 0 max 16384 2024-09-12T12:23:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 Processor -- start 2024-09-12T12:23:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 -- start start 2024-09-12T12:23:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299585af0 0x56029958c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x56029958e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299590cb0 0x5602995a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1afff0 con 0x560299586410 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1ad860 con 0x560299590cb0 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd7857ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1ab6d0 con 0x560299585af0 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x56029958e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x56029958e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49702/0 (socket says 172.21.15.137:49702) 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 -- 172.21.15.137:0/1040835243 learned_addr learned my addr 172.21.15.137:0/1040835243 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd780803640 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299585af0 0x56029958c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 -- 172.21.15.137:0/1040835243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299585af0 msgr2=0x56029958c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299585af0 0x56029958c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd781004640 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299590cb0 0x5602995a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 -- 172.21.15.137:0/1040835243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299590cb0 msgr2=0x5602995a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299590cb0 0x5602995a5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 -- 172.21.15.137:0/1040835243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560299586240 con 0x560299586410 2024-09-12T12:23:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.014+0000 7fd77bfff640 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x56029958e4d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009900 tx=0x7fd76c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8b2c78d7ff21af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7727fc640 1 -- 172.21.15.137:0/1040835243 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c035030 con 0x560299586410 2024-09-12T12:23:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7727fc640 1 -- 172.21.15.137:0/1040835243 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76c0317b0 con 0x560299586410 2024-09-12T12:23:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7727fc640 1 -- 172.21.15.137:0/1040835243 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0047a0 con 0x560299586410 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 -- 172.21.15.137:0/1040835243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 msgr2=0x56029958e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x56029958e4d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009900 tx=0x7fd76c02a410 comp rx=0 tx=0).stop 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 -- 172.21.15.137:0/1040835243 shutdown_connections 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299590cb0 0x5602995a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x56029958e4d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1040835243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299585af0 0x56029958c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 -- 172.21.15.137:0/1040835243 >> 172.21.15.137:0/1040835243 conn(0x560299582060 msgr2=0x56029958fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 -- 172.21.15.137:0/1040835243 shutdown_connections 2024-09-12T12:23:22.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 -- 172.21.15.137:0/1040835243 wait complete. 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 -- start start 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.015+0000 7fd7857ff740 1 Processor -- start 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x5602995a6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x5602995a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd7857ff740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299590cb0 0x5602995a4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd7857ff740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1abd20 con 0x560299586410 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd7857ff740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1ad190 con 0x560299585af0 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd7857ff740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1ac3f0 con 0x560299590cb0 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd7857ff740 10 client.0 _ll_register_callbacks cb 0x7fd77409b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x5602995a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x5602995a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49706/0 (socket says 172.21.15.137:49706) 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 -- 172.21.15.137:0/1251784292 learned_addr learned my addr 172.21.15.137:0/1251784292 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd780803640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x5602995a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd781004640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299590cb0 0x5602995a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 -- 172.21.15.137:0/1251784292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299590cb0 msgr2=0x5602995a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299590cb0 0x5602995a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 -- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 msgr2=0x5602995a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x5602995a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 -- 172.21.15.137:0/1251784292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029960e850 con 0x560299586410 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd781004640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299590cb0 0x5602995a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd77bfff640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x5602995a3b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd76c032fd0 tx=0x7fd76c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546e01deb2c304aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.016+0000 7fd75bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd75bfff640 1 -- 172.21.15.137:0/1251784292 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c04d070 con 0x560299586410 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd75bfff640 1 -- 172.21.15.137:0/1251784292 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fd76c004240 con 0x560299586410 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd76c0317b0 con 0x560299586410 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd75bfff640 1 -- 172.21.15.137:0/1251784292 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd76c059050 con 0x560299586410 2024-09-12T12:23:22.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 10 client.16701 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56029960fa30 con 0x560299586410 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd75bfff640 1 -- 172.21.15.137:0/1251784292 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76c0317b0 con 0x560299586410 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd75bfff640 1 -- 172.21.15.137:0/1251784292 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fd76c003860 con 0x560299586410 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 10 client.16701 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 20 client.16701 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560299610160 0x560299612580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 4 client.16701 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602994f1d70 con 0x560299610160 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.017+0000 7fd780803640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560299610160 0x560299612580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.018+0000 7fd75bfff640 10 client.16701 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:22.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.018+0000 7fd780803640 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560299610160 0x560299612580 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6b58a991669136b server_cookie=2e0ccdb9dcfa584d in_seq=0 out_seq=0 2024-09-12T12:23:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd75bfff640 1 -- 172.21.15.137:0/1251784292 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602994f1d70 con 0x560299610160 2024-09-12T12:23:22.058 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd75bfff640 10 client.16701 handle_command_reply: tid=0 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 client.16701 shutdown 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 msgr2=0x5602995a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x5602995a3b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd76c032fd0 tx=0x7fd76c0315b0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 shutdown_connections 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560299590cb0 0x5602995a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560299610160 0x560299612580 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560299586410 0x5602995a3b50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 --2- 172.21.15.137:0/1251784292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560299585af0 0x5602995a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd75bfff640 0 client.16701 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 >> 172.21.15.137:0/1251784292 conn(0x560299582060 msgr2=0x5602995809a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 shutdown_connections 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 1 -- 172.21.15.137:0/1251784292 wait complete. 2024-09-12T12:23:22.059 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.057+0000 7fd7857ff740 20 client.16701 trim_cache size 0 max 16384 2024-09-12T12:23:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:23:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:23:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:23:22.060 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7680a2da0 msgr2=0x7fd7680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7680a2da0 0x7fd7680a5260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd764005fd0 tx=0x7fd764005e20 comp rx=0 tx=0).stop 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c10c0d0 msgr2=0x7fd77c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c10c0d0 0x7fd77c19cc20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd77400eaa0 tx=0x7fd77400ef60 comp rx=0 tx=0).stop 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 shutdown_connections 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd7680a2da0 0x7fd7680a5260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd77c10c0d0 0x7fd77c19cc20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.061 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd77c1035c0 0x7fd77c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 --2- 172.21.15.137:0/1130290844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd77c102ca0 0x7fd77c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 >> 172.21.15.137:0/1130290844 conn(0x7fd77c0fc6b0 msgr2=0x7fd77c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 shutdown_connections 2024-09-12T12:23:22.062 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:22.058+0000 7fd782a67640 1 -- 172.21.15.137:0/1130290844 wait complete. 2024-09-12T12:23:22.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.61 MB/sec execute 417 sec latency 2881.125 ms 2024-09-12T12:23:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:22 smithi137 ceph-mon[30950]: pgmap v319: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-09-12T12:23:23.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:22 smithi189 ceph-mon[35475]: pgmap v319: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-09-12T12:23:23.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:22 smithi171 ceph-mon[35098]: pgmap v319: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-09-12T12:23:23.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.59 MB/sec execute 418 sec latency 3881.204 ms 2024-09-12T12:23:24.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.58 MB/sec execute 419 sec latency 4881.290 ms 2024-09-12T12:23:24.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:24 smithi189 ceph-mon[35475]: pgmap v320: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-12T12:23:24.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:24 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:23:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:24 smithi171 ceph-mon[35098]: pgmap v320: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-12T12:23:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:24 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:23:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:24 smithi137 ceph-mon[30950]: pgmap v320: 97 pgs: 97 active+clean; 226 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 135 op/s 2024-09-12T12:23:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:24 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:23:25.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.56 MB/sec execute 420 sec latency 5881.329 ms 2024-09-12T12:23:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:23:26.701 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:23:26.702 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.54 MB/sec execute 421 sec latency 6881.426 ms 2024-09-12T12:23:26.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:23:26.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:23:26.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:23:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:27 smithi171 ceph-mon[35098]: pgmap v321: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-09-12T12:23:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:27 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:23:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:27 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:23:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:27 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:23:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:27 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:23:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:27 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:23:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:27 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:23:27.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.52 MB/sec execute 422 sec latency 7881.541 ms 2024-09-12T12:23:27.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:27 smithi137 ceph-mon[30950]: pgmap v321: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-09-12T12:23:27.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:27 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:23:27.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:27 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:23:27.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:27 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:23:27.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:27 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:23:27.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:27 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:23:27.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:27 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:23:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:27 smithi189 ceph-mon[35475]: pgmap v321: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 165 op/s 2024-09-12T12:23:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:27 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:23:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:27 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:23:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:27 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:23:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:27 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:23:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:27 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:23:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:27 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:23:28.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.50 MB/sec execute 423 sec latency 8881.628 ms 2024-09-12T12:23:29.643 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:29 smithi189 ceph-mon[35475]: pgmap v322: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-09-12T12:23:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:29 smithi171 ceph-mon[35098]: pgmap v322: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-09-12T12:23:29.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.49 MB/sec execute 424 sec latency 9881.713 ms 2024-09-12T12:23:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:29 smithi137 ceph-mon[30950]: pgmap v322: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-09-12T12:23:30.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.47 MB/sec execute 425 sec latency 10881.799 ms 2024-09-12T12:23:31.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:31 smithi171 ceph-mon[35098]: pgmap v323: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-12T12:23:31.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.45 MB/sec execute 426 sec latency 11881.900 ms 2024-09-12T12:23:31.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:31 smithi137 ceph-mon[30950]: pgmap v323: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-12T12:23:31.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:31 smithi189 ceph-mon[35475]: pgmap v323: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-09-12T12:23:32.748 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.43 MB/sec execute 427 sec latency 12881.981 ms 2024-09-12T12:23:33.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:33 smithi171 ceph-mon[35098]: pgmap v324: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 739 KiB/s wr, 57 op/s 2024-09-12T12:23:33.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.42 MB/sec execute 428 sec latency 13882.069 ms 2024-09-12T12:23:33.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:33 smithi137 ceph-mon[30950]: pgmap v324: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 739 KiB/s wr, 57 op/s 2024-09-12T12:23:33.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:33 smithi189 ceph-mon[35475]: pgmap v324: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 739 KiB/s wr, 57 op/s 2024-09-12T12:23:34.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.40 MB/sec execute 429 sec latency 14882.170 ms 2024-09-12T12:23:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 -- 172.21.15.137:0/380562611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 msgr2=0x7f62d0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 --2- 172.21.15.137:0/380562611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d0100c60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009ca0 tx=0x7f62c002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 -- 172.21.15.137:0/380562611 shutdown_connections 2024-09-12T12:23:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 --2- 172.21.15.137:0/380562611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62d010e430 0x7f62d0110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 --2- 172.21.15.137:0/380562611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62d0101230 0x7f62d010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 --2- 172.21.15.137:0/380562611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d0100c60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 -- 172.21.15.137:0/380562611 >> 172.21.15.137:0/380562611 conn(0x7f62d0077990 msgr2=0x7f62d00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 -- 172.21.15.137:0/380562611 shutdown_connections 2024-09-12T12:23:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.154+0000 7f62d841e640 1 -- 172.21.15.137:0/380562611 wait complete. 2024-09-12T12:23:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 Processor -- start 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 -- start start 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62d0101230 0x7f62d019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62d010e430 0x7f62d019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01140e0 con 0x7f62d0100880 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0113f60 con 0x7f62d0101230 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d841e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0114260 con 0x7f62d010e430 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53906/0 (socket says 172.21.15.137:53906) 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 -- 172.21.15.137:0/2426499481 learned_addr learned my addr 172.21.15.137:0/2426499481 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d59b9640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62d0101230 0x7f62d019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d69bb640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62d010e430 0x7f62d019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 -- 172.21.15.137:0/2426499481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62d010e430 msgr2=0x7f62d019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62d010e430 0x7f62d019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 -- 172.21.15.137:0/2426499481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62d0101230 msgr2=0x7f62d019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62d0101230 0x7f62d019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.155+0000 7f62d61ba640 1 -- 172.21.15.137:0/2426499481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d019f580 con 0x7f62d0100880 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.156+0000 7f62d61ba640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d019a590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f62c00097b0 tx=0x7f62c0031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.156+0000 7f62cf7fe640 1 -- 172.21.15.137:0/2426499481 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c002fba0 con 0x7f62d0100880 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.156+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d019f810 con 0x7f62d0100880 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.156+0000 7f62cf7fe640 1 -- 172.21.15.137:0/2426499481 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c002fd40 con 0x7f62d0100880 2024-09-12T12:23:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.156+0000 7f62cf7fe640 1 -- 172.21.15.137:0/2426499481 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c003b860 con 0x7f62d0100880 2024-09-12T12:23:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.156+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d01a7100 con 0x7f62d0100880 2024-09-12T12:23:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.158+0000 7f62cf7fe640 1 -- 172.21.15.137:0/2426499481 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f62c0030c30 con 0x7f62d0100880 2024-09-12T12:23:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 Processor -- start 2024-09-12T12:23:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 -- start start 2024-09-12T12:23:35.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd41770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd41cb0 0x56520bd440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56520bc9b2f0 con 0x56520bd36af0 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56520bc43270 con 0x56520bd41cb0 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.159+0000 7f62db28e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56520bc972b0 con 0x56520bd37410 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d69bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd41770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd41770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:34742/0 (socket says 172.21.15.137:34742) 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 -- 172.21.15.137:0/291564001 learned_addr learned my addr 172.21.15.137:0/291564001 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d69bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd36ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53908/0 (socket says 172.21.15.137:53908) 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d59b9640 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd41cb0 0x56520bd440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62cf7fe640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62b80a2da0 0x7f62b80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d69bb640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62b80a2da0 0x7f62b80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 -- 172.21.15.137:0/291564001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd41cb0 msgr2=0x56520bd440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd41cb0 0x56520bd440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 -- 172.21.15.137:0/291564001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 msgr2=0x56520bd36ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd36ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 -- 172.21.15.137:0/291564001 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56520bc42340 con 0x56520bd37410 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d59b9640 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd41cb0 0x56520bd440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d69bb640 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd36ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62cf7fe640 1 -- 172.21.15.137:0/2426499481 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f62c00f0980 con 0x7f62d0100880 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d61ba640 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd41770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ee1a0 tx=0x7f62c00bca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58e162d786bc3a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62d69bb640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62b80a2da0 0x7f62b80a5260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005fd0 tx=0x7f62c8004730 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62ccbfe640 1 -- 172.21.15.137:0/291564001 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0043330 con 0x56520bd37410 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62ccbfe640 1 -- 172.21.15.137:0/291564001 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0036070 con 0x56520bd37410 2024-09-12T12:23:35.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.160+0000 7f62ccbfe640 1 -- 172.21.15.137:0/291564001 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00c5c40 con 0x56520bd37410 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 -- 172.21.15.137:0/291564001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 msgr2=0x56520bd41770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd41770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ee1a0 tx=0x7f62c00bca60 comp rx=0 tx=0).stop 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 -- 172.21.15.137:0/291564001 shutdown_connections 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd41cb0 0x56520bd440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd41770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 --2- 172.21.15.137:0/291564001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd36ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 -- 172.21.15.137:0/291564001 >> 172.21.15.137:0/291564001 conn(0x56520bd33060 msgr2=0x56520bd33440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 -- 172.21.15.137:0/291564001 shutdown_connections 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.161+0000 7f62db28e740 1 -- 172.21.15.137:0/291564001 wait complete. 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 -- start start 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 Processor -- start 2024-09-12T12:23:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd3ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd37410 0x56520bd3f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 0x56520bd3f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56520bc972b0 con 0x56520bd37410 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56520bc24290 con 0x56520bd36af0 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56520bd44640 con 0x56520bd41cb0 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62db28e740 10 client.0 _ll_register_callbacks cb 0x56520bc55140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62d59b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd37410 0x56520bd3f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62d59b9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd37410 0x56520bd3f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53924/0 (socket says 172.21.15.137:53924) 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62d59b9640 1 -- 172.21.15.137:0/1346130228 learned_addr learned my addr 172.21.15.137:0/1346130228 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62d61ba640 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd3ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.162+0000 7f62d69bb640 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 0x56520bd3f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62d59b9640 1 -- 172.21.15.137:0/1346130228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 msgr2=0x56520bd3f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62d59b9640 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 0x56520bd3f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62d59b9640 1 -- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 msgr2=0x56520bd3ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62d59b9640 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd3ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62d59b9640 1 -- 172.21.15.137:0/1346130228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56520b9ae210 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62d59b9640 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd37410 0x56520bd3f0f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f62c4010820 tx=0x7f62c4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377ce27d72006cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62b5ffb640 1 -- 172.21.15.137:0/1346130228 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c401e060 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62b5ffb640 1 -- 172.21.15.137:0/1346130228 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f62c400fb90 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c0036070 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62b5ffb640 1 -- 172.21.15.137:0/1346130228 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f62c400fd90 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62db28e740 10 client.16713 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56520ba7a620 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62b5ffb640 1 -- 172.21.15.137:0/1346130228 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0036070 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.163+0000 7f62b5ffb640 1 -- 172.21.15.137:0/1346130228 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f62c4021550 con 0x56520bd37410 2024-09-12T12:23:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62db28e740 10 client.16713 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62db28e740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62db28e740 20 client.16713 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62db28e740 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bd51800 0x56520bd53bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62db28e740 4 client.16713 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56520bca8a20 con 0x56520bd51800 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62d61ba640 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bd51800 0x56520bd53bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62d61ba640 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bd51800 0x56520bd53bf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd181026995ee36 server_cookie=f68cb64545f60fa0 in_seq=0 out_seq=0 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.164+0000 7f62b5ffb640 10 client.16713 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62b5ffb640 1 -- 172.21.15.137:0/1346130228 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56520bca8a20 con 0x56520bd51800 2024-09-12T12:23:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62b5ffb640 10 client.16713 handle_command_reply: tid=0 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 client.16713 shutdown 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd37410 msgr2=0x56520bd3f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd37410 0x56520bd3f0f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f62c4010820 tx=0x7f62c4010c20 comp rx=0 tx=0).stop 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 shutdown_connections 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 0x56520bd3f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bd51800 0x56520bd53bf0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd37410 0x56520bd3f0f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62b5ffb640 0 client.16713 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 --2- 172.21.15.137:0/1346130228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd3ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 >> 172.21.15.137:0/1346130228 conn(0x56520bd33060 msgr2=0x56520bd38dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 shutdown_connections 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 1 -- 172.21.15.137:0/1346130228 wait complete. 2024-09-12T12:23:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.165+0000 7f62db28e740 20 client.16713 trim_cache size 0 max 16384 2024-09-12T12:23:35.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.179+0000 7f62db28e740 1 Processor -- start 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.179+0000 7f62db28e740 1 -- start start 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62db28e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd39560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62db28e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd37410 0x56520bd3f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62db28e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 0x56520bd59500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62db28e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01b2250 con 0x56520bd36af0 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62db28e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01afac0 con 0x56520bd37410 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62db28e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01ad930 con 0x56520bd41cb0 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd39560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd39560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53940/0 (socket says 172.21.15.137:53940) 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 -- 172.21.15.137:0/4056184772 learned_addr learned my addr 172.21.15.137:0/4056184772 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d59b9640 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd37410 0x56520bd3f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 -- 172.21.15.137:0/4056184772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 msgr2=0x56520bd59500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 0x56520bd59500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 -- 172.21.15.137:0/4056184772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd37410 msgr2=0x56520bd3f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd37410 0x56520bd3f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 -- 172.21.15.137:0/4056184772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56520bd2e2d0 con 0x56520bd36af0 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d59b9640 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd37410 0x56520bd3f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62d61ba640 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd39560 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009a30 tx=0x7f62c00b93a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88547670a13233c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62b7fff640 1 -- 172.21.15.137:0/4056184772 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00bcfd0 con 0x56520bd36af0 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62b7fff640 1 -- 172.21.15.137:0/4056184772 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0036040 con 0x56520bd36af0 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.180+0000 7f62b7fff640 1 -- 172.21.15.137:0/4056184772 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c00ee2b0 con 0x56520bd36af0 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- 172.21.15.137:0/4056184772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 msgr2=0x56520bd39560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd39560 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009a30 tx=0x7f62c00b93a0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- 172.21.15.137:0/4056184772 shutdown_connections 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd41cb0 0x56520bd59500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd37410 0x56520bd3f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 --2- 172.21.15.137:0/4056184772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd36af0 0x56520bd39560 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- 172.21.15.137:0/4056184772 >> 172.21.15.137:0/4056184772 conn(0x56520bd33060 msgr2=0x56520bd40c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- 172.21.15.137:0/4056184772 shutdown_connections 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- 172.21.15.137:0/4056184772 wait complete. 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- start start 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 Processor -- start 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd49d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd45040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd41cb0 0x56520bd45580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01adf80 con 0x56520bd41cb0 2024-09-12T12:23:35.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01af3f0 con 0x56520bd36af0 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.181+0000 7f62db28e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01ae650 con 0x56520bd37410 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62db28e740 10 client.0 _ll_register_callbacks cb 0x7f62c400a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd41cb0 0x56520bd45580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd41cb0 0x56520bd45580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53954/0 (socket says 172.21.15.137:53954) 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 -- 172.21.15.137:0/2567461793 learned_addr learned my addr 172.21.15.137:0/2567461793 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d61ba640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd49d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d59b9640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd45040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 -- 172.21.15.137:0/2567461793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 msgr2=0x56520bd45040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd45040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 -- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 msgr2=0x56520bd49d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd49d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 -- 172.21.15.137:0/2567461793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56520bd45ac0 con 0x56520bd41cb0 2024-09-12T12:23:35.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d61ba640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd49d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62d69bb640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd41cb0 0x56520bd45580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f62c8011ea0 tx=0x7f62c800a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad73624a2ee6922 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62b5ffb640 1 -- 172.21.15.137:0/2567461793 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8019070 con 0x56520bd41cb0 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62b5ffb640 1 -- 172.21.15.137:0/2567461793 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f62c8005020 con 0x56520bd41cb0 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62c0036040 con 0x56520bd41cb0 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62b5ffb640 1 -- 172.21.15.137:0/2567461793 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f62c8005220 con 0x56520bd41cb0 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62db28e740 10 client.16719 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.182+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56520bdbc230 con 0x56520bd41cb0 2024-09-12T12:23:35.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62b5ffb640 1 -- 172.21.15.137:0/2567461793 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0036040 con 0x56520bd41cb0 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62b5ffb640 1 -- 172.21.15.137:0/2567461793 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f62c8031020 con 0x56520bd41cb0 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62db28e740 10 client.16719 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62db28e740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62db28e740 20 client.16719 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62db28e740 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bdbc9b0 0x56520bdbedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62db28e740 4 client.16719 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56520bca2d70 con 0x56520bdbc9b0 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62d61ba640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bdbc9b0 0x56520bdbedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62b5ffb640 10 client.16719 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.183+0000 7f62d61ba640 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bdbc9b0 0x56520bdbedd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=addf2af6f5818ff0 server_cookie=f1a93c60f4d4e10f in_seq=0 out_seq=0 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.210+0000 7f62b5ffb640 1 -- 172.21.15.137:0/2567461793 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56520bca2d70 con 0x56520bdbc9b0 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.210+0000 7f62b5ffb640 10 client.16719 handle_command_reply: tid=0 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.210+0000 7f62db28e740 1 client.16719 shutdown 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd41cb0 msgr2=0x56520bd45580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd41cb0 0x56520bd45580 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f62c8011ea0 tx=0x7f62c800a970 comp rx=0 tx=0).stop 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 shutdown_connections 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56520bdbc9b0 0x56520bdbedd0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56520bd41cb0 0x56520bd45580 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62b5ffb640 0 client.16719 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56520bd37410 0x56520bd45040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 --2- 172.21.15.137:0/2567461793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56520bd36af0 0x56520bd49d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 >> 172.21.15.137:0/2567461793 conn(0x56520bd33060 msgr2=0x56520bd319a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 shutdown_connections 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 1 -- 172.21.15.137:0/2567461793 wait complete. 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62db28e740 20 client.16719 trim_cache size 0 max 16384 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:23:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:23:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62b80a2da0 msgr2=0x7f62b80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62b80a2da0 0x7f62b80a5260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62c8005fd0 tx=0x7f62c8004730 comp rx=0 tx=0).stop 2024-09-12T12:23:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 msgr2=0x7f62d019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d019a590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f62c00097b0 tx=0x7f62c0031b10 comp rx=0 tx=0).stop 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 shutdown_connections 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62b80a2da0 0x7f62b80a5260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62d010e430 0x7f62d019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62d0101230 0x7f62d019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 --2- 172.21.15.137:0/2426499481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62d0100880 0x7f62d019a590 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.211+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 >> 172.21.15.137:0/2426499481 conn(0x7f62d0077990 msgr2=0x7f62d00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.212+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 shutdown_connections 2024-09-12T12:23:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:35.212+0000 7f62d841e640 1 -- 172.21.15.137:0/2426499481 wait complete. 2024-09-12T12:23:35.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:35 smithi171 ceph-mon[35098]: pgmap v325: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 739 KiB/s wr, 57 op/s 2024-09-12T12:23:35.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.38 MB/sec execute 430 sec latency 15882.272 ms 2024-09-12T12:23:35.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:35 smithi137 ceph-mon[30950]: pgmap v325: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 739 KiB/s wr, 57 op/s 2024-09-12T12:23:35.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:35 smithi189 ceph-mon[35475]: pgmap v325: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 739 KiB/s wr, 57 op/s 2024-09-12T12:23:36.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.36 MB/sec execute 431 sec latency 16882.401 ms 2024-09-12T12:23:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:37 smithi171 ceph-mon[35098]: pgmap v326: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 772 KiB/s wr, 62 op/s 2024-09-12T12:23:37.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.35 MB/sec execute 432 sec latency 17882.440 ms 2024-09-12T12:23:37.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:37 smithi137 ceph-mon[30950]: pgmap v326: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 772 KiB/s wr, 62 op/s 2024-09-12T12:23:37.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:37 smithi189 ceph-mon[35475]: pgmap v326: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 772 KiB/s wr, 62 op/s 2024-09-12T12:23:38.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.33 MB/sec execute 433 sec latency 18882.519 ms 2024-09-12T12:23:39.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:23:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:39 smithi171 ceph-mon[35098]: pgmap v327: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 152 KiB/s wr, 20 op/s 2024-09-12T12:23:39.701 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:39 smithi137 ceph-mon[30950]: pgmap v327: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 152 KiB/s wr, 20 op/s 2024-09-12T12:23:39.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.31 MB/sec execute 434 sec latency 19882.575 ms 2024-09-12T12:23:39.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:39 smithi189 ceph-mon[35475]: pgmap v327: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 152 KiB/s wr, 20 op/s 2024-09-12T12:23:39.933 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:23:40.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.438+0000 7f89daaa6640 1 -- 172.21.15.137:0/536484539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102ef0 msgr2=0x7f89d410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:40.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.438+0000 7f89daaa6640 1 --2- 172.21.15.137:0/536484539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102ef0 0x7f89d410da40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009ca0 tx=0x7f89cc02a950 comp rx=0 tx=0).stop 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.438+0000 7f89daaa6640 1 -- 172.21.15.137:0/536484539 shutdown_connections 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.438+0000 7f89daaa6640 1 --2- 172.21.15.137:0/536484539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102ef0 0x7f89d410da40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.438+0000 7f89daaa6640 1 --2- 172.21.15.137:0/536484539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89d4102570 0x7f89d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.438+0000 7f89daaa6640 1 --2- 172.21.15.137:0/536484539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d4100e20 0x7f89d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.438+0000 7f89daaa6640 1 -- 172.21.15.137:0/536484539 >> 172.21.15.137:0/536484539 conn(0x7f89d40fc7d0 msgr2=0x7f89d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 -- 172.21.15.137:0/536484539 shutdown_connections 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 -- 172.21.15.137:0/536484539 wait complete. 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 Processor -- start 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 -- start start 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89d4100e20 0x7f89d40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102570 0x7f89d40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:40.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d4102ef0 0x7f89d40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4110f80 con 0x7f89d4102570 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4110e00 con 0x7f89d4100e20 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.439+0000 7f89daaa6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89d4111100 con 0x7f89d4102ef0 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102570 0x7f89d40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102570 0x7f89d40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43006/0 (socket says 172.21.15.137:43006) 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 -- 172.21.15.137:0/1309254298 learned_addr learned my addr 172.21.15.137:0/1309254298 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d8842640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89d4100e20 0x7f89d40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d9043640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d4102ef0 0x7f89d40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 -- 172.21.15.137:0/1309254298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d4102ef0 msgr2=0x7f89d40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d4102ef0 0x7f89d40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 -- 172.21.15.137:0/1309254298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89d4100e20 msgr2=0x7f89d40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:40.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89d4100e20 0x7f89d40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 -- 172.21.15.137:0/1309254298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d41a68d0 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d3fff640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102570 0x7f89d40fd1c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c920 tx=0x7f89c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d1ffb640 1 -- 172.21.15.137:0/1309254298 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0007c70 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d1ffb640 1 -- 172.21.15.137:0/1309254298 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c0007e10 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d41a6bc0 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.440+0000 7f89d1ffb640 1 -- 172.21.15.137:0/1309254298 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c0018890 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.441+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d41a7100 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.441+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d4101200 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.442+0000 7f89d1ffb640 1 -- 172.21.15.137:0/1309254298 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f89c0018a70 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.443+0000 7f89d1ffb640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f89c40a2cf0 0x7f89c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.443+0000 7f89d1ffb640 1 -- 172.21.15.137:0/1309254298 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f89c00cf500 con 0x7f89d4102570 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.443+0000 7f89d8842640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f89c40a2cf0 0x7f89c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:40.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.444+0000 7f89d8842640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f89c40a2cf0 0x7f89c40a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005fd0 tx=0x7f89c800b390 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:40.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.447+0000 7f89d1ffb640 1 -- 172.21.15.137:0/1309254298 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89c0014030 con 0x7f89d4102570 2024-09-12T12:23:40.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.656+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89d410a0a0 con 0x7f89d4102570 2024-09-12T12:23:40.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.656+0000 7f89d1ffb640 1 -- 172.21.15.137:0/1309254298 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11936 (secure 0 0 0) 0x7f89c009c140 con 0x7f89d4102570 2024-09-12T12:23:40.657 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:23:40.657 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":37,"btime":"2024-09-12T12:22:28:627072+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:22:28.158799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:23:40.657 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 37 2024-09-12T12:23:40.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f89c40a2cf0 msgr2=0x7f89c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:40.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f89c40a2cf0 0x7f89c40a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f89c8005fd0 tx=0x7f89c800b390 comp rx=0 tx=0).stop 2024-09-12T12:23:40.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102570 msgr2=0x7f89d40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102570 0x7f89d40fd1c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c920 tx=0x7f89c000cde0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 shutdown_connections 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f89c40a2cf0 0x7f89c40a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89d4102ef0 0x7f89d40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89d4102570 0x7f89d40fd1c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 --2- 172.21.15.137:0/1309254298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89d4100e20 0x7f89d40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 >> 172.21.15.137:0/1309254298 conn(0x7f89d40fc7d0 msgr2=0x7f89d406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 shutdown_connections 2024-09-12T12:23:40.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:40.659+0000 7f89daaa6640 1 -- 172.21.15.137:0/1309254298 wait complete. 2024-09-12T12:23:40.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.30 MB/sec execute 435 sec latency 20882.612 ms 2024-09-12T12:23:41.473 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 304f6872 --release --await' 2024-09-12T12:23:41.473 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 304f6872 --release --await 2024-09-12T12:23:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:41 smithi171 ceph-mon[35098]: pgmap v328: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 176 KiB/s wr, 30 op/s 2024-09-12T12:23:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:41 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1309254298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:41.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 795916 7.28 MB/sec execute 436 sec latency 21882.646 ms 2024-09-12T12:23:41.760 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:23:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:41 smithi137 ceph-mon[30950]: pgmap v328: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 176 KiB/s wr, 30 op/s 2024-09-12T12:23:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:41 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1309254298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:41.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:41 smithi189 ceph-mon[35475]: pgmap v328: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 176 KiB/s wr, 30 op/s 2024-09-12T12:23:41.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:41 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1309254298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.259+0000 7fc872911640 1 -- 172.21.15.137:0/1138564056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 msgr2=0x7fc86c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.259+0000 7fc872911640 1 --2- 172.21.15.137:0/1138564056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c10ea90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc864009ca0 tx=0x7fc86402a8c0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 -- 172.21.15.137:0/1138564056 shutdown_connections 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 --2- 172.21.15.137:0/1138564056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c10ea90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 --2- 172.21.15.137:0/1138564056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc86c102790 0x7fc86c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 --2- 172.21.15.137:0/1138564056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc86c101de0 0x7fc86c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 -- 172.21.15.137:0/1138564056 >> 172.21.15.137:0/1138564056 conn(0x7fc86c0fd8c0 msgr2=0x7fc86c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 -- 172.21.15.137:0/1138564056 shutdown_connections 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 -- 172.21.15.137:0/1138564056 wait complete. 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 Processor -- start 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.260+0000 7fc872911640 1 -- start start 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc86c101de0 0x7fc86c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc86c102790 0x7fc86c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c111f90 con 0x7fc86c106f10 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c111e10 con 0x7fc86c101de0 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c112110 con 0x7fc86c102790 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43026/0 (socket says 172.21.15.137:43026) 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 -- 172.21.15.137:0/2328573641 learned_addr learned my addr 172.21.15.137:0/2328573641 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc86bfff640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc86c101de0 0x7fc86c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc86c102790 0x7fc86c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 -- 172.21.15.137:0/2328573641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc86c102790 msgr2=0x7fc86c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc86c102790 0x7fc86c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 -- 172.21.15.137:0/2328573641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc86c101de0 msgr2=0x7fc86c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc86c101de0 0x7fc86c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 -- 172.21.15.137:0/2328573641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86c1a6910 con 0x7fc86c106f10 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc86bfff640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc86c101de0 0x7fc86c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc870eae640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c074f70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc864031a60 tx=0x7fc864031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc8697fa640 1 -- 172.21.15.137:0/2328573641 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc864003c30 con 0x7fc86c106f10 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86c1a6ba0 con 0x7fc86c106f10 2024-09-12T12:23:42.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc8697fa640 1 -- 172.21.15.137:0/2328573641 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc864003dd0 con 0x7fc86c106f10 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc8697fa640 1 -- 172.21.15.137:0/2328573641 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8640047e0 con 0x7fc86c106f10 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.261+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86c1a7130 con 0x7fc86c106f10 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc8697fa640 1 -- 172.21.15.137:0/2328573641 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc86402f790 con 0x7fc86c106f10 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 Processor -- start 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 -- start start 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a212c060 0x55c1a212c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a2136d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a2137240 0x55c1a2139630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c1a2091620 con 0x55c1a212c980 2024-09-12T12:23:42.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c1a2038e50 con 0x55c1a212c060 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc874cea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c1a208d5e0 con 0x55c1a2137240 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a2136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc8697fa640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8580a2cf0 0x7fc8580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a2136d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43034/0 (socket says 172.21.15.137:43034) 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc86bfff640 1 -- 172.21.15.137:0/1196355126 learned_addr learned my addr 172.21.15.137:0/1196355126 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a2137240 0x55c1a2139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a212c060 0x55c1a212c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:23:42.386 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 37, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 44, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "304f6872": { 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 44, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 142.6, 2024-09-12T12:23:42.387 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:23:42.388 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.264+0000 7fc870eae640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8580a2cf0 0x7fc8580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc8697fa640 1 -- 172.21.15.137:0/2328573641 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fc8640f07d0 con 0x7fc86c106f10 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc86bfff640 1 -- 172.21.15.137:0/1196355126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a2137240 msgr2=0x55c1a2139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc86bfff640 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a2137240 0x55c1a2139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc86bfff640 1 -- 172.21.15.137:0/1196355126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a212c060 msgr2=0x55c1a212c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc86bfff640 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a212c060 0x55c1a212c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc86bfff640 1 -- 172.21.15.137:0/1196355126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a2029d40 con 0x55c1a212c980 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc86bfff640 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a2136d00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc85c007850 tx=0x7fc85c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3a5a63251f7a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc870eae640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8580a2cf0 0x7fc8580a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8640ee100 tx=0x7fc864033040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc8627fc640 1 -- 172.21.15.137:0/1196355126 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00d8d0 con 0x55c1a212c980 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc8627fc640 1 -- 172.21.15.137:0/1196355126 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc85c018070 con 0x55c1a212c980 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.265+0000 7fc8627fc640 1 -- 172.21.15.137:0/1196355126 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0123e0 con 0x55c1a212c980 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 -- 172.21.15.137:0/1196355126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 msgr2=0x55c1a2136d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a2136d00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc85c007850 tx=0x7fc85c007d10 comp rx=0 tx=0).stop 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 -- 172.21.15.137:0/1196355126 shutdown_connections 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a2137240 0x55c1a2139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a2136d00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 --2- 172.21.15.137:0/1196355126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a212c060 0x55c1a212c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 -- 172.21.15.137:0/1196355126 >> 172.21.15.137:0/1196355126 conn(0x55c1a21285f0 msgr2=0x55c1a21289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 -- 172.21.15.137:0/1196355126 shutdown_connections 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 -- 172.21.15.137:0/1196355126 wait complete. 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 -- start start 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.266+0000 7fc874cea740 1 Processor -- start 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c060 0x55c1a2134070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c980 0x55c1a21345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a2134af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c1a2139e10 con 0x55c1a212c060 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c1a2019f10 con 0x55c1a2137240 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c1a2139f90 con 0x55c1a212c980 2024-09-12T12:23:42.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 10 client.0 _ll_register_callbacks cb 0x55c1a203ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:42.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c060 0x55c1a2134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c060 0x55c1a2134070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43050/0 (socket says 172.21.15.137:43050) 2024-09-12T12:23:42.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 -- 172.21.15.137:0/373863183 learned_addr learned my addr 172.21.15.137:0/373863183 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86bfff640 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c980 0x55c1a21345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 -- 172.21.15.137:0/373863183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c980 msgr2=0x55c1a21345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c980 0x55c1a21345b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 -- 172.21.15.137:0/373863183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 msgr2=0x55c1a2134af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a2134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 -- 172.21.15.137:0/373863183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a1da0580 con 0x55c1a212c060 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c060 0x55c1a2134070 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc85400eb20 tx=0x7fc8540108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dbfba76f5ee1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc84b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc84b7fe640 1 -- 172.21.15.137:0/373863183 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854004020 con 0x55c1a212c060 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc84b7fe640 1 -- 172.21.15.137:0/373863183 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fc8540041c0 con 0x55c1a212c060 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85c018070 con 0x55c1a212c060 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc84b7fe640 1 -- 172.21.15.137:0/373863183 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc8540043c0 con 0x55c1a212c060 2024-09-12T12:23:42.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 10 client.16737 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.267+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c1a1e6a960 con 0x55c1a212c060 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc84b7fe640 1 -- 172.21.15.137:0/373863183 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc85c018070 con 0x55c1a212c060 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc84b7fe640 1 -- 172.21.15.137:0/373863183 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fc854004b00 con 0x55c1a212c060 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc874cea740 10 client.16737 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc874cea740 10 client.16737 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc874cea740 20 client.16737 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc874cea740 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a2146d90 0x55c1a21491a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc874cea740 4 client.16737 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1a209eb90 con 0x55c1a2146d90 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.268+0000 7fc86bfff640 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a2146d90 0x55c1a21491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.269+0000 7fc84b7fe640 10 client.16737 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.269+0000 7fc86bfff640 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a2146d90 0x55c1a21491a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f11909ae8134e162 server_cookie=a0e844755e2e5b0f in_seq=0 out_seq=0 2024-09-12T12:23:42.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.269+0000 7fc84b7fe640 1 -- 172.21.15.137:0/373863183 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1a209eb90 con 0x55c1a2146d90 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.269+0000 7fc84b7fe640 10 client.16737 handle_command_reply: tid=0 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.269+0000 7fc874cea740 1 client.16737 shutdown 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.269+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c060 msgr2=0x55c1a2134070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.269+0000 7fc874cea740 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c060 0x55c1a2134070 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc85400eb20 tx=0x7fc8540108b0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 shutdown_connections 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a2134af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c980 0x55c1a21345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a2146d90 0x55c1a21491a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 --2- 172.21.15.137:0/373863183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c060 0x55c1a2134070 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc84b7fe640 0 client.16737 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 >> 172.21.15.137:0/373863183 conn(0x55c1a21285f0 msgr2=0x55c1a2127750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 shutdown_connections 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 1 -- 172.21.15.137:0/373863183 wait complete. 2024-09-12T12:23:42.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.270+0000 7fc874cea740 20 client.16737 trim_cache size 0 max 16384 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.286+0000 7fc874cea740 1 Processor -- start 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.286+0000 7fc874cea740 1 -- start start 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc874cea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a2127640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc874cea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a214ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc874cea740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a214efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc874cea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1b23c0 con 0x55c1a212c980 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc874cea740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1afc30 con 0x55c1a2137240 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc874cea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1adaa0 con 0x55c1a212c060 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a214ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a214ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43058/0 (socket says 172.21.15.137:43058) 2024-09-12T12:23:42.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 -- 172.21.15.137:0/626039698 learned_addr learned my addr 172.21.15.137:0/626039698 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a2127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 -- 172.21.15.137:0/626039698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 msgr2=0x55c1a2127640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a2127640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 -- 172.21.15.137:0/626039698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 msgr2=0x55c1a214efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a214efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 -- 172.21.15.137:0/626039698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a2128420 con 0x55c1a212c980 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a2127640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc86bfff640 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a214ea70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0077a0 tx=0x7fc85c00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd54002d56658af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc861ffb640 1 -- 172.21.15.137:0/626039698 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00d8d0 con 0x55c1a212c980 2024-09-12T12:23:42.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc861ffb640 1 -- 172.21.15.137:0/626039698 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc85c02d070 con 0x55c1a212c980 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.287+0000 7fc861ffb640 1 -- 172.21.15.137:0/626039698 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c028a70 con 0x55c1a212c980 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 -- 172.21.15.137:0/626039698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 msgr2=0x55c1a214ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a214ea70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0077a0 tx=0x7fc85c00f040 comp rx=0 tx=0).stop 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 -- 172.21.15.137:0/626039698 shutdown_connections 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a214efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a214ea70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 --2- 172.21.15.137:0/626039698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a2127640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 -- 172.21.15.137:0/626039698 >> 172.21.15.137:0/626039698 conn(0x55c1a21285f0 msgr2=0x55c1a2126f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 -- 172.21.15.137:0/626039698 shutdown_connections 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 -- 172.21.15.137:0/626039698 wait complete. 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.288+0000 7fc874cea740 1 -- start start 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 1 Processor -- start 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a213f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a213fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a213a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1ae0f0 con 0x55c1a212c980 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1af560 con 0x55c1a2137240 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc86c1ae7c0 con 0x55c1a212c060 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc874cea740 10 client.0 _ll_register_callbacks cb 0x7fc854014af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a213fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a213fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43060/0 (socket says 172.21.15.137:43060) 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 -- 172.21.15.137:0/339555990 learned_addr learned my addr 172.21.15.137:0/339555990 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a213f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a213a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 -- 172.21.15.137:0/339555990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 msgr2=0x55c1a213f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a213f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 -- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 msgr2=0x55c1a213a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a213a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86bfff640 1 -- 172.21.15.137:0/339555990 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a213ac20 con 0x55c1a212c980 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.289+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a213a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc86bfff640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a213fad0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc85c004700 tx=0x7fc85c0072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40153a4d632b497 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc84b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc84b7fe640 1 -- 172.21.15.137:0/339555990 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c028550 con 0x55c1a212c980 2024-09-12T12:23:42.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc84b7fe640 1 -- 172.21.15.137:0/339555990 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fc85c0287f0 con 0x55c1a212c980 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc85c02d070 con 0x55c1a212c980 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc84b7fe640 1 -- 172.21.15.137:0/339555990 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc85c042050 con 0x55c1a212c980 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 10 client.16743 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c1a21b2610 con 0x55c1a212c980 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc84b7fe640 1 -- 172.21.15.137:0/339555990 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc85c02d070 con 0x55c1a212c980 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc84b7fe640 1 -- 172.21.15.137:0/339555990 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fc85c028030 con 0x55c1a212c980 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 10 client.16743 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 10 client.16743 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 20 client.16743 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a21b2ad0 0x55c1a21b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 4 client.16743 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "304f6872", "release": true, "await": true}] 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.290+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "304f6872", "release": true, "await": true}) -- 0x55c1a20990a0 con 0x55c1a21b2ad0 2024-09-12T12:23:42.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.291+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a21b2ad0 0x55c1a21b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.291+0000 7fc86b7fe640 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a21b2ad0 0x55c1a21b4ef0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e016b72e66755dc server_cookie=d3c82866248b4935 in_seq=0 out_seq=0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.291+0000 7fc84b7fe640 10 client.16743 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc84b7fe640 1 -- 172.21.15.137:0/339555990 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c1a20990a0 con 0x55c1a21b2ad0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc84b7fe640 10 client.16743 handle_command_reply: tid=0 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 client.16743 shutdown 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 msgr2=0x55c1a213fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a213fad0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc85c004700 tx=0x7fc85c0072b0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 shutdown_connections 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c1a2137240 0x55c1a213a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c1a21b2ad0 0x55c1a21b4ef0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c1a212c980 0x55c1a213fad0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 --2- 172.21.15.137:0/339555990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c1a212c060 0x55c1a213f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc84b7fe640 0 client.16743 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 >> 172.21.15.137:0/339555990 conn(0x55c1a21285f0 msgr2=0x55c1a2127750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 shutdown_connections 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 1 -- 172.21.15.137:0/339555990 wait complete. 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.298+0000 7fc874cea740 20 client.16743 trim_cache size 0 max 16384 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8580a2cf0 msgr2=0x7fc8580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8580a2cf0 0x7fc8580a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc8640ee100 tx=0x7fc864033040 comp rx=0 tx=0).stop 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 msgr2=0x7fc86c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:42.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c074f70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc864031a60 tx=0x7fc864031a90 comp rx=0 tx=0).stop 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 shutdown_connections 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8580a2cf0 0x7fc8580a51b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc86c106f10 0x7fc86c074f70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc86c102790 0x7fc86c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 --2- 172.21.15.137:0/2328573641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc86c101de0 0x7fc86c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 >> 172.21.15.137:0/2328573641 conn(0x7fc86c0fd8c0 msgr2=0x7fc86c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 shutdown_connections 2024-09-12T12:23:42.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:23:42.299+0000 7fc872911640 1 -- 172.21.15.137:0/2328573641 wait complete. 2024-09-12T12:23:42.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 796844 7.28 MB/sec execute 437 sec latency 22434.165 ms 2024-09-12T12:23:43.219 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '304f6872', total seconds elapsed: 26.1 2024-09-12T12:23:43.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:43 smithi137 ceph-mon[30950]: pgmap v329: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-12T12:23:43.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:43 smithi171 ceph-mon[35098]: pgmap v329: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-12T12:23:43.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 799272 7.29 MB/sec execute 438 sec latency 11.196 ms 2024-09-12T12:23:43.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:43 smithi189 ceph-mon[35475]: pgmap v329: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-12T12:23:44.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 801263 7.29 MB/sec execute 439 sec latency 15.432 ms 2024-09-12T12:23:45.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:45 smithi171 ceph-mon[35098]: pgmap v330: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-12T12:23:45.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 803078 7.30 MB/sec execute 440 sec latency 13.411 ms 2024-09-12T12:23:45.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:45 smithi137 ceph-mon[30950]: pgmap v330: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-12T12:23:45.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:45 smithi189 ceph-mon[35475]: pgmap v330: 97 pgs: 97 active+clean; 224 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 57 KiB/s wr, 14 op/s 2024-09-12T12:23:46.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 805736 7.32 MB/sec execute 441 sec latency 14.980 ms 2024-09-12T12:23:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:47 smithi171 ceph-mon[35098]: pgmap v331: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 475 KiB/s wr, 60 op/s 2024-09-12T12:23:47.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 807357 7.32 MB/sec execute 442 sec latency 18.669 ms 2024-09-12T12:23:47.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:47 smithi137 ceph-mon[30950]: pgmap v331: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 475 KiB/s wr, 60 op/s 2024-09-12T12:23:47.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:47 smithi189 ceph-mon[35475]: pgmap v331: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 475 KiB/s wr, 60 op/s 2024-09-12T12:23:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/1306493171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df41009b0 msgr2=0x7f7df4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/1306493171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df41009b0 0x7f7df4100e10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009930 tx=0x7f7de802a420 comp rx=0 tx=0).stop 2024-09-12T12:23:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/1306493171 shutdown_connections 2024-09-12T12:23:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/1306493171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7df4101350 0x7f7df410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/1306493171 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df41009b0 0x7f7df4100e10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/1306493171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7df41069c0 0x7f7df4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/1306493171 >> 172.21.15.137:0/1306493171 conn(0x7f7df4077990 msgr2=0x7f7df40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/1306493171 shutdown_connections 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.307+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/1306493171 wait complete. 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 Processor -- start 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 -- start start 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7df41009b0 0x7f7df4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df4101350 0x7f7df41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7df41069c0 0x7f7df419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4110e00 con 0x7f7df4101350 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4110c80 con 0x7f7df41069c0 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7dfb7a6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4110f80 con 0x7f7df41009b0 2024-09-12T12:23:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df8d41640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df4101350 0x7f7df41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9542640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7df41009b0 0x7f7df4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df8d41640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df4101350 0x7f7df41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43078/0 (socket says 172.21.15.137:43078) 2024-09-12T12:23:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df8d41640 1 -- 172.21.15.137:0/3572253845 learned_addr learned my addr 172.21.15.137:0/3572253845 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9d43640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7df41069c0 0x7f7df419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9542640 1 -- 172.21.15.137:0/3572253845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7df41069c0 msgr2=0x7f7df419bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9542640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7df41069c0 0x7f7df419bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9542640 1 -- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df4101350 msgr2=0x7f7df41977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9542640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df4101350 0x7f7df41977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9542640 1 -- 172.21.15.137:0/3572253845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df419c280 con 0x7f7df41009b0 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df8d41640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df4101350 0x7f7df41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df9542640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7df41009b0 0x7f7df4197290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7dec010c00 tx=0x7f7dec00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.308+0000 7f7df27fc640 1 -- 172.21.15.137:0/3572253845 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec01d070 con 0x7f7df41009b0 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.309+0000 7f7df27fc640 1 -- 172.21.15.137:0/3572253845 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec018b10 con 0x7f7df41009b0 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.309+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df419c570 con 0x7f7df41009b0 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.309+0000 7f7df27fc640 1 -- 172.21.15.137:0/3572253845 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec016c10 con 0x7f7df41009b0 2024-09-12T12:23:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.309+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df40698a0 con 0x7f7df41009b0 2024-09-12T12:23:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 Processor -- start 2024-09-12T12:23:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df27fc640 1 -- 172.21.15.137:0/3572253845 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7dec020070 con 0x7f7df41009b0 2024-09-12T12:23:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 -- start start 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f83af0 0x55b026f83ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f84410 0x55b026f8e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f8ecb0 0x55b026f910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b026ee82f0 con 0x55b026f83af0 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b026e90270 con 0x55b026f8ecb0 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b026ee42b0 con 0x55b026f84410 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f83af0 0x55b026f83ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f83af0 0x55b026f83ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43094/0 (socket says 172.21.15.137:43094) 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 -- 172.21.15.137:0/963929394 learned_addr learned my addr 172.21.15.137:0/963929394 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9542640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f84410 0x55b026f8e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df8d41640 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f8ecb0 0x55b026f910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 -- 172.21.15.137:0/963929394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f84410 msgr2=0x55b026f8e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f84410 0x55b026f8e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 -- 172.21.15.137:0/963929394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f8ecb0 msgr2=0x55b026f910a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f8ecb0 0x55b026f910a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df9d43640 1 -- 172.21.15.137:0/963929394 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b026e8f340 con 0x55b026f83af0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.312+0000 7f7df27fc640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7df9d43640 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f83af0 0x55b026f83ed0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009fd0 tx=0x7f7de40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49deacf24ce1339 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7df9542640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7ddb7fe640 1 -- 172.21.15.137:0/963929394 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4004d20 con 0x55b026f83af0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7df27fc640 1 -- 172.21.15.137:0/3572253845 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f7dec0d11d0 con 0x7f7df41009b0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7ddb7fe640 1 -- 172.21.15.137:0/963929394 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de4004050 con 0x55b026f83af0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7ddb7fe640 1 -- 172.21.15.137:0/963929394 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4005260 con 0x55b026f83af0 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7dfe61f740 1 -- 172.21.15.137:0/963929394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f83af0 msgr2=0x55b026f83ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7df9542640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0ce9f0 tx=0x7f7dec0d4c60 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.313+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f83af0 0x55b026f83ed0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009fd0 tx=0x7f7de40118d0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- 172.21.15.137:0/963929394 shutdown_connections 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f8ecb0 0x55b026f910a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f84410 0x55b026f8e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/963929394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f83af0 0x55b026f83ed0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- 172.21.15.137:0/963929394 >> 172.21.15.137:0/963929394 conn(0x55b026f80060 msgr2=0x55b026f80440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- 172.21.15.137:0/963929394 shutdown_connections 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- 172.21.15.137:0/963929394 wait complete. 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- start start 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 Processor -- start 2024-09-12T12:23:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f83af0 0x55b026f8bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f84410 0x55b026f8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f8ecb0 0x55b026f8c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b026f91880 con 0x55b026f84410 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b026e71290 con 0x55b026f83af0 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b026f91a00 con 0x55b026f8ecb0 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7dfe61f740 10 client.0 _ll_register_callbacks cb 0x55b026ea2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7df8d41640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f84410 0x55b026f8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f83af0 0x55b026f8bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f83af0 0x55b026f8bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:60130/0 (socket says 172.21.15.137:60130) 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.314+0000 7f7df9d43640 1 -- 172.21.15.137:0/1751106452 learned_addr learned my addr 172.21.15.137:0/1751106452 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:48.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df9d43640 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f8ecb0 0x55b026f8c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df9d43640 1 -- 172.21.15.137:0/1751106452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f8ecb0 msgr2=0x55b026f8c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df9d43640 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f8ecb0 0x55b026f8c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df9d43640 1 -- 172.21.15.137:0/1751106452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f84410 msgr2=0x55b026f8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df9d43640 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f84410 0x55b026f8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df9d43640 1 -- 172.21.15.137:0/1751106452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b026bfb210 con 0x55b026f83af0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df8d41640 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f84410 0x55b026f8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7df9d43640 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f83af0 0x55b026f8bb50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7de40042e0 tx=0x7f7de4005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e89ec0d47ab8228 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/1751106452 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4004750 con 0x55b026f83af0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de4004050 con 0x55b026f83af0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/1751106452 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f7de400f4e0 con 0x55b026f83af0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/1751106452 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7de4036050 con 0x55b026f83af0 2024-09-12T12:23:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dfe61f740 10 client.26102 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b026cc7620 con 0x55b026f83af0 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.315+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/1751106452 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de4004050 con 0x55b026f83af0 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/1751106452 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7de4025040 con 0x55b026f83af0 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dfe61f740 10 client.26102 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dfe61f740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dfe61f740 20 client.26102 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b026f9eb70 0x55b026fa0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dfe61f740 4 client.26102 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b026ef5a20 con 0x55b026f9eb70 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7df8d41640 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b026f9eb70 0x55b026fa0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7dd8ff9640 10 client.26102 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.316+0000 7f7df8d41640 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b026f9eb70 0x55b026fa0f60 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c007c3e77b99c98 server_cookie=64552b2715b51d6f in_seq=0 out_seq=0 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/1751106452 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b026ef5a20 con 0x55b026f9eb70 2024-09-12T12:23:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dd8ff9640 10 client.26102 handle_command_reply: tid=0 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 client.26102 shutdown 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f83af0 msgr2=0x55b026f8bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f83af0 0x55b026f8bb50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7de40042e0 tx=0x7f7de4005bd0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 shutdown_connections 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f8ecb0 0x55b026f8c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b026f9eb70 0x55b026fa0f60 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f84410 0x55b026f8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/1751106452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f83af0 0x55b026f8bb50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dd8ff9640 0 client.26102 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 >> 172.21.15.137:0/1751106452 conn(0x55b026f80060 msgr2=0x55b026f85d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 shutdown_connections 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 1 -- 172.21.15.137:0/1751106452 wait complete. 2024-09-12T12:23:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.317+0000 7f7dfe61f740 20 client.26102 trim_cache size 0 max 16384 2024-09-12T12:23:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 Processor -- start 2024-09-12T12:23:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 -- start start 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f8c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026fa6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026fa6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41adef0 con 0x55b026f8ecb0 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ab760 con 0x55b026f84410 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41a95d0 con 0x55b026f83af0 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f8c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026fa6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f8c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54380/0 (socket says 172.21.15.137:54380) 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 -- 172.21.15.137:0/751728469 learned_addr learned my addr 172.21.15.137:0/751728469 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df8d41640 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026fa6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 -- 172.21.15.137:0/751728469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 msgr2=0x55b026fa6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026fa6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 -- 172.21.15.137:0/751728469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 msgr2=0x55b026fa6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026fa6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 -- 172.21.15.137:0/751728469 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b026f7a2f0 con 0x55b026f83af0 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.332+0000 7f7df9d43640 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f8c400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005d70 tx=0x7f7de40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c72e38123a10c800 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7ddaffd640 1 -- 172.21.15.137:0/751728469 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4028480 con 0x55b026f83af0 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7ddaffd640 1 -- 172.21.15.137:0/751728469 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de402d070 con 0x55b026f83af0 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7ddaffd640 1 -- 172.21.15.137:0/751728469 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de40047b0 con 0x55b026f83af0 2024-09-12T12:23:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 -- 172.21.15.137:0/751728469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 msgr2=0x55b026f8c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f8c400 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7de4005d70 tx=0x7f7de40118d0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 -- 172.21.15.137:0/751728469 shutdown_connections 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026fa6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026fa6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/751728469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f8c400 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 -- 172.21.15.137:0/751728469 >> 172.21.15.137:0/751728469 conn(0x55b026f80060 msgr2=0x55b026f8ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 -- 172.21.15.137:0/751728469 shutdown_connections 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.333+0000 7f7dfe61f740 1 -- 172.21.15.137:0/751728469 wait complete. 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 -- start start 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 Processor -- start 2024-09-12T12:23:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f91c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026f92190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026f926d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41a9c20 con 0x55b026f8ecb0 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ab090 con 0x55b026f84410 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.334+0000 7f7dfe61f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41aa2f0 con 0x55b026f83af0 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7dfe61f740 10 client.0 _ll_register_callbacks cb 0x7f7de40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026f92190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026f926d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026f92190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:60142/0 (socket says 172.21.15.137:60142) 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 -- 172.21.15.137:0/3147933638 learned_addr learned my addr 172.21.15.137:0/3147933638 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df9d43640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026f926d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43114/0 (socket says 172.21.15.137:43114) 2024-09-12T12:23:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df9d43640 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f91c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 -- 172.21.15.137:0/3147933638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 msgr2=0x55b026f91c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f91c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 -- 172.21.15.137:0/3147933638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 msgr2=0x55b026f926d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026f926d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 -- 172.21.15.137:0/3147933638 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0270077b0 con 0x55b026f84410 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7df8d41640 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026f92190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7de80092a0 tx=0x7f7de80041a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=805c746eea5082a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7dd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/3147933638 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8035040 con 0x55b026f84410 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de402d070 con 0x55b026f84410 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.335+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/3147933638 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f7de8005220 con 0x55b026f84410 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/3147933638 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7de8005420 con 0x55b026f84410 2024-09-12T12:23:48.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 10 client.26108 fetch_fsmap learned FSMap version 37 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55b027008b30 con 0x55b026f84410 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/3147933638 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de402d070 con 0x55b026f84410 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/3147933638 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f7de8059020 con 0x55b026f84410 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 10 client.26108 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 10 client.26108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 20 client.26108 populate_metadata read hostname 'smithi137' 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b027009290 0x55b02700b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 4 client.26108 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b026eefd70 con 0x55b027009290 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7df9d43640 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b027009290 0x55b02700b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7dd8ff9640 10 client.26108 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:48.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.336+0000 7f7df9d43640 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b027009290 0x55b02700b6b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc875ff885415a07 server_cookie=53804cf0ca76ecea in_seq=0 out_seq=0 2024-09-12T12:23:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dd8ff9640 1 -- 172.21.15.137:0/3147933638 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b026eefd70 con 0x55b027009290 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dd8ff9640 10 client.26108 handle_command_reply: tid=0 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 client.26108 shutdown 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 msgr2=0x55b026f92190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026f92190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7de80092a0 tx=0x7f7de80041a0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 shutdown_connections 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b027009290 0x55b02700b6b0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b026f8ecb0 0x55b026f926d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b026f84410 0x55b026f92190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dd8ff9640 0 client.26108 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 --2- 172.21.15.137:0/3147933638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b026f83af0 0x55b026f91c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 >> 172.21.15.137:0/3147933638 conn(0x55b026f80060 msgr2=0x55b026f7e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 shutdown_connections 2024-09-12T12:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.391+0000 7f7dfe61f740 1 -- 172.21.15.137:0/3147933638 wait complete. 2024-09-12T12:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.392+0000 7f7dfe61f740 20 client.26108 trim_cache size 0 max 16384 2024-09-12T12:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:23:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:23:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.392+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7ddc0a2da0 msgr2=0x7f7ddc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.392+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0ce9f0 tx=0x7f7dec0d4c60 comp rx=0 tx=0).stop 2024-09-12T12:23:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.392+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7df41009b0 msgr2=0x7f7df4197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:23:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.392+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7df41009b0 0x7f7df4197290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7dec010c00 tx=0x7f7dec00c6a0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 shutdown_connections 2024-09-12T12:23:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7ddc0a2da0 0x7f7ddc0a5260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7df41069c0 0x7f7df419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7df4101350 0x7f7df41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 --2- 172.21.15.137:0/3572253845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7df41009b0 0x7f7df4197290 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:23:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 >> 172.21.15.137:0/3572253845 conn(0x7f7df4077990 msgr2=0x7f7df4103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:23:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 shutdown_connections 2024-09-12T12:23:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:23:48.393+0000 7f7dfb7a6640 1 -- 172.21.15.137:0/3572253845 wait complete. 2024-09-12T12:23:48.749 INFO:tasks.workunit.client.0.smithi137.stdout: 1 809662 7.34 MB/sec execute 443 sec latency 27.306 ms 2024-09-12T12:23:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:49 smithi171 ceph-mon[35098]: pgmap v332: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 442 KiB/s wr, 55 op/s 2024-09-12T12:23:49.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 811021 7.33 MB/sec execute 444 sec latency 39.401 ms 2024-09-12T12:23:49.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:49 smithi137 ceph-mon[30950]: pgmap v332: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 442 KiB/s wr, 55 op/s 2024-09-12T12:23:49.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:49 smithi189 ceph-mon[35475]: pgmap v332: 97 pgs: 97 active+clean; 229 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 442 KiB/s wr, 55 op/s 2024-09-12T12:23:50.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 813411 7.35 MB/sec execute 445 sec latency 45.130 ms 2024-09-12T12:23:51.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:51 smithi171 ceph-mon[35098]: pgmap v333: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-12T12:23:51.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 814908 7.34 MB/sec execute 446 sec latency 16.988 ms 2024-09-12T12:23:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:51 smithi137 ceph-mon[30950]: pgmap v333: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-12T12:23:51.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:51 smithi189 ceph-mon[35475]: pgmap v333: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-12T12:23:52.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 817140 7.36 MB/sec execute 447 sec latency 12.947 ms 2024-09-12T12:23:53.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:53 smithi171 ceph-mon[35098]: pgmap v334: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-12T12:23:53.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 819164 7.36 MB/sec execute 448 sec latency 23.029 ms 2024-09-12T12:23:53.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:53 smithi137 ceph-mon[30950]: pgmap v334: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-12T12:23:53.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:53 smithi189 ceph-mon[35475]: pgmap v334: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-12T12:23:54.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 821019 7.37 MB/sec execute 449 sec latency 22.046 ms 2024-09-12T12:23:55.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:55 smithi171 ceph-mon[35098]: pgmap v335: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-12T12:23:55.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 823769 7.39 MB/sec execute 450 sec latency 16.564 ms 2024-09-12T12:23:55.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:55 smithi137 ceph-mon[30950]: pgmap v335: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-12T12:23:55.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:55 smithi189 ceph-mon[35475]: pgmap v335: 97 pgs: 97 active+clean; 235 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-12T12:23:56.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 825172 7.39 MB/sec execute 451 sec latency 19.014 ms 2024-09-12T12:23:57.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:57 smithi171 ceph-mon[35098]: pgmap v336: 97 pgs: 97 active+clean; 238 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 284 op/s 2024-09-12T12:23:57.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 827530 7.40 MB/sec execute 452 sec latency 18.736 ms 2024-09-12T12:23:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:57 smithi137 ceph-mon[30950]: pgmap v336: 97 pgs: 97 active+clean; 238 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 284 op/s 2024-09-12T12:23:57.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:57 smithi189 ceph-mon[35475]: pgmap v336: 97 pgs: 97 active+clean; 238 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.7 MiB/s wr, 284 op/s 2024-09-12T12:23:58.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 829441 7.40 MB/sec execute 453 sec latency 16.832 ms 2024-09-12T12:23:59.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:23:59 smithi171 ceph-mon[35098]: pgmap v337: 97 pgs: 97 active+clean; 238 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-12T12:23:59.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 831370 7.41 MB/sec execute 454 sec latency 20.915 ms 2024-09-12T12:23:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:23:59 smithi137 ceph-mon[30950]: pgmap v337: 97 pgs: 97 active+clean; 238 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-12T12:23:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:23:59 smithi189 ceph-mon[35475]: pgmap v337: 97 pgs: 97 active+clean; 238 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-12T12:24:00.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 833704 7.42 MB/sec execute 455 sec latency 17.614 ms 2024-09-12T12:24:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.486+0000 7f550a25e640 1 -- 172.21.15.137:0/2616392301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504107b70 msgr2=0x7f5504107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.486+0000 7f550a25e640 1 --2- 172.21.15.137:0/2616392301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504107b70 0x7f5504107f50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009930 tx=0x7f54ec02a420 comp rx=0 tx=0).stop 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.486+0000 7f550a25e640 1 -- 172.21.15.137:0/2616392301 shutdown_connections 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.486+0000 7f550a25e640 1 --2- 172.21.15.137:0/2616392301 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5504102470 0x7f550410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.486+0000 7f550a25e640 1 --2- 172.21.15.137:0/2616392301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5504101ad0 0x7f5504101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.486+0000 7f550a25e640 1 --2- 172.21.15.137:0/2616392301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504107b70 0x7f5504107f50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.487+0000 7f550a25e640 1 -- 172.21.15.137:0/2616392301 >> 172.21.15.137:0/2616392301 conn(0x7f55040fd7d0 msgr2=0x7f55040ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.487+0000 7f550a25e640 1 -- 172.21.15.137:0/2616392301 shutdown_connections 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.487+0000 7f550a25e640 1 -- 172.21.15.137:0/2616392301 wait complete. 2024-09-12T12:24:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.487+0000 7f550a25e640 1 Processor -- start 2024-09-12T12:24:01.490 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.487+0000 7f550a25e640 1 -- start start 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f550a25e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504101ad0 0x7f5504198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f550a25e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5504102470 0x7f5504198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f550a25e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5504107b70 0x7f550419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f550a25e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111f20 con 0x7f5504101ad0 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f550a25e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5504111da0 con 0x7f5504107b70 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f550a25e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55041120a0 con 0x7f5504102470 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504101ad0 0x7f5504198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f5502ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5504102470 0x7f5504198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504101ad0 0x7f5504198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40892/0 (socket says 172.21.15.137:40892) 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 -- 172.21.15.137:0/111835419 learned_addr learned my addr 172.21.15.137:0/111835419 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f5503fff640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5504107b70 0x7f550419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 -- 172.21.15.137:0/111835419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5504102470 msgr2=0x7f5504198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5504102470 0x7f5504198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 -- 172.21.15.137:0/111835419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5504107b70 msgr2=0x7f550419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5504107b70 0x7f550419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 -- 172.21.15.137:0/111835419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550419d340 con 0x7f5504101ad0 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f55037fe640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504101ad0 0x7f5504198350 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009230 tx=0x7f54ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.488+0000 7f5500ff9640 1 -- 172.21.15.137:0/111835419 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec03f030 con 0x7f5504101ad0 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.489+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550419d5d0 con 0x7f5504101ad0 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.489+0000 7f5500ff9640 1 -- 172.21.15.137:0/111835419 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec004440 con 0x7f5504101ad0 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.489+0000 7f5500ff9640 1 -- 172.21.15.137:0/111835419 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec035070 con 0x7f5504101ad0 2024-09-12T12:24:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.489+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55041a4ec0 con 0x7f5504101ad0 2024-09-12T12:24:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.491+0000 7f5500ff9640 1 -- 172.21.15.137:0/111835419 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f54ec042030 con 0x7f5504101ad0 2024-09-12T12:24:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.491+0000 7f550cffa740 1 Processor -- start 2024-09-12T12:24:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.491+0000 7f550cffa740 1 -- start start 2024-09-12T12:24:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f550cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774deed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f550cffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.499 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5500ff9640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54f00a2da0 0x7f54f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f550cffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f550cffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5568774432f0 con 0x5568774e9cb0 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f550cffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5568773eb270 con 0x5568774df410 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f550cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55687743f2b0 con 0x5568774deaf0 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5502ffd640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54f00a2da0 0x7f54f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774deed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:35706/0 (socket says 172.21.15.137:35706) 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 -- 172.21.15.137:0/2165626764 learned_addr learned my addr 172.21.15.137:0/2165626764 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f55037fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5500ff9640 1 -- 172.21.15.137:0/111835419 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f54ec0ef710 con 0x7f5504101ad0 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 -- 172.21.15.137:0/2165626764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 msgr2=0x5568774e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 -- 172.21.15.137:0/2165626764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 msgr2=0x5568774ec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774ec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 -- 172.21.15.137:0/2165626764 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568773ea340 con 0x5568774deaf0 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5502ffd640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54f00a2da0 0x7f54f00a5260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5504107190 tx=0x7f54f4005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f5503fff640 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774deed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005f40 tx=0x7f54fc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a22963440deb253 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f54f9ffb640 1 -- 172.21.15.137:0/2165626764 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc011130 con 0x5568774deaf0 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f54f9ffb640 1 -- 172.21.15.137:0/2165626764 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54fc0112d0 con 0x5568774deaf0 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.492+0000 7f54f9ffb640 1 -- 172.21.15.137:0/2165626764 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00fca0 con 0x5568774deaf0 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 -- 172.21.15.137:0/2165626764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 msgr2=0x5568774deed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774deed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005f40 tx=0x7f54fc005bd0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 -- 172.21.15.137:0/2165626764 shutdown_connections 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 --2- 172.21.15.137:0/2165626764 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774deed0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 -- 172.21.15.137:0/2165626764 >> 172.21.15.137:0/2165626764 conn(0x5568774db060 msgr2=0x5568774db440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 -- 172.21.15.137:0/2165626764 shutdown_connections 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 -- 172.21.15.137:0/2165626764 wait complete. 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.493+0000 7f550cffa740 1 -- start start 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 1 Processor -- start 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5568774ec880 con 0x5568774e9cb0 2024-09-12T12:24:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5568773cc290 con 0x5568774df410 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5568774eca00 con 0x5568774deaf0 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f550cffa740 10 client.0 _ll_register_callbacks cb 0x5568773fd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f55037fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774e75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40912/0 (socket says 172.21.15.137:40912) 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 -- 172.21.15.137:0/2695514052 learned_addr learned my addr 172.21.15.137:0/2695514052 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5502ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 -- 172.21.15.137:0/2695514052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 msgr2=0x5568774e6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774e6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 -- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 msgr2=0x5568774e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5503fff640 1 -- 172.21.15.137:0/2695514052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556877156210 con 0x5568774e9cb0 2024-09-12T12:24:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.494+0000 7f5502ffd640 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f5503fff640 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774e75f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005f10 tx=0x7f54fc004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e600dda7edb0c8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f54e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f54e2ffd640 1 -- 172.21.15.137:0/2695514052 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00fe30 con 0x5568774e9cb0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54fc0112d0 con 0x5568774e9cb0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f54e2ffd640 1 -- 172.21.15.137:0/2695514052 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f54fc003970 con 0x5568774e9cb0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f54e2ffd640 1 -- 172.21.15.137:0/2695514052 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f54fc003b70 con 0x5568774e9cb0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 10 client.16773 fetch_fsmap learned FSMap version 37 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556877222620 con 0x5568774e9cb0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f54e2ffd640 1 -- 172.21.15.137:0/2695514052 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54fc0112d0 con 0x5568774e9cb0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f54e2ffd640 1 -- 172.21.15.137:0/2695514052 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f54fc030020 con 0x5568774e9cb0 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 10 client.16773 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 20 client.16773 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5568774f9a60 0x5568774fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 4 client.16773 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556877450a20 con 0x5568774f9a60 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.495+0000 7f55037fe640 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5568774f9a60 0x5568774fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.496+0000 7f54e2ffd640 10 client.16773 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.496+0000 7f55037fe640 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5568774f9a60 0x5568774fbe50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35dacf1a893b27b3 server_cookie=512860189ed9aa6d in_seq=0 out_seq=0 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f54e2ffd640 1 -- 172.21.15.137:0/2695514052 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556877450a20 con 0x5568774f9a60 2024-09-12T12:24:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f54e2ffd640 10 client.16773 handle_command_reply: tid=0 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 client.16773 shutdown 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 msgr2=0x5568774e75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774e75f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f54fc005f10 tx=0x7f54fc004ec0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 shutdown_connections 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5568774f9a60 0x5568774fbe50 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774e9cb0 0x5568774e75f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f54e2ffd640 0 client.16773 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 --2- 172.21.15.137:0/2695514052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774deaf0 0x5568774e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 >> 172.21.15.137:0/2695514052 conn(0x5568774db060 msgr2=0x5568774e0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 shutdown_connections 2024-09-12T12:24:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 1 -- 172.21.15.137:0/2695514052 wait complete. 2024-09-12T12:24:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.497+0000 7f550cffa740 20 client.16773 trim_cache size 0 max 16384 2024-09-12T12:24:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 Processor -- start 2024-09-12T12:24:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 -- start start 2024-09-12T12:24:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774deaf0 0x5568774e1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x556877501650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55041b0090 con 0x5568774deaf0 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ad900 con 0x5568774df410 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f550cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ab770 con 0x5568774e9cb0 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774deaf0 0x5568774e1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774deaf0 0x5568774e1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40916/0 (socket says 172.21.15.137:40916) 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 -- 172.21.15.137:0/1498636443 learned_addr learned my addr 172.21.15.137:0/1498636443 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f5502ffd640 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f5503fff640 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x556877501650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 -- 172.21.15.137:0/1498636443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 msgr2=0x556877501650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x556877501650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 -- 172.21.15.137:0/1498636443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 msgr2=0x5568774e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 -- 172.21.15.137:0/1498636443 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568774dc110 con 0x5568774deaf0 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.512+0000 7f55037fe640 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774deaf0 0x5568774e1560 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f54ec02a460 tx=0x7f54ec049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbef815c62850c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f54f97fa640 1 -- 172.21.15.137:0/1498636443 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec049040 con 0x5568774deaf0 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f54f97fa640 1 -- 172.21.15.137:0/1498636443 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec0d4e50 con 0x5568774deaf0 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f54f97fa640 1 -- 172.21.15.137:0/1498636443 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec041d80 con 0x5568774deaf0 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 -- 172.21.15.137:0/1498636443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774deaf0 msgr2=0x5568774e1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774deaf0 0x5568774e1560 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f54ec02a460 tx=0x7f54ec049bd0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 -- 172.21.15.137:0/1498636443 shutdown_connections 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x556877501650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774df410 0x5568774e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 --2- 172.21.15.137:0/1498636443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774deaf0 0x5568774e1560 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 -- 172.21.15.137:0/1498636443 >> 172.21.15.137:0/1498636443 conn(0x5568774db060 msgr2=0x5568774e8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.513+0000 7f550cffa740 1 -- 172.21.15.137:0/1498636443 shutdown_connections 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 -- 172.21.15.137:0/1498636443 wait complete. 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 -- start start 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 Processor -- start 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774deaf0 0x5568774f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774df410 0x5568774f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x5568774e3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f55041abdc0 con 0x5568774df410 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ad230 con 0x5568774deaf0 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55041ac490 con 0x5568774e9cb0 2024-09-12T12:24:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f550cffa740 10 client.0 _ll_register_callbacks cb 0x7f54fc0115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f5502ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774df410 0x5568774f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f5502ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774df410 0x5568774f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40918/0 (socket says 172.21.15.137:40918) 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f5502ffd640 1 -- 172.21.15.137:0/3887332652 learned_addr learned my addr 172.21.15.137:0/3887332652 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f5503fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x5568774e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.514+0000 7f55037fe640 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774deaf0 0x5568774f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f5502ffd640 1 -- 172.21.15.137:0/3887332652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 msgr2=0x5568774e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f5502ffd640 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x5568774e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f5502ffd640 1 -- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774deaf0 msgr2=0x5568774f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f5502ffd640 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774deaf0 0x5568774f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f5502ffd640 1 -- 172.21.15.137:0/3887332652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568774e3a50 con 0x5568774df410 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f5503fff640 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x5568774e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f5502ffd640 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774df410 0x5568774f23c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f54f40118d0 tx=0x7f54f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1457ff3569087c7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f54e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f54e2ffd640 1 -- 172.21.15.137:0/3887332652 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f402c060 con 0x5568774df410 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f54e2ffd640 1 -- 172.21.15.137:0/3887332652 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f54f4003b40 con 0x5568774df410 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ec0d4e50 con 0x5568774df410 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f54e2ffd640 1 -- 172.21.15.137:0/3887332652 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f54f4003d40 con 0x5568774df410 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 10 client.16779 fetch_fsmap learned FSMap version 37 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x556877564080 con 0x5568774df410 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f54e2ffd640 1 -- 172.21.15.137:0/3887332652 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec0d4e50 con 0x5568774df410 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f54e2ffd640 1 -- 172.21.15.137:0/3887332652 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f54f4026070 con 0x5568774df410 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 10 client.16779 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 20 client.16779 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556877564860 0x556877566c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 4 client.16779 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:24:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.515+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55687744ad70 con 0x556877564860 2024-09-12T12:24:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.516+0000 7f55037fe640 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556877564860 0x556877566c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.516+0000 7f54e2ffd640 10 client.16779 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:01.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.516+0000 7f55037fe640 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556877564860 0x556877566c80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7c4e458bfce392 server_cookie=4c0f81d96bef29fd in_seq=0 out_seq=0 2024-09-12T12:24:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f54e2ffd640 1 -- 172.21.15.137:0/3887332652 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55687744ad70 con 0x556877564860 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f54e2ffd640 10 client.16779 handle_command_reply: tid=0 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 client.16779 shutdown 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774df410 msgr2=0x5568774f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774df410 0x5568774f23c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f54f40118d0 tx=0x7f54f4004730 comp rx=0 tx=0).stop 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 shutdown_connections 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5568774e9cb0 0x5568774e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556877564860 0x556877566c80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5568774df410 0x5568774f23c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 --2- 172.21.15.137:0/3887332652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5568774deaf0 0x5568774f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f54e2ffd640 0 client.16779 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 >> 172.21.15.137:0/3887332652 conn(0x5568774db060 msgr2=0x5568774d99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 shutdown_connections 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 1 -- 172.21.15.137:0/3887332652 wait complete. 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.594+0000 7f550cffa740 20 client.16779 trim_cache size 0 max 16384 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:24:01.597 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:24:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54f00a2da0 msgr2=0x7f54f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54f00a2da0 0x7f54f00a5260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5504107190 tx=0x7f54f4005f90 comp rx=0 tx=0).stop 2024-09-12T12:24:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504101ad0 msgr2=0x7f5504198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504101ad0 0x7f5504198350 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009230 tx=0x7f54ec004040 comp rx=0 tx=0).stop 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 shutdown_connections 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54f00a2da0 0x7f54f00a5260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5504107b70 0x7f550419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5504102470 0x7f5504198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 --2- 172.21.15.137:0/111835419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5504101ad0 0x7f5504198350 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 >> 172.21.15.137:0/111835419 conn(0x7f55040fd7d0 msgr2=0x7f55040feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 shutdown_connections 2024-09-12T12:24:01.599 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:01.595+0000 7f550a25e640 1 -- 172.21.15.137:0/111835419 wait complete. 2024-09-12T12:24:01.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:01 smithi171 ceph-mon[35098]: pgmap v338: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 338 op/s 2024-09-12T12:24:01.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 834753 7.42 MB/sec execute 456 sec latency 37.571 ms 2024-09-12T12:24:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:01 smithi137 ceph-mon[30950]: pgmap v338: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 338 op/s 2024-09-12T12:24:01.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:01 smithi189 ceph-mon[35475]: pgmap v338: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.2 MiB/s wr, 338 op/s 2024-09-12T12:24:02.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 836010 7.42 MB/sec execute 457 sec latency 36.910 ms 2024-09-12T12:24:03.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:03 smithi171 ceph-mon[35098]: pgmap v339: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-12T12:24:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:03 smithi171 ceph-mon[35098]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T12:24:03.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 838138 7.43 MB/sec execute 458 sec latency 33.556 ms 2024-09-12T12:24:03.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:03 smithi137 ceph-mon[30950]: pgmap v339: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-12T12:24:03.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:03 smithi137 ceph-mon[30950]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T12:24:03.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:03 smithi189 ceph-mon[35475]: pgmap v339: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 205 op/s 2024-09-12T12:24:03.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:03 smithi189 ceph-mon[35475]: osdmap e98: 12 total, 12 up, 12 in 2024-09-12T12:24:04.750 INFO:tasks.workunit.client.0.smithi137.stdout: 1 839556 7.43 MB/sec execute 459 sec latency 17.235 ms 2024-09-12T12:24:05.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:05 smithi171 ceph-mon[35098]: pgmap v341: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-12T12:24:05.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 841840 7.44 MB/sec execute 460 sec latency 13.408 ms 2024-09-12T12:24:05.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:05 smithi137 ceph-mon[30950]: pgmap v341: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-12T12:24:05.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:05 smithi189 ceph-mon[35475]: pgmap v341: 97 pgs: 97 active+clean; 247 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-12T12:24:06.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 843363 7.44 MB/sec execute 461 sec latency 21.134 ms 2024-09-12T12:24:06.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:06 smithi137 ceph-mon[30950]: pgmap v342: 97 pgs: 97 active+clean; 220 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2024-09-12T12:24:06.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:06 smithi189 ceph-mon[35475]: pgmap v342: 97 pgs: 97 active+clean; 220 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2024-09-12T12:24:06.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:06 smithi171 ceph-mon[35098]: pgmap v342: 97 pgs: 97 active+clean; 220 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2024-09-12T12:24:07.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 845432 7.45 MB/sec execute 462 sec latency 12.405 ms 2024-09-12T12:24:08.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 847437 7.45 MB/sec execute 463 sec latency 20.880 ms 2024-09-12T12:24:08.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:08 smithi171 ceph-mon[35098]: pgmap v343: 97 pgs: 97 active+clean; 220 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2024-09-12T12:24:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:08 smithi171 ceph-mon[35098]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T12:24:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:08 smithi137 ceph-mon[30950]: pgmap v343: 97 pgs: 97 active+clean; 220 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2024-09-12T12:24:09.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:08 smithi137 ceph-mon[30950]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T12:24:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:08 smithi189 ceph-mon[35475]: pgmap v343: 97 pgs: 97 active+clean; 220 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.2 MiB/s wr, 324 op/s 2024-09-12T12:24:09.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:08 smithi189 ceph-mon[35475]: osdmap e99: 12 total, 12 up, 12 in 2024-09-12T12:24:09.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 849183 7.46 MB/sec execute 464 sec latency 20.505 ms 2024-09-12T12:24:10.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 851358 7.47 MB/sec execute 465 sec latency 18.597 ms 2024-09-12T12:24:10.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:10 smithi171 ceph-mon[35098]: pgmap v345: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 396 op/s 2024-09-12T12:24:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:10 smithi137 ceph-mon[30950]: pgmap v345: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 396 op/s 2024-09-12T12:24:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:10 smithi189 ceph-mon[35475]: pgmap v345: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.6 MiB/s wr, 396 op/s 2024-09-12T12:24:11.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 852994 7.47 MB/sec execute 466 sec latency 20.982 ms 2024-09-12T12:24:12.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 855449 7.48 MB/sec execute 467 sec latency 22.497 ms 2024-09-12T12:24:12.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:12 smithi171 ceph-mon[35098]: pgmap v346: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 348 op/s 2024-09-12T12:24:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:12 smithi137 ceph-mon[30950]: pgmap v346: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 348 op/s 2024-09-12T12:24:13.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:12 smithi189 ceph-mon[35475]: pgmap v346: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 348 op/s 2024-09-12T12:24:13.751 INFO:tasks.workunit.client.0.smithi137.stdout: 1 856722 7.48 MB/sec execute 468 sec latency 13.723 ms 2024-09-12T12:24:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 -- 172.21.15.137:0/2845750292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4101ad0 msgr2=0x7f6ae4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 --2- 172.21.15.137:0/2845750292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4101ad0 0x7f6ae4101f30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009930 tx=0x7f6ad402a420 comp rx=0 tx=0).stop 2024-09-12T12:24:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 -- 172.21.15.137:0/2845750292 shutdown_connections 2024-09-12T12:24:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 --2- 172.21.15.137:0/2845750292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ae4102470 0x7f6ae410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 --2- 172.21.15.137:0/2845750292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4101ad0 0x7f6ae4101f30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 --2- 172.21.15.137:0/2845750292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ae4107b70 0x7f6ae4107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 -- 172.21.15.137:0/2845750292 >> 172.21.15.137:0/2845750292 conn(0x7f6ae40fd7d0 msgr2=0x7f6ae40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 -- 172.21.15.137:0/2845750292 shutdown_connections 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.691+0000 7f6aec65e640 1 -- 172.21.15.137:0/2845750292 wait complete. 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 Processor -- start 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 -- start start 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ae4101ad0 0x7f6ae4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ae4102470 0x7f6ae4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4107b70 0x7f6ae419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4111ee0 con 0x7f6ae4107b70 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4111d60 con 0x7f6ae4101ad0 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aec65e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4112060 con 0x7f6ae4102470 2024-09-12T12:24:14.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4107b70 0x7f6ae419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ae4101ad0 0x7f6ae4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4107b70 0x7f6ae419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47698/0 (socket says 172.21.15.137:47698) 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 -- 172.21.15.137:0/490543918 learned_addr learned my addr 172.21.15.137:0/490543918 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ae4102470 0x7f6ae4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 -- 172.21.15.137:0/490543918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ae4102470 msgr2=0x7f6ae4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ae4102470 0x7f6ae4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 -- 172.21.15.137:0/490543918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ae4101ad0 msgr2=0x7f6ae4198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ae4101ad0 0x7f6ae4198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aeabfb640 1 -- 172.21.15.137:0/490543918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae419d340 con 0x7f6ae4107b70 2024-09-12T12:24:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.692+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ae4101ad0 0x7f6ae4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.693+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4107b70 0x7f6ae419cc10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00eb10 tx=0x7f6adc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.693+0000 7f6adb7fe640 1 -- 172.21.15.137:0/490543918 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc004b20 con 0x7f6ae4107b70 2024-09-12T12:24:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.693+0000 7f6adb7fe640 1 -- 172.21.15.137:0/490543918 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc004cc0 con 0x7f6ae4107b70 2024-09-12T12:24:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.693+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae419d630 con 0x7f6ae4107b70 2024-09-12T12:24:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.693+0000 7f6adb7fe640 1 -- 172.21.15.137:0/490543918 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc010760 con 0x7f6ae4107b70 2024-09-12T12:24:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.693+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae41a4ec0 con 0x7f6ae4107b70 2024-09-12T12:24:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6adb7fe640 1 -- 172.21.15.137:0/490543918 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6adc005020 con 0x7f6ae4107b70 2024-09-12T12:24:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 Processor -- start 2024-09-12T12:24:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 -- start start 2024-09-12T12:24:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5022ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb50300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bbb4f872f0 con 0x55bbb502dcb0 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bbb4f2f270 con 0x55bbb5022af0 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.696+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbb4f832b0 con 0x55bbb5023410 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6ae9bf9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb50300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:49912/0 (socket says 172.21.15.137:49912) 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6ae9bf9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5022ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 -- 172.21.15.137:0/563901746 learned_addr learned my addr 172.21.15.137:0/563901746 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 -- 172.21.15.137:0/563901746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 msgr2=0x55bbb5022ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/563901746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5022ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 -- 172.21.15.137:0/563901746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 msgr2=0x55bbb50300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/563901746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb50300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 -- 172.21.15.137:0/563901746 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb4f2e340 con 0x55bbb5023410 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6adb7fe640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ad00a2da0 0x7f6ad00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ad00a2da0 0x7f6ad00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/563901746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502d770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00114f0 tx=0x7f6ae00119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382cc455a5697fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6adb7fe640 1 -- 172.21.15.137:0/490543918 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f6adc0cf9e0 con 0x7f6ae4107b70 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6ad8bfe640 1 -- 172.21.15.137:0/563901746 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae001e070 con 0x55bbb5023410 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6ad8bfe640 1 -- 172.21.15.137:0/563901746 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae000fa20 con 0x55bbb5023410 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.697+0000 7f6ad8bfe640 1 -- 172.21.15.137:0/563901746 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0019ac0 con 0x55bbb5023410 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 -- 172.21.15.137:0/563901746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 msgr2=0x55bbb502d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ad00a2da0 0x7f6ad00a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009900 tx=0x7f6ad4000bc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/563901746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502d770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00114f0 tx=0x7f6ae00119b0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 -- 172.21.15.137:0/563901746 shutdown_connections 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/563901746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb50300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/563901746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502d770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/563901746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5022ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 -- 172.21.15.137:0/563901746 >> 172.21.15.137:0/563901746 conn(0x55bbb501f060 msgr2=0x55bbb501f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 -- 172.21.15.137:0/563901746 shutdown_connections 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 -- 172.21.15.137:0/563901746 wait complete. 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.698+0000 7f6aef3f8740 1 -- start start 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 1 Processor -- start 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5030880 con 0x55bbb502dcb0 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55bbb4f10290 con 0x55bbb5022af0 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bbb5030a00 con 0x55bbb5023410 2024-09-12T12:24:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aef3f8740 10 client.0 _ll_register_callbacks cb 0x55bbb4f41140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aeabfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:44938/0 (socket says 172.21.15.137:44938) 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 -- 172.21.15.137:0/297623974 learned_addr learned my addr 172.21.15.137:0/297623974 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 -- 172.21.15.137:0/297623974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 msgr2=0x55bbb502b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 -- 172.21.15.137:0/297623974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 msgr2=0x55bbb502b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 -- 172.21.15.137:0/297623974 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb4c9a210 con 0x55bbb5022af0 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502ab10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001c040 tx=0x7f6ae0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8247050801cdfe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.699+0000 7f6acdffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6acdffb640 1 -- 172.21.15.137:0/297623974 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0027060 con 0x55bbb5022af0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae000fa20 con 0x55bbb5022af0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6acdffb640 1 -- 172.21.15.137:0/297623974 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f6ae0004050 con 0x55bbb5022af0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6acdffb640 1 -- 172.21.15.137:0/297623974 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6ae001e040 con 0x55bbb5022af0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 10 client.26138 fetch_fsmap learned FSMap version 37 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bbb4d66620 con 0x55bbb5022af0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6acdffb640 1 -- 172.21.15.137:0/297623974 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae000fa20 con 0x55bbb5022af0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6acdffb640 1 -- 172.21.15.137:0/297623974 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6ae0021030 con 0x55bbb5022af0 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 10 client.26138 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 20 client.26138 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb503db80 0x55bbb503ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 4 client.26138 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.700+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbb4f94a20 con 0x55bbb503db80 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.701+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb503db80 0x55bbb503ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.701+0000 7f6acdffb640 10 client.26138 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.701+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb503db80 0x55bbb503ff70 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6020b2188497af6e server_cookie=c1c5d34fbe541abb in_seq=0 out_seq=0 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6acdffb640 1 -- 172.21.15.137:0/297623974 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbb4f94a20 con 0x55bbb503db80 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6acdffb640 10 client.26138 handle_command_reply: tid=0 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 client.26138 shutdown 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 msgr2=0x55bbb502ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502ab10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001c040 tx=0x7f6ae0005dc0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 shutdown_connections 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb503db80 0x55bbb503ff70 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6acdffb640 0 client.26138 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/297623974 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502ab10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 >> 172.21.15.137:0/297623974 conn(0x55bbb501f060 msgr2=0x55bbb5024d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 shutdown_connections 2024-09-12T12:24:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 1 -- 172.21.15.137:0/297623974 wait complete. 2024-09-12T12:24:14.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.702+0000 7f6aef3f8740 20 client.26138 trim_cache size 0 max 16384 2024-09-12T12:24:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 Processor -- start 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 -- start start 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb5045630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41b0120 con 0x55bbb502dcb0 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ad990 con 0x55bbb5022af0 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ab800 con 0x55bbb5023410 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aeabfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb5045630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6ae9bf9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:44948/0 (socket says 172.21.15.137:44948) 2024-09-12T12:24:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aeabfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb5045630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47714/0 (socket says 172.21.15.137:47714) 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aea3fa640 1 -- 172.21.15.137:0/405362634 learned_addr learned my addr 172.21.15.137:0/405362634 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aeabfb640 1 -- 172.21.15.137:0/405362634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 msgr2=0x55bbb502ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aeabfb640 1 -- 172.21.15.137:0/405362634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 msgr2=0x55bbb502b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aeabfb640 1 -- 172.21.15.137:0/405362634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbb501ee90 con 0x55bbb502dcb0 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.717+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.718+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb5045630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0cd150 tx=0x7f6adc09bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677bcccec3989195 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.718+0000 7f6acffff640 1 -- 172.21.15.137:0/405362634 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc09c620 con 0x55bbb502dcb0 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.718+0000 7f6acffff640 1 -- 172.21.15.137:0/405362634 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc0b7070 con 0x55bbb502dcb0 2024-09-12T12:24:14.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.718+0000 7f6acffff640 1 -- 172.21.15.137:0/405362634 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0a4700 con 0x55bbb502dcb0 2024-09-12T12:24:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.718+0000 7f6aef3f8740 1 -- 172.21.15.137:0/405362634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 msgr2=0x55bbb5045630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.718+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb5045630 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0cd150 tx=0x7f6adc09bdb0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.718+0000 7f6aef3f8740 1 -- 172.21.15.137:0/405362634 shutdown_connections 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb5045630 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/405362634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb502b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 -- 172.21.15.137:0/405362634 >> 172.21.15.137:0/405362634 conn(0x55bbb501f060 msgr2=0x55bbb502c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 -- 172.21.15.137:0/405362634 shutdown_connections 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 -- 172.21.15.137:0/405362634 wait complete. 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 -- start start 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 Processor -- start 2024-09-12T12:24:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5036230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41abe50 con 0x55bbb502dcb0 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ad2c0 con 0x55bbb5022af0 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41ac520 con 0x55bbb5023410 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aef3f8740 10 client.0 _ll_register_callbacks cb 0x7f6ae001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5036230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.725 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6ae9bf9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aeabfb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aea3fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5036230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:44952/0 (socket says 172.21.15.137:44952) 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6aea3fa640 1 -- 172.21.15.137:0/2390888271 learned_addr learned my addr 172.21.15.137:0/2390888271 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.719+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:49944/0 (socket says 172.21.15.137:49944) 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aea3fa640 1 -- 172.21.15.137:0/2390888271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 msgr2=0x55bbb502e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aea3fa640 1 -- 172.21.15.137:0/2390888271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 msgr2=0x55bbb502ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aea3fa640 1 -- 172.21.15.137:0/2390888271 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbb5023a20 con 0x55bbb5022af0 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aeabfb640 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aea3fa640 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5036230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0005330 tx=0x7f6ae0005360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bf72690ee2ead6f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6acdffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6acdffb640 1 -- 172.21.15.137:0/2390888271 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0005ab0 con 0x55bbb5022af0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6adc0b7070 con 0x55bbb5022af0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6acdffb640 1 -- 172.21.15.137:0/2390888271 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f6ae0019ad0 con 0x55bbb5022af0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6acdffb640 1 -- 172.21.15.137:0/2390888271 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6ae0011070 con 0x55bbb5022af0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aef3f8740 10 client.26144 fetch_fsmap learned FSMap version 37 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.720+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55bbb5034980 con 0x55bbb5022af0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6acdffb640 1 -- 172.21.15.137:0/2390888271 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6adc0b7070 con 0x55bbb5022af0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6acdffb640 1 -- 172.21.15.137:0/2390888271 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f6ae0033020 con 0x55bbb5022af0 2024-09-12T12:24:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6aef3f8740 10 client.26144 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6aef3f8740 10 client.26144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6aef3f8740 20 client.26144 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb50a88a0 0x55bbb50aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6aef3f8740 4 client.26144 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbb4f8ed70 con 0x55bbb50a88a0 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.721+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb50a88a0 0x55bbb50aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.722+0000 7f6acdffb640 10 client.26144 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.722+0000 7f6ae9bf9640 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb50a88a0 0x55bbb50aac90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2f2b45a8ca7485 server_cookie=3e7ed59df7fa5d13 in_seq=0 out_seq=0 2024-09-12T12:24:14.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 859306 7.50 MB/sec execute 469 sec latency 29.673 ms 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.798+0000 7f6acdffb640 1 -- 172.21.15.137:0/2390888271 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbb4f8ed70 con 0x55bbb50a88a0 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.798+0000 7f6acdffb640 10 client.26144 handle_command_reply: tid=0 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.798+0000 7f6aef3f8740 1 client.26144 shutdown 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.798+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 msgr2=0x55bbb5036230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.798+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5036230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0005330 tx=0x7f6ae0005360 comp rx=0 tx=0).stop 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 shutdown_connections 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55bbb50a88a0 0x55bbb50aac90 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6acdffb640 0 client.26144 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbb502dcb0 0x55bbb502ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bbb5023410 0x55bbb502e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 --2- 172.21.15.137:0/2390888271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55bbb5022af0 0x55bbb5036230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 >> 172.21.15.137:0/2390888271 conn(0x55bbb501f060 msgr2=0x55bbb501e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 shutdown_connections 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 1 -- 172.21.15.137:0/2390888271 wait complete. 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.799+0000 7f6aef3f8740 20 client.26144 trim_cache size 0 max 16384 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:24:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:24:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ad00a2da0 msgr2=0x7f6ad00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ad00a2da0 0x7f6ad00a5260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009900 tx=0x7f6ad4000bc0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4107b70 msgr2=0x7f6ae419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4107b70 0x7f6ae419cc10 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00eb10 tx=0x7f6adc00c2d0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 shutdown_connections 2024-09-12T12:24:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6ad00a2da0 0x7f6ad00a5260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ae4107b70 0x7f6ae419cc10 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ae4102470 0x7f6ae4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 --2- 172.21.15.137:0/490543918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ae4101ad0 0x7f6ae4198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 >> 172.21.15.137:0/490543918 conn(0x7f6ae40fd7d0 msgr2=0x7f6ae40fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 shutdown_connections 2024-09-12T12:24:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:14.800+0000 7f6aec65e640 1 -- 172.21.15.137:0/490543918 wait complete. 2024-09-12T12:24:14.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:14 smithi171 ceph-mon[35098]: pgmap v347: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-12T12:24:15.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:14 smithi137 ceph-mon[30950]: pgmap v347: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-12T12:24:15.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:14 smithi189 ceph-mon[35475]: pgmap v347: 97 pgs: 97 active+clean; 229 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 317 op/s 2024-09-12T12:24:15.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 860310 7.49 MB/sec execute 470 sec latency 69.144 ms 2024-09-12T12:24:16.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 862956 7.51 MB/sec execute 471 sec latency 16.798 ms 2024-09-12T12:24:16.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:16 smithi171 ceph-mon[35098]: pgmap v348: 97 pgs: 97 active+clean; 239 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-09-12T12:24:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:16 smithi137 ceph-mon[30950]: pgmap v348: 97 pgs: 97 active+clean; 239 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-09-12T12:24:17.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:16 smithi189 ceph-mon[35475]: pgmap v348: 97 pgs: 97 active+clean; 239 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-09-12T12:24:17.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 864237 7.51 MB/sec execute 472 sec latency 22.872 ms 2024-09-12T12:24:18.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 866581 7.52 MB/sec execute 473 sec latency 15.960 ms 2024-09-12T12:24:18.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:18 smithi171 ceph-mon[35098]: pgmap v349: 97 pgs: 97 active+clean; 239 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-09-12T12:24:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:18 smithi137 ceph-mon[30950]: pgmap v349: 97 pgs: 97 active+clean; 239 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-09-12T12:24:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:18 smithi189 ceph-mon[35475]: pgmap v349: 97 pgs: 97 active+clean; 239 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 226 op/s 2024-09-12T12:24:19.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 867961 7.51 MB/sec execute 474 sec latency 17.834 ms 2024-09-12T12:24:20.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 870110 7.53 MB/sec execute 475 sec latency 44.591 ms 2024-09-12T12:24:20.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:20 smithi171 ceph-mon[35098]: pgmap v350: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 342 op/s 2024-09-12T12:24:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:20 smithi137 ceph-mon[30950]: pgmap v350: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 342 op/s 2024-09-12T12:24:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:20 smithi189 ceph-mon[35475]: pgmap v350: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 342 op/s 2024-09-12T12:24:21.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 871354 7.52 MB/sec execute 476 sec latency 22.276 ms 2024-09-12T12:24:22.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 873661 7.54 MB/sec execute 477 sec latency 15.097 ms 2024-09-12T12:24:22.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:22 smithi171 ceph-mon[35098]: pgmap v351: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 245 op/s 2024-09-12T12:24:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:22 smithi137 ceph-mon[30950]: pgmap v351: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 245 op/s 2024-09-12T12:24:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:22 smithi189 ceph-mon[35475]: pgmap v351: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 245 op/s 2024-09-12T12:24:23.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 874993 7.53 MB/sec execute 478 sec latency 23.016 ms 2024-09-12T12:24:24.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 877299 7.55 MB/sec execute 479 sec latency 14.475 ms 2024-09-12T12:24:24.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:24 smithi171 ceph-mon[35098]: pgmap v352: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 245 op/s 2024-09-12T12:24:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:24 smithi137 ceph-mon[30950]: pgmap v352: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 245 op/s 2024-09-12T12:24:25.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:24 smithi189 ceph-mon[35475]: pgmap v352: 97 pgs: 97 active+clean; 235 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 245 op/s 2024-09-12T12:24:25.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 878632 7.54 MB/sec execute 480 sec latency 56.821 ms 2024-09-12T12:24:26.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 880863 7.55 MB/sec execute 481 sec latency 14.413 ms 2024-09-12T12:24:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:26 smithi171 ceph-mon[35098]: pgmap v353: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 360 op/s 2024-09-12T12:24:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:24:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:26.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:26 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:26 smithi137 ceph-mon[30950]: pgmap v353: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 360 op/s 2024-09-12T12:24:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:24:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:26 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:26 smithi189 ceph-mon[35475]: pgmap v353: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 360 op/s 2024-09-12T12:24:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:24:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:27.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:26 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:27.757 INFO:tasks.workunit.client.0.smithi137.stdout: 1 882392 7.55 MB/sec execute 482 sec latency 16.458 ms 2024-09-12T12:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 -- 172.21.15.137:0/3577448909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 msgr2=0x7fea6010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 --2- 172.21.15.137:0/3577448909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea6010e9a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fea58009c10 tx=0x7fea5802a6e0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 -- 172.21.15.137:0/3577448909 shutdown_connections 2024-09-12T12:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 --2- 172.21.15.137:0/3577448909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea6010e9a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 --2- 172.21.15.137:0/3577448909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea60101ad0 0x7fea60101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 --2- 172.21.15.137:0/3577448909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea60107b70 0x7fea60107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 -- 172.21.15.137:0/3577448909 >> 172.21.15.137:0/3577448909 conn(0x7fea600fd7d0 msgr2=0x7fea600ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 -- 172.21.15.137:0/3577448909 shutdown_connections 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.892+0000 7fea676c0640 1 -- 172.21.15.137:0/3577448909 wait complete. 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 Processor -- start 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 -- start start 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea60101ad0 0x7fea60198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea60198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea60107b70 0x7fea6019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea60111ed0 con 0x7fea60102470 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea60111d50 con 0x7fea60101ad0 2024-09-12T12:24:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea676c0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea60112050 con 0x7fea60107b70 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea60198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea60198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45680/0 (socket says 172.21.15.137:45680) 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 -- 172.21.15.137:0/2645652285 learned_addr learned my addr 172.21.15.137:0/2645652285 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea6545c640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea60101ad0 0x7fea60198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea65c5d640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea60107b70 0x7fea6019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 -- 172.21.15.137:0/2645652285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea60107b70 msgr2=0x7fea6019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea60107b70 0x7fea6019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 -- 172.21.15.137:0/2645652285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea60101ad0 msgr2=0x7fea60198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea60101ad0 0x7fea60198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 -- 172.21.15.137:0/2645652285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6019d320 con 0x7fea60102470 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.893+0000 7fea64c5b640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea60198870 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e9f0 tx=0x7fea5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.894+0000 7fea567fc640 1 -- 172.21.15.137:0/2645652285 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5000cd80 con 0x7fea60102470 2024-09-12T12:24:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.894+0000 7fea567fc640 1 -- 172.21.15.137:0/2645652285 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea50005500 con 0x7fea60102470 2024-09-12T12:24:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.894+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6019d610 con 0x7fea60102470 2024-09-12T12:24:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.894+0000 7fea567fc640 1 -- 172.21.15.137:0/2645652285 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50010690 con 0x7fea60102470 2024-09-12T12:24:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.894+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea601a4ea0 con 0x7fea60102470 2024-09-12T12:24:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.896+0000 7fea567fc640 1 -- 172.21.15.137:0/2645652285 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fea500046c0 con 0x7fea60102470 2024-09-12T12:24:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 Processor -- start 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 -- start start 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea567fc640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea4c0a2da0 0x7fea4c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a25ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a2704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56121a1c32f0 con 0x56121a25f2b0 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56121a16b270 con 0x56121a25e990 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6a454740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56121a1bf2b0 con 0x56121a26e0c0 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea567fc640 1 -- 172.21.15.137:0/2645652285 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fea500cf8b0 con 0x7fea60102470 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea65c5d640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea4c0a2da0 0x7fea4c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a25ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a25ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:60710/0 (socket says 172.21.15.137:60710) 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 -- 172.21.15.137:0/2062688037 learned_addr learned my addr 172.21.15.137:0/2062688037 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea64c5b640 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea65c5d640 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a2704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 -- 172.21.15.137:0/2062688037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 msgr2=0x56121a2704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a2704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 -- 172.21.15.137:0/2062688037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 msgr2=0x56121a26db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea6545c640 1 -- 172.21.15.137:0/2062688037 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121a16a340 con 0x56121a25e990 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.897+0000 7fea65c5d640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea4c0a2da0 0x7fea4c0a5260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea58004af0 tx=0x7fea58030dd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6545c640 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a25ed70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009c60 tx=0x7fea5c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94cfe44be5ed4628 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea4b7fe640 1 -- 172.21.15.137:0/2062688037 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c005220 con 0x56121a25e990 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea4b7fe640 1 -- 172.21.15.137:0/2062688037 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c009e10 con 0x56121a25e990 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea4b7fe640 1 -- 172.21.15.137:0/2062688037 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c019850 con 0x56121a25e990 2024-09-12T12:24:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6a454740 1 -- 172.21.15.137:0/2062688037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 msgr2=0x56121a25ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6a454740 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a25ed70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009c60 tx=0x7fea5c011810 comp rx=0 tx=0).stop 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6a454740 1 -- 172.21.15.137:0/2062688037 shutdown_connections 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6a454740 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a2704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6a454740 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6a454740 1 --2- 172.21.15.137:0/2062688037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a25ed70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.898+0000 7fea6a454740 1 -- 172.21.15.137:0/2062688037 >> 172.21.15.137:0/2062688037 conn(0x56121a25b060 msgr2=0x56121a25b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 -- 172.21.15.137:0/2062688037 shutdown_connections 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 -- 172.21.15.137:0/2062688037 wait complete. 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 -- start start 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 Processor -- start 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a26afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a26ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56121a270cf0 con 0x56121a25f2b0 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56121a14c290 con 0x56121a25e990 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56121a270e70 con 0x56121a26e0c0 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.899+0000 7fea6a454740 10 client.0 _ll_register_callbacks cb 0x56121a17d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a26afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea64c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a26ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a26afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:60712/0 (socket says 172.21.15.137:60712) 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea64c5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45700/0 (socket says 172.21.15.137:45700) 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea6545c640 1 -- 172.21.15.137:0/3260759913 learned_addr learned my addr 172.21.15.137:0/3260759913 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a26ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:58656/0 (socket says 172.21.15.137:58656) 2024-09-12T12:24:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 -- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 msgr2=0x56121a26afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a26afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 -- 172.21.15.137:0/3260759913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 msgr2=0x56121a26b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 -- 172.21.15.137:0/3260759913 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561219ed6210 con 0x56121a26e0c0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea6545c640 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a26afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea65c5d640 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a26ba20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea5802ac00 tx=0x7fea5802a6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356fb2021a6134b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea48ff9640 1 -- 172.21.15.137:0/3260759913 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea58036050 con 0x56121a26e0c0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c009e10 con 0x56121a26e0c0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea48ff9640 1 -- 172.21.15.137:0/3260759913 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fea58004450 con 0x56121a26e0c0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea48ff9640 1 -- 172.21.15.137:0/3260759913 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea5805a050 con 0x56121a26e0c0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea6a454740 10 client.26224 fetch_fsmap learned FSMap version 37 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.900+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x561219fa2620 con 0x56121a26e0c0 2024-09-12T12:24:27.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea48ff9640 1 -- 172.21.15.137:0/3260759913 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c009e10 con 0x56121a26e0c0 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea48ff9640 1 -- 172.21.15.137:0/3260759913 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fea5803c040 con 0x56121a26e0c0 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6a454740 10 client.26224 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6a454740 10 client.26224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6a454740 20 client.26224 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6a454740 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a27e0b0 0x56121a2804a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6a454740 4 client.26224 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56121a1d0a20 con 0x56121a27e0b0 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6545c640 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a27e0b0 0x56121a2804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea48ff9640 10 client.26224 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.901+0000 7fea6545c640 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a27e0b0 0x56121a2804a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774883239612d55c server_cookie=b4ebb5b2ea1fe583 in_seq=0 out_seq=0 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea48ff9640 1 -- 172.21.15.137:0/3260759913 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56121a1d0a20 con 0x56121a27e0b0 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea48ff9640 10 client.26224 handle_command_reply: tid=0 2024-09-12T12:24:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 client.26224 shutdown 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 msgr2=0x56121a26ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a26ba20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fea5802ac00 tx=0x7fea5802a6e0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 shutdown_connections 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a26e0c0 0x56121a26ba20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a27e0b0 0x56121a2804a0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a25f2b0 0x56121a26b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea48ff9640 0 client.26224 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 --2- 172.21.15.137:0/3260759913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a26afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 >> 172.21.15.137:0/3260759913 conn(0x56121a25b060 msgr2=0x56121a25a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 shutdown_connections 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 1 -- 172.21.15.137:0/3260759913 wait complete. 2024-09-12T12:24:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.903+0000 7fea6a454740 20 client.26224 trim_cache size 0 max 16384 2024-09-12T12:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.918+0000 7fea6a454740 1 Processor -- start 2024-09-12T12:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.918+0000 7fea6a454740 1 -- start start 2024-09-12T12:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6a454740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a285db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6a454740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a2862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6a454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a2681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6a454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ade70 con 0x56121a26e0c0 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6a454740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ab6e0 con 0x56121a25e990 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6a454740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea601a9550 con 0x56121a25f2b0 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a285db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea65c5d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a2681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a285db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:60716/0 (socket says 172.21.15.137:60716) 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 -- 172.21.15.137:0/406045037 learned_addr learned my addr 172.21.15.137:0/406045037 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea64c5b640 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a2862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 -- 172.21.15.137:0/406045037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 msgr2=0x56121a2862f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a2862f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 -- 172.21.15.137:0/406045037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 msgr2=0x56121a2681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a2681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 -- 172.21.15.137:0/406045037 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121a269f30 con 0x56121a25e990 2024-09-12T12:24:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea65c5d640 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a2681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea6545c640 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a285db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009b80 tx=0x7fea5c0194d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=855711a4193459b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea4affd640 1 -- 172.21.15.137:0/406045037 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c030070 con 0x56121a25e990 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea4affd640 1 -- 172.21.15.137:0/406045037 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c004c20 con 0x56121a25e990 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.919+0000 7fea4affd640 1 -- 172.21.15.137:0/406045037 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c02bb50 con 0x56121a25e990 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 -- 172.21.15.137:0/406045037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 msgr2=0x56121a285db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a285db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009b80 tx=0x7fea5c0194d0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 -- 172.21.15.137:0/406045037 shutdown_connections 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a2681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a2862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 --2- 172.21.15.137:0/406045037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a285db0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 -- 172.21.15.137:0/406045037 >> 172.21.15.137:0/406045037 conn(0x56121a25b060 msgr2=0x56121a26d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 -- 172.21.15.137:0/406045037 shutdown_connections 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 -- 172.21.15.137:0/406045037 wait complete. 2024-09-12T12:24:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.920+0000 7fea6a454740 1 -- start start 2024-09-12T12:24:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 1 Processor -- start 2024-09-12T12:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a271dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a272310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a272850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fea601a9ba0 con 0x56121a26e0c0 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea601ab010 con 0x56121a25e990 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea601aa270 con 0x56121a25f2b0 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6a454740 10 client.0 _ll_register_callbacks cb 0x7fea5c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a271dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea65c5d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a272850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea64c5b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a272310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a271dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:60728/0 (socket says 172.21.15.137:60728) 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 -- 172.21.15.137:0/542183511 learned_addr learned my addr 172.21.15.137:0/542183511 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 -- 172.21.15.137:0/542183511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 msgr2=0x56121a272310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a272310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 -- 172.21.15.137:0/542183511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 msgr2=0x56121a272850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a272850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea6545c640 1 -- 172.21.15.137:0/542183511 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121a25f0e0 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.921+0000 7fea65c5d640 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a272850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6545c640 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a271dd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea5c02b770 tx=0x7fea5c009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2d9db108e3e3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea48ff9640 1 -- 172.21.15.137:0/542183511 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c030070 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea48ff9640 1 -- 172.21.15.137:0/542183511 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fea5c02b3d0 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea5c004c20 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea48ff9640 1 -- 172.21.15.137:0/542183511 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fea5c011d10 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 10 client.26162 fetch_fsmap learned FSMap version 37 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56121a2e8e70 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea48ff9640 1 -- 172.21.15.137:0/542183511 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea5c004c20 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea48ff9640 1 -- 172.21.15.137:0/542183511 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7fea5c034470 con 0x56121a25e990 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 10 client.26162 fetch_fsmap finished waiting for FSMap version 37 2024-09-12T12:24:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 20 client.26162 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a2e9330 0x56121a2eb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 4 client.26162 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:24:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.922+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56121a1cad70 con 0x56121a2e9330 2024-09-12T12:24:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.923+0000 7fea64c5b640 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a2e9330 0x56121a2eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.923+0000 7fea48ff9640 10 client.26162 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:27.923+0000 7fea64c5b640 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a2e9330 0x56121a2eb720 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0cae5902e3a8d4 server_cookie=c86961b05b76aaef in_seq=0 out_seq=0 2024-09-12T12:24:27.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:24:27.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:24:27.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.003+0000 7fea48ff9640 1 -- 172.21.15.137:0/542183511 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56121a1cad70 con 0x56121a2e9330 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.003+0000 7fea48ff9640 10 client.26162 handle_command_reply: tid=0 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.003+0000 7fea6a454740 1 client.26162 shutdown 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 msgr2=0x56121a271dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a271dd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea5c02b770 tx=0x7fea5c009290 comp rx=0 tx=0).stop 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 shutdown_connections 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56121a2e9330 0x56121a2eb720 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56121a26e0c0 0x56121a272850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea48ff9640 0 client.26162 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56121a25f2b0 0x56121a272310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 --2- 172.21.15.137:0/542183511 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56121a25e990 0x56121a271dd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 >> 172.21.15.137:0/542183511 conn(0x56121a25b060 msgr2=0x56121a26c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 shutdown_connections 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 1 -- 172.21.15.137:0/542183511 wait complete. 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.004+0000 7fea6a454740 20 client.26162 trim_cache size 0 max 16384 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:24:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:24:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea4c0a2da0 msgr2=0x7fea4c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea4c0a2da0 0x7fea4c0a5260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea58004af0 tx=0x7fea58030dd0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 msgr2=0x7fea60198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea60198870 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fea5000e9f0 tx=0x7fea5000edf0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 shutdown_connections 2024-09-12T12:24:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fea4c0a2da0 0x7fea4c0a5260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea60107b70 0x7fea6019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fea60102470 0x7fea60198870 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 --2- 172.21.15.137:0/2645652285 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea60101ad0 0x7fea60198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 >> 172.21.15.137:0/2645652285 conn(0x7fea600fd7d0 msgr2=0x7fea600fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 shutdown_connections 2024-09-12T12:24:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:28.005+0000 7fea676c0640 1 -- 172.21.15.137:0/2645652285 wait complete. 2024-09-12T12:24:28.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:24:28.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:24:28.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:28.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:24:28.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:24:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:24:28.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 884444 7.56 MB/sec execute 483 sec latency 43.327 ms 2024-09-12T12:24:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:28 smithi137 ceph-mon[30950]: pgmap v354: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-09-12T12:24:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:28 smithi189 ceph-mon[35475]: pgmap v354: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-09-12T12:24:29.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:28 smithi171 ceph-mon[35098]: pgmap v354: 97 pgs: 97 active+clean; 243 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 265 op/s 2024-09-12T12:24:29.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 886473 7.56 MB/sec execute 484 sec latency 21.715 ms 2024-09-12T12:24:30.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 888164 7.57 MB/sec execute 485 sec latency 51.161 ms 2024-09-12T12:24:31.000 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:30 smithi171 ceph-mon[35098]: pgmap v355: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-09-12T12:24:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:30 smithi137 ceph-mon[30950]: pgmap v355: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-09-12T12:24:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:30 smithi189 ceph-mon[35475]: pgmap v355: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 423 op/s 2024-09-12T12:24:31.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 890752 7.58 MB/sec execute 486 sec latency 29.184 ms 2024-09-12T12:24:32.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 892168 7.58 MB/sec execute 487 sec latency 22.091 ms 2024-09-12T12:24:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:32 smithi137 ceph-mon[30950]: pgmap v356: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-12T12:24:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:32 smithi189 ceph-mon[35475]: pgmap v356: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-12T12:24:33.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:32 smithi171 ceph-mon[35098]: pgmap v356: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-12T12:24:33.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 894794 7.60 MB/sec execute 488 sec latency 15.419 ms 2024-09-12T12:24:34.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 896153 7.60 MB/sec execute 489 sec latency 19.575 ms 2024-09-12T12:24:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:34 smithi137 ceph-mon[30950]: pgmap v357: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-12T12:24:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:34 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:24:35.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:34 smithi189 ceph-mon[35475]: pgmap v357: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-12T12:24:35.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:34 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:24:35.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:34 smithi171 ceph-mon[35098]: pgmap v357: 97 pgs: 97 active+clean; 238 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 271 op/s 2024-09-12T12:24:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:34 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:24:35.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 898487 7.61 MB/sec execute 490 sec latency 44.334 ms 2024-09-12T12:24:36.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 900178 7.61 MB/sec execute 491 sec latency 30.411 ms 2024-09-12T12:24:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:36 smithi137 ceph-mon[30950]: pgmap v358: 97 pgs: 97 active+clean; 246 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 396 op/s 2024-09-12T12:24:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:36 smithi189 ceph-mon[35475]: pgmap v358: 97 pgs: 97 active+clean; 246 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 396 op/s 2024-09-12T12:24:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:36 smithi171 ceph-mon[35098]: pgmap v358: 97 pgs: 97 active+clean; 246 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 396 op/s 2024-09-12T12:24:37.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 902300 7.62 MB/sec execute 492 sec latency 11.197 ms 2024-09-12T12:24:38.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 904352 7.62 MB/sec execute 493 sec latency 19.338 ms 2024-09-12T12:24:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:38 smithi137 ceph-mon[30950]: pgmap v359: 97 pgs: 97 active+clean; 246 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2024-09-12T12:24:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:38 smithi189 ceph-mon[35475]: pgmap v359: 97 pgs: 97 active+clean; 246 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2024-09-12T12:24:39.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:38 smithi171 ceph-mon[35098]: pgmap v359: 97 pgs: 97 active+clean; 246 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2024-09-12T12:24:39.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 906193 7.63 MB/sec execute 494 sec latency 21.612 ms 2024-09-12T12:24:40.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 908497 7.64 MB/sec execute 495 sec latency 19.595 ms 2024-09-12T12:24:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:40 smithi137 ceph-mon[30950]: pgmap v360: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 400 op/s 2024-09-12T12:24:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:40 smithi189 ceph-mon[35475]: pgmap v360: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 400 op/s 2024-09-12T12:24:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 -- 172.21.15.137:0/1018811727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 msgr2=0x7fb96c10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 --2- 172.21.15.137:0/1018811727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 0x7fb96c10e560 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb964009ca0 tx=0x7fb96402a950 comp rx=0 tx=0).stop 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 -- 172.21.15.137:0/1018811727 shutdown_connections 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 --2- 172.21.15.137:0/1018811727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 0x7fb96c10e560 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 --2- 172.21.15.137:0/1018811727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c103640 0x7fb96c10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 --2- 172.21.15.137:0/1018811727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c102d20 0x7fb96c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 -- 172.21.15.137:0/1018811727 >> 172.21.15.137:0/1018811727 conn(0x7fb96c077990 msgr2=0x7fb96c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 -- 172.21.15.137:0/1018811727 shutdown_connections 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.098+0000 7fb971623640 1 -- 172.21.15.137:0/1018811727 wait complete. 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 Processor -- start 2024-09-12T12:24:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 -- start start 2024-09-12T12:24:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c102d20 0x7fb96c100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c103640 0x7fb96c100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 0x7fb96c1013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c111f10 con 0x7fb96c10c0f0 2024-09-12T12:24:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c111d90 con 0x7fb96c102d20 2024-09-12T12:24:41.101 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb971623640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c112090 con 0x7fb96c103640 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb96affd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c102d20 0x7fb96c100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb96b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 0x7fb96c1013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.099+0000 7fb96affd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c102d20 0x7fb96c100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:44528/0 (socket says 172.21.15.137:44528) 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96affd640 1 -- 172.21.15.137:0/1411838920 learned_addr learned my addr 172.21.15.137:0/1411838920 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c103640 0x7fb96c100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96b7fe640 1 -- 172.21.15.137:0/1411838920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c103640 msgr2=0x7fb96c100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96b7fe640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c103640 0x7fb96c100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96b7fe640 1 -- 172.21.15.137:0/1411838920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c102d20 msgr2=0x7fb96c100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96b7fe640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c102d20 0x7fb96c100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96b7fe640 1 -- 172.21.15.137:0/1411838920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c1a6860 con 0x7fb96c10c0f0 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb96b7fe640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 0x7fb96c1013f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb964005e30 tx=0x7fb964003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb953fff640 1 -- 172.21.15.137:0/1411838920 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964034040 con 0x7fb96c10c0f0 2024-09-12T12:24:41.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb953fff640 1 -- 172.21.15.137:0/1411838920 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb96402f940 con 0x7fb96c10c0f0 2024-09-12T12:24:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c1a6af0 con 0x7fb96c10c0f0 2024-09-12T12:24:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb953fff640 1 -- 172.21.15.137:0/1411838920 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9640045e0 con 0x7fb96c10c0f0 2024-09-12T12:24:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.100+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a7030 con 0x7fb96c10c0f0 2024-09-12T12:24:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.102+0000 7fb953fff640 1 -- 172.21.15.137:0/1411838920 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb964037020 con 0x7fb96c10c0f0 2024-09-12T12:24:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 Processor -- start 2024-09-12T12:24:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 -- start start 2024-09-12T12:24:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c06a410 0x56426c074770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c074cb0 0x56426c0770a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56426bfce2f0 con 0x56426c074cb0 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56426bf76270 con 0x56426c069af0 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb9743be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56426bfca2b0 con 0x56426c06a410 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb953fff640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb9540a2e20 0x7fb9540a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c074cb0 0x56426c0770a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c06a410 0x56426c074770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c074cb0 0x56426c0770a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46510/0 (socket says 172.21.15.137:46510) 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96a7fc640 1 -- 172.21.15.137:0/1768039057 learned_addr learned my addr 172.21.15.137:0/1768039057 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96b7fe640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb9540a2e20 0x7fb9540a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb953fff640 1 -- 172.21.15.137:0/1411838920 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fb9640eff00 con 0x7fb96c10c0f0 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96affd640 1 -- 172.21.15.137:0/1768039057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 msgr2=0x56426c069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96affd640 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96affd640 1 -- 172.21.15.137:0/1768039057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c074cb0 msgr2=0x56426c0770a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96affd640 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c074cb0 0x56426c0770a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96affd640 1 -- 172.21.15.137:0/1768039057 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56426bf75340 con 0x56426c06a410 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.103+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c074cb0 0x56426c0770a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.104+0000 7fb96b7fe640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb9540a2e20 0x7fb9540a52e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9640ed830 tx=0x7fb964087620 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.104+0000 7fb96affd640 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c06a410 0x56426c074770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009d20 tx=0x7fb95c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581e259665f441fd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.104+0000 7fb9517fa640 1 -- 172.21.15.137:0/1768039057 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c005020 con 0x56426c06a410 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.104+0000 7fb9517fa640 1 -- 172.21.15.137:0/1768039057 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c0047a0 con 0x56426c06a410 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.104+0000 7fb9517fa640 1 -- 172.21.15.137:0/1768039057 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c019650 con 0x56426c06a410 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.104+0000 7fb9743be740 1 -- 172.21.15.137:0/1768039057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c06a410 msgr2=0x56426c074770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.104+0000 7fb9743be740 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c06a410 0x56426c074770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009d20 tx=0x7fb95c011810 comp rx=0 tx=0).stop 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- 172.21.15.137:0/1768039057 shutdown_connections 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c074cb0 0x56426c0770a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c06a410 0x56426c074770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 --2- 172.21.15.137:0/1768039057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- 172.21.15.137:0/1768039057 >> 172.21.15.137:0/1768039057 conn(0x56426c066060 msgr2=0x56426c066440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- 172.21.15.137:0/1768039057 shutdown_connections 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- 172.21.15.137:0/1768039057 wait complete. 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- start start 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 Processor -- start 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c072090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c0725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56426c077880 con 0x56426c06a410 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56426bf57290 con 0x56426c069af0 2024-09-12T12:24:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56426c077a00 con 0x56426c074cb0 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb9743be740 10 client.0 _ll_register_callbacks cb 0x56426bf88140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb96affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb96affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c072090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46522/0 (socket says 172.21.15.137:46522) 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb96affd640 1 -- 172.21.15.137:0/109675751 learned_addr learned my addr 172.21.15.137:0/109675751 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.105+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c0725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb96a7fc640 1 -- 172.21.15.137:0/109675751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 msgr2=0x56426c0725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c0725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb96a7fc640 1 -- 172.21.15.137:0/109675751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 msgr2=0x56426c072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c072090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb96a7fc640 1 -- 172.21.15.137:0/109675751 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56426bce1210 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb96affd640 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c071b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb960012c30 tx=0x7fb960009ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf98892d1067bee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb94eff5640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb94eff5640 1 -- 172.21.15.137:0/109675751 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96001a040 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb94eff5640 1 -- 172.21.15.137:0/109675751 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fb960011250 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb95c0047a0 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb94eff5640 1 -- 172.21.15.137:0/109675751 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb960005020 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb9743be740 10 client.26174 fetch_fsmap learned FSMap version 38 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56426bdad620 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.106+0000 7fb94eff5640 1 -- 172.21.15.137:0/109675751 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c0047a0 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb94eff5640 1 -- 172.21.15.137:0/109675751 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb960027030 con 0x56426c069af0 2024-09-12T12:24:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb9743be740 10 client.26174 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb9743be740 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb9743be740 20 client.26174 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb9743be740 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c084bf0 0x56426c086fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb9743be740 4 client.26174 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56426bfdba20 con 0x56426c084bf0 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb96affd640 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c084bf0 0x56426c086fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb96affd640 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c084bf0 0x56426c086fe0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e66f8c5d3115e863 server_cookie=becb0cfb0e594419 in_seq=0 out_seq=0 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.107+0000 7fb94eff5640 10 client.26174 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb94eff5640 1 -- 172.21.15.137:0/109675751 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56426bfdba20 con 0x56426c084bf0 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb94eff5640 10 client.26174 handle_command_reply: tid=0 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 client.26174 shutdown 2024-09-12T12:24:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 msgr2=0x56426c071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c071b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb960012c30 tx=0x7fb960009ae0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 shutdown_connections 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c0725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c084bf0 0x56426c086fe0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 --2- 172.21.15.137:0/109675751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c071b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb94eff5640 0 client.26174 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 >> 172.21.15.137:0/109675751 conn(0x56426c066060 msgr2=0x56426c06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 shutdown_connections 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 1 -- 172.21.15.137:0/109675751 wait complete. 2024-09-12T12:24:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.108+0000 7fb9743be740 20 client.26174 trim_cache size 0 max 16384 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.123+0000 7fb9743be740 1 Processor -- start 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.123+0000 7fb9743be740 1 -- start start 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.123+0000 7fb9743be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.123+0000 7fb9743be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c08c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.123+0000 7fb9743be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c08cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb9743be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1b2190 con 0x56426c06a410 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb9743be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1afa60 con 0x56426c069af0 2024-09-12T12:24:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb9743be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1ad8d0 con 0x56426c074cb0 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c08c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c08c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46538/0 (socket says 172.21.15.137:46538) 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 -- 172.21.15.137:0/3565094718 learned_addr learned my addr 172.21.15.137:0/3565094718 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c08cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 -- 172.21.15.137:0/3565094718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 msgr2=0x56426c08cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c08cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 -- 172.21.15.137:0/3565094718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 msgr2=0x56426c072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 -- 172.21.15.137:0/3565094718 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56426c087640 con 0x56426c06a410 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb96affd640 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c08c870 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009850 tx=0x7fb95c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7754bc22dac7561 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb950ff9640 1 -- 172.21.15.137:0/3565094718 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c030030 con 0x56426c06a410 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb950ff9640 1 -- 172.21.15.137:0/3565094718 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c004c60 con 0x56426c06a410 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.124+0000 7fb950ff9640 1 -- 172.21.15.137:0/3565094718 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0055e0 con 0x56426c06a410 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 -- 172.21.15.137:0/3565094718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 msgr2=0x56426c08c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c08c870 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009850 tx=0x7fb95c005f90 comp rx=0 tx=0).stop 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 -- 172.21.15.137:0/3565094718 shutdown_connections 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c08cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c06a410 0x56426c08c870 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 --2- 172.21.15.137:0/3565094718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c069af0 0x56426c072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 -- 172.21.15.137:0/3565094718 >> 172.21.15.137:0/3565094718 conn(0x56426c066060 msgr2=0x56426c073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 -- 172.21.15.137:0/3565094718 shutdown_connections 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 -- 172.21.15.137:0/3565094718 wait complete. 2024-09-12T12:24:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 -- start start 2024-09-12T12:24:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.125+0000 7fb9743be740 1 Processor -- start 2024-09-12T12:24:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c069af0 0x56426c07d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c06a410 0x56426c078720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c078c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1adf20 con 0x56426c069af0 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1af390 con 0x56426c06a410 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1ae5f0 con 0x56426c074cb0 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c069af0 0x56426c07d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 10 client.0 _ll_register_callbacks cb 0x7fb960012560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c069af0 0x56426c07d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46548/0 (socket says 172.21.15.137:46548) 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 -- 172.21.15.137:0/2774990860 learned_addr learned my addr 172.21.15.137:0/2774990860 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96affd640 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c06a410 0x56426c078720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c078c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 -- 172.21.15.137:0/2774990860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 msgr2=0x56426c078c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c078c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 -- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c06a410 msgr2=0x56426c078720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c06a410 0x56426c078720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 -- 172.21.15.137:0/2774990860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56426c06c940 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb96a7fc640 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c069af0 0x56426c07d400 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb960004d00 tx=0x7fb9600126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0ffa715fca5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb94eff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb94eff5640 1 -- 172.21.15.137:0/2774990860 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9600053c0 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb94eff5640 1 -- 172.21.15.137:0/2774990860 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fb960005560 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.126+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb95c004c60 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb94eff5640 1 -- 172.21.15.137:0/2774990860 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb960010430 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 10 client.16833 fetch_fsmap learned FSMap version 38 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56426c069920 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb94eff5640 1 -- 172.21.15.137:0/2774990860 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb95c004c60 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb94eff5640 1 -- 172.21.15.137:0/2774990860 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7fb96003e070 con 0x56426c069af0 2024-09-12T12:24:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 10 client.16833 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 20 client.16833 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c0efab0 0x56426c0f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 4 client.16833 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56426bfd5d70 con 0x56426c0efab0 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.127+0000 7fb96affd640 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c0efab0 0x56426c0f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.128+0000 7fb96affd640 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c0efab0 0x56426c0f1ed0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4457dfc338e8c411 server_cookie=11a84044b385f510 in_seq=0 out_seq=0 2024-09-12T12:24:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.128+0000 7fb94eff5640 10 client.16833 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:41.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:40 smithi171 ceph-mon[35098]: pgmap v360: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 400 op/s 2024-09-12T12:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb94eff5640 1 -- 172.21.15.137:0/2774990860 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56426bfd5d70 con 0x56426c0efab0 2024-09-12T12:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb94eff5640 10 client.16833 handle_command_reply: tid=0 2024-09-12T12:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 client.16833 shutdown 2024-09-12T12:24:41.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c069af0 msgr2=0x56426c07d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c069af0 0x56426c07d400 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb960004d00 tx=0x7fb9600126e0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 shutdown_connections 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426c074cb0 0x56426c078c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56426c06a410 0x56426c078720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56426c0efab0 0x56426c0f1ed0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb94eff5640 0 client.16833 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 --2- 172.21.15.137:0/2774990860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56426c069af0 0x56426c07d400 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 >> 172.21.15.137:0/2774990860 conn(0x56426c066060 msgr2=0x56426c073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 shutdown_connections 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 1 -- 172.21.15.137:0/2774990860 wait complete. 2024-09-12T12:24:41.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.201+0000 7fb9743be740 20 client.16833 trim_cache size 0 max 16384 2024-09-12T12:24:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:24:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:24:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:24:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb9540a2e20 msgr2=0x7fb9540a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb9540a2e20 0x7fb9540a52e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9640ed830 tx=0x7fb964087620 comp rx=0 tx=0).stop 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 msgr2=0x7fb96c1013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 0x7fb96c1013f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb964005e30 tx=0x7fb964003b80 comp rx=0 tx=0).stop 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 shutdown_connections 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb9540a2e20 0x7fb9540a52e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb96c10c0f0 0x7fb96c1013f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb96c103640 0x7fb96c100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 --2- 172.21.15.137:0/1411838920 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb96c102d20 0x7fb96c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:41.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 >> 172.21.15.137:0/1411838920 conn(0x7fb96c077990 msgr2=0x7fb96c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 shutdown_connections 2024-09-12T12:24:41.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:41.202+0000 7fb971623640 1 -- 172.21.15.137:0/1411838920 wait complete. 2024-09-12T12:24:41.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 909960 7.64 MB/sec execute 496 sec latency 65.340 ms 2024-09-12T12:24:42.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 912692 7.66 MB/sec execute 497 sec latency 13.622 ms 2024-09-12T12:24:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:42 smithi137 ceph-mon[30950]: pgmap v361: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-12T12:24:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:42 smithi189 ceph-mon[35475]: pgmap v361: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-12T12:24:43.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:42 smithi171 ceph-mon[35098]: pgmap v361: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-12T12:24:43.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 914243 7.65 MB/sec execute 498 sec latency 21.687 ms 2024-09-12T12:24:44.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 916481 7.66 MB/sec execute 499 sec latency 14.971 ms 2024-09-12T12:24:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:44 smithi137 ceph-mon[30950]: pgmap v362: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-12T12:24:45.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:44 smithi189 ceph-mon[35475]: pgmap v362: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-12T12:24:45.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:44 smithi171 ceph-mon[35098]: pgmap v362: 97 pgs: 97 active+clean; 254 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 242 op/s 2024-09-12T12:24:45.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 917466 7.69 MB/sec execute 500 sec latency 41.506 ms 2024-09-12T12:24:46.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 920246 7.71 MB/sec execute 501 sec latency 42.942 ms 2024-09-12T12:24:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:46 smithi137 ceph-mon[30950]: pgmap v363: 97 pgs: 97 active+clean; 247 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 409 op/s 2024-09-12T12:24:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:46 smithi189 ceph-mon[35475]: pgmap v363: 97 pgs: 97 active+clean; 247 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 409 op/s 2024-09-12T12:24:47.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:46 smithi171 ceph-mon[35098]: pgmap v363: 97 pgs: 97 active+clean; 247 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 409 op/s 2024-09-12T12:24:47.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 921740 7.71 MB/sec execute 502 sec latency 15.355 ms 2024-09-12T12:24:48.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 924084 7.72 MB/sec execute 503 sec latency 12.917 ms 2024-09-12T12:24:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:48 smithi137 ceph-mon[30950]: pgmap v364: 97 pgs: 97 active+clean; 247 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-09-12T12:24:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:48 smithi189 ceph-mon[35475]: pgmap v364: 97 pgs: 97 active+clean; 247 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-09-12T12:24:49.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:48 smithi171 ceph-mon[35098]: pgmap v364: 97 pgs: 97 active+clean; 247 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 284 op/s 2024-09-12T12:24:49.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 925651 7.72 MB/sec execute 504 sec latency 17.660 ms 2024-09-12T12:24:50.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 927783 7.73 MB/sec execute 505 sec latency 12.677 ms 2024-09-12T12:24:51.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:50 smithi137 ceph-mon[30950]: pgmap v365: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-09-12T12:24:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:50 smithi189 ceph-mon[35475]: pgmap v365: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-09-12T12:24:51.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:50 smithi171 ceph-mon[35098]: pgmap v365: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-09-12T12:24:51.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 929540 7.72 MB/sec execute 506 sec latency 66.600 ms 2024-09-12T12:24:52.758 INFO:tasks.workunit.client.0.smithi137.stdout: 1 931476 7.74 MB/sec execute 507 sec latency 11.282 ms 2024-09-12T12:24:53.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:52 smithi137 ceph-mon[30950]: pgmap v366: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-09-12T12:24:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:52 smithi189 ceph-mon[35475]: pgmap v366: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-09-12T12:24:53.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:52 smithi171 ceph-mon[35098]: pgmap v366: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-09-12T12:24:53.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 933895 7.75 MB/sec execute 508 sec latency 14.339 ms 2024-09-12T12:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 -- 172.21.15.137:0/1376015918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 msgr2=0x7f9f10101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1376015918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f10101f30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009930 tx=0x7f9ef802a420 comp rx=0 tx=0).stop 2024-09-12T12:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 -- 172.21.15.137:0/1376015918 shutdown_connections 2024-09-12T12:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1376015918 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10102470 0x7f9f1010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1376015918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f10101f30 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1376015918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f10107b70 0x7f9f10107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 -- 172.21.15.137:0/1376015918 >> 172.21.15.137:0/1376015918 conn(0x7f9f100fd7d0 msgr2=0x7f9f100ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 -- 172.21.15.137:0/1376015918 shutdown_connections 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.293+0000 7f9f160dd640 1 -- 172.21.15.137:0/1376015918 wait complete. 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 Processor -- start 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 -- start start 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10102470 0x7f9f10074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f10107b70 0x7f9f10074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10111f30 con 0x7f9f10101ad0 2024-09-12T12:24:54.297 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10111db0 con 0x7f9f10102470 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f160dd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101120b0 con 0x7f9f10107b70 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58856/0 (socket says 172.21.15.137:58856) 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/1143003517 learned_addr learned my addr 172.21.15.137:0/1143003517 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f07fff640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10102470 0x7f9f10074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f10107b70 0x7f9f10074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/1143003517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f10107b70 msgr2=0x7f9f10074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f10107b70 0x7f9f10074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/1143003517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10102470 msgr2=0x7f9f10074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10102470 0x7f9f10074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/1143003517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f101a6880 con 0x7f9f10101ad0 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f07fff640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10102470 0x7f9f10074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.294+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f100744f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000e9f0 tx=0x7f9f0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.295+0000 7f9f0d7fa640 1 -- 172.21.15.137:0/1143003517 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0000cd80 con 0x7f9f10101ad0 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.295+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f101a6b70 con 0x7f9f10101ad0 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.295+0000 7f9f0d7fa640 1 -- 172.21.15.137:0/1143003517 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f00010910 con 0x7f9f10101ad0 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.295+0000 7f9f0d7fa640 1 -- 172.21.15.137:0/1143003517 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f000188f0 con 0x7f9f10101ad0 2024-09-12T12:24:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.295+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101a70e0 con 0x7f9f10101ad0 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.297+0000 7f9f0d7fa640 1 -- 172.21.15.137:0/1143003517 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9f00010ac0 con 0x7f9f10101ad0 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.297+0000 7f9f18e7b740 1 Processor -- start 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.297+0000 7f9f18e7b740 1 -- start start 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a339ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a344770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a344cb0 0x55d18a3470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d18a29e2f0 con 0x55d18a344cb0 2024-09-12T12:24:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d18a246270 con 0x55d18a33a410 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d18a29a2b0 con 0x55d18a339af0 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a339ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a344770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a339ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:58526/0 (socket says 172.21.15.137:58526) 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 -- 172.21.15.137:0/1160326361 learned_addr learned my addr 172.21.15.137:0/1160326361 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f07fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a344cb0 0x55d18a3470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0d7fa640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9efc0a2da0 0x7f9efc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 -- 172.21.15.137:0/1160326361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 msgr2=0x55d18a344770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/1160326361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a344770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 -- 172.21.15.137:0/1160326361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a344cb0 msgr2=0x55d18a3470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/1160326361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a344cb0 0x55d18a3470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 -- 172.21.15.137:0/1160326361 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18a245340 con 0x55d18a339af0 2024-09-12T12:24:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0d7fa640 1 -- 172.21.15.137:0/1143003517 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f9f000cf790 con 0x7f9f10101ad0 2024-09-12T12:24:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9efc0a2da0 0x7f9efc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/1160326361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a339ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009fd0 tx=0x7f9f080118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9180639ac24fa5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.298+0000 7f9f05ffb640 1 -- 172.21.15.137:0/1160326361 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004050 con 0x55d18a339af0 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f05ffb640 1 -- 172.21.15.137:0/1160326361 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f08003e70 con 0x55d18a339af0 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f05ffb640 1 -- 172.21.15.137:0/1160326361 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08019400 con 0x55d18a339af0 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 -- 172.21.15.137:0/1160326361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 msgr2=0x55d18a339ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9efc0a2da0 0x7f9efc0a5260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9f0801d000 tx=0x7f9f08019780 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/1160326361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a339ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009fd0 tx=0x7f9f080118d0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 -- 172.21.15.137:0/1160326361 shutdown_connections 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/1160326361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a344cb0 0x55d18a3470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/1160326361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a344770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/1160326361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a339ed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 -- 172.21.15.137:0/1160326361 >> 172.21.15.137:0/1160326361 conn(0x55d18a336060 msgr2=0x55d18a336440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 -- 172.21.15.137:0/1160326361 shutdown_connections 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.299+0000 7f9f18e7b740 1 -- 172.21.15.137:0/1160326361 wait complete. 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 -- start start 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 Processor -- start 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a341b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a342070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a344cb0 0x55d18a3425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d18a347880 con 0x55d18a339af0 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d18a227290 con 0x55d18a33a410 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d18a347a00 con 0x55d18a344cb0 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f18e7b740 10 client.0 _ll_register_callbacks cb 0x55d18a258140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a341b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f07fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a342070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f0ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a344cb0 0x55d18a3425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a341b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58874/0 (socket says 172.21.15.137:58874) 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.300+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/737842729 learned_addr learned my addr 172.21.15.137:0/737842729 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/737842729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a344cb0 msgr2=0x55d18a3425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a344cb0 0x55d18a3425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/737842729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 msgr2=0x55d18a342070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a342070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/737842729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d189fb1210 con 0x55d18a339af0 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a344cb0 0x55d18a3425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:24:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a341b30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ccf60 tx=0x7f9f000980e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8209225e433c5a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9eeeffd640 1 -- 172.21.15.137:0/737842729 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f000cd390 con 0x55d18a339af0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9eeeffd640 1 -- 172.21.15.137:0/737842729 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f9f0009b3d0 con 0x55d18a339af0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f08003e70 con 0x55d18a339af0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9eeeffd640 1 -- 172.21.15.137:0/737842729 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9f0009bd80 con 0x55d18a339af0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f18e7b740 10 client.16845 fetch_fsmap learned FSMap version 38 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d18a07d620 con 0x55d18a339af0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9eeeffd640 1 -- 172.21.15.137:0/737842729 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f08003e70 con 0x55d18a339af0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9eeeffd640 1 -- 172.21.15.137:0/737842729 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f9f00018ad0 con 0x55d18a339af0 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f18e7b740 10 client.16845 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.301+0000 7f9f18e7b740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.302+0000 7f9f18e7b740 20 client.16845 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.302+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a354b80 0x55d18a356f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.302+0000 7f9f18e7b740 4 client.16845 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.302+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d18a2aba20 con 0x55d18a354b80 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.302+0000 7f9f07fff640 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a354b80 0x55d18a356f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.302+0000 7f9eeeffd640 10 client.16845 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.302+0000 7f9f07fff640 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a354b80 0x55d18a356f90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f611fbe2551b859 server_cookie=21775d13c12287c3 in_seq=0 out_seq=0 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9eeeffd640 1 -- 172.21.15.137:0/737842729 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d18a2aba20 con 0x55d18a354b80 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9eeeffd640 10 client.16845 handle_command_reply: tid=0 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 client.16845 shutdown 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 msgr2=0x55d18a341b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a341b30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9f000ccf60 tx=0x7f9f000980e0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 shutdown_connections 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a344cb0 0x55d18a3425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a33a410 0x55d18a342070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a354b80 0x55d18a356f90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/737842729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a341b30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9eeeffd640 0 client.16845 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 >> 172.21.15.137:0/737842729 conn(0x55d18a336060 msgr2=0x55d18a33bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 shutdown_connections 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 1 -- 172.21.15.137:0/737842729 wait complete. 2024-09-12T12:24:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.303+0000 7f9f18e7b740 20 client.16845 trim_cache size 0 max 16384 2024-09-12T12:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 Processor -- start 2024-09-12T12:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 -- start start 2024-09-12T12:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a3423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a33a410 0x55d18a35c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a35cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101b2240 con 0x55d18a33a410 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101afb10 con 0x55d18a344cb0 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101ad980 con 0x55d18a339af0 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a33a410 0x55d18a35c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a33a410 0x55d18a35c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58890/0 (socket says 172.21.15.137:58890) 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 -- 172.21.15.137:0/380925836 learned_addr learned my addr 172.21.15.137:0/380925836 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a35cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a3423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 -- 172.21.15.137:0/380925836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 msgr2=0x55d18a3423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a3423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 -- 172.21.15.137:0/380925836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 msgr2=0x55d18a35cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a35cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 -- 172.21.15.137:0/380925836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d18a335e90 con 0x55d18a33a410 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.318+0000 7f9f07fff640 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a33a410 0x55d18a35c890 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9ef802aa10 tx=0x7f9ef8005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0be17961d07f9f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f057fa640 1 -- 172.21.15.137:0/380925836 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef804b070 con 0x55d18a33a410 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f057fa640 1 -- 172.21.15.137:0/380925836 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef8004c40 con 0x55d18a33a410 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f057fa640 1 -- 172.21.15.137:0/380925836 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8035040 con 0x55d18a33a410 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 -- 172.21.15.137:0/380925836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a33a410 msgr2=0x55d18a35c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a33a410 0x55d18a35c890 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9ef802aa10 tx=0x7f9ef8005310 comp rx=0 tx=0).stop 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 -- 172.21.15.137:0/380925836 shutdown_connections 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a35cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a33a410 0x55d18a35c890 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/380925836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a339af0 0x55d18a3423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 -- 172.21.15.137:0/380925836 >> 172.21.15.137:0/380925836 conn(0x55d18a336060 msgr2=0x55d18a343bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 -- 172.21.15.137:0/380925836 shutdown_connections 2024-09-12T12:24:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.319+0000 7f9f18e7b740 1 -- 172.21.15.137:0/380925836 wait complete. 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 -- start start 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 Processor -- start 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a3488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a33a410 0x55d18a348e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a349370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101adfd0 con 0x55d18a339af0 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101af440 con 0x55d18a344cb0 2024-09-12T12:24:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f101ae6a0 con 0x55d18a33a410 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f18e7b740 10 client.0 _ll_register_callbacks cb 0x7f9f000d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a3488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f0f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a3488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58892/0 (socket says 172.21.15.137:58892) 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/2668773470 learned_addr learned my addr 172.21.15.137:0/2668773470 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f07fff640 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a33a410 0x55d18a348e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.320+0000 7f9f0ffff640 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a349370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/2668773470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a33a410 msgr2=0x55d18a348e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a33a410 0x55d18a348e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 msgr2=0x55d18a349370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a349370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f0f7fe640 1 -- 172.21.15.137:0/2668773470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d18a3be860 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f0f7fe640 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a3488f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9f000cfbe0 tx=0x7f9f0009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1959671e13d9ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9eeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9eeeffd640 1 -- 172.21.15.137:0/2668773470 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f000cd390 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9eeeffd640 1 -- 172.21.15.137:0/2668773470 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f9f00010430 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef8004c40 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9eeeffd640 1 -- 172.21.15.137:0/2668773470 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9f0009b8c0 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 10 client.16851 fetch_fsmap learned FSMap version 38 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d18a3bfa70 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9eeeffd640 1 -- 172.21.15.137:0/2668773470 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef8004c40 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9eeeffd640 1 -- 172.21.15.137:0/2668773470 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7f9f000a1410 con 0x55d18a339af0 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 10 client.16851 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T12:24:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:24:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 20 client.16851 populate_metadata read hostname 'smithi137' 2024-09-12T12:24:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a3c0180 0x55d18a3c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:24:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 4 client.16851 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:24:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.321+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d18a2a5d70 con 0x55d18a3c0180 2024-09-12T12:24:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.322+0000 7f9f07fff640 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a3c0180 0x55d18a3c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:24:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.322+0000 7f9eeeffd640 10 client.16851 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.322+0000 7f9f07fff640 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a3c0180 0x55d18a3c2570 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f331f456dcf4b4 server_cookie=e3a8b80124f16ef2 in_seq=0 out_seq=0 2024-09-12T12:24:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.413+0000 7f9eeeffd640 1 -- 172.21.15.137:0/2668773470 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d18a2a5d70 con 0x55d18a3c0180 2024-09-12T12:24:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.413+0000 7f9eeeffd640 10 client.16851 handle_command_reply: tid=0 2024-09-12T12:24:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.413+0000 7f9f18e7b740 1 client.16851 shutdown 2024-09-12T12:24:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.413+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 msgr2=0x55d18a3488f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.413+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a3488f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9f000cfbe0 tx=0x7f9f0009ae70 comp rx=0 tx=0).stop 2024-09-12T12:24:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 shutdown_connections 2024-09-12T12:24:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d18a344cb0 0x55d18a349370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d18a33a410 0x55d18a348e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d18a3c0180 0x55d18a3c2570 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9eeeffd640 0 client.16851 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 --2- 172.21.15.137:0/2668773470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d18a339af0 0x55d18a3488f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 >> 172.21.15.137:0/2668773470 conn(0x55d18a336060 msgr2=0x55d18a3349a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 shutdown_connections 2024-09-12T12:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 1 -- 172.21.15.137:0/2668773470 wait complete. 2024-09-12T12:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.414+0000 7f9f18e7b740 20 client.16851 trim_cache size 0 max 16384 2024-09-12T12:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9efc0a2da0 msgr2=0x7f9efc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9efc0a2da0 0x7f9efc0a5260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9f0801d000 tx=0x7f9f08019780 comp rx=0 tx=0).stop 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 msgr2=0x7f9f100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f100744f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000e9f0 tx=0x7f9f0000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 shutdown_connections 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9efc0a2da0 0x7f9efc0a5260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f10107b70 0x7f9f10074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10102470 0x7f9f10074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 --2- 172.21.15.137:0/1143003517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f10101ad0 0x7f9f100744f0 secure :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000e9f0 tx=0x7f9f0000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 >> 172.21.15.137:0/1143003517 conn(0x7f9f100fd7d0 msgr2=0x7f9f100fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 shutdown_connections 2024-09-12T12:24:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:24:54.415+0000 7f9f160dd640 1 -- 172.21.15.137:0/1143003517 wait complete. 2024-09-12T12:24:54.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 935345 7.74 MB/sec execute 509 sec latency 43.945 ms 2024-09-12T12:24:55.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:54 smithi137 ceph-mon[30950]: pgmap v367: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-09-12T12:24:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:54 smithi189 ceph-mon[35475]: pgmap v367: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-09-12T12:24:55.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:54 smithi171 ceph-mon[35098]: pgmap v367: 97 pgs: 97 active+clean; 255 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2024-09-12T12:24:55.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 938132 7.76 MB/sec execute 510 sec latency 15.405 ms 2024-09-12T12:24:56.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 939415 7.76 MB/sec execute 511 sec latency 33.388 ms 2024-09-12T12:24:57.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:56 smithi137 ceph-mon[30950]: pgmap v368: 97 pgs: 97 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-09-12T12:24:57.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:56 smithi189 ceph-mon[35475]: pgmap v368: 97 pgs: 97 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-09-12T12:24:57.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:56 smithi171 ceph-mon[35098]: pgmap v368: 97 pgs: 97 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.7 MiB/s wr, 443 op/s 2024-09-12T12:24:57.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 941835 7.77 MB/sec execute 512 sec latency 13.091 ms 2024-09-12T12:24:58.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 943424 7.77 MB/sec execute 513 sec latency 16.539 ms 2024-09-12T12:24:59.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:24:58 smithi137 ceph-mon[30950]: pgmap v369: 97 pgs: 97 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-09-12T12:24:59.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:24:58 smithi189 ceph-mon[35475]: pgmap v369: 97 pgs: 97 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-09-12T12:24:59.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:24:58 smithi171 ceph-mon[35098]: pgmap v369: 97 pgs: 97 active+clean; 250 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 276 op/s 2024-09-12T12:24:59.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 945581 7.78 MB/sec execute 514 sec latency 20.449 ms 2024-09-12T12:25:00.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 948085 7.79 MB/sec execute 515 sec latency 19.739 ms 2024-09-12T12:25:01.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:00 smithi171 ceph-mon[35098]: pgmap v370: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 389 op/s 2024-09-12T12:25:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:00 smithi137 ceph-mon[30950]: pgmap v370: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 389 op/s 2024-09-12T12:25:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:00 smithi189 ceph-mon[35475]: pgmap v370: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 389 op/s 2024-09-12T12:25:01.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 949112 7.79 MB/sec execute 516 sec latency 57.917 ms 2024-09-12T12:25:02.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 951576 7.80 MB/sec execute 517 sec latency 18.711 ms 2024-09-12T12:25:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:02 smithi137 ceph-mon[30950]: pgmap v371: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-09-12T12:25:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:02 smithi189 ceph-mon[35475]: pgmap v371: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-09-12T12:25:03.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:02 smithi171 ceph-mon[35098]: pgmap v371: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 275 op/s 2024-09-12T12:25:03.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 952602 7.79 MB/sec execute 518 sec latency 38.387 ms 2024-09-12T12:25:04.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:03 smithi137 ceph-mon[30950]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T12:25:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:03 smithi189 ceph-mon[35475]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T12:25:04.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:03 smithi171 ceph-mon[35098]: osdmap e100: 12 total, 12 up, 12 in 2024-09-12T12:25:04.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 954954 7.80 MB/sec execute 519 sec latency 18.379 ms 2024-09-12T12:25:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:04 smithi137 ceph-mon[30950]: pgmap v373: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 330 op/s 2024-09-12T12:25:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:04 smithi189 ceph-mon[35475]: pgmap v373: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 330 op/s 2024-09-12T12:25:05.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:04 smithi171 ceph-mon[35098]: pgmap v373: 97 pgs: 97 active+clean; 258 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 330 op/s 2024-09-12T12:25:05.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 956667 7.80 MB/sec execute 520 sec latency 15.430 ms 2024-09-12T12:25:06.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 958766 7.81 MB/sec execute 521 sec latency 67.692 ms 2024-09-12T12:25:06.977 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-12T12:25:06.977 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:25:07.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:06 smithi137 ceph-mon[30950]: pgmap v374: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-12T12:25:07.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:06 smithi189 ceph-mon[35475]: pgmap v374: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-12T12:25:07.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:06 smithi171 ceph-mon[35098]: pgmap v374: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-12T12:25:07.248 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:25:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 -- 172.21.15.137:0/617714803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc8101380 msgr2=0x7febc810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 --2- 172.21.15.137:0/617714803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc8101380 0x7febc810d8b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7febc0009cd0 tx=0x7febc002a8e0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 -- 172.21.15.137:0/617714803 shutdown_connections 2024-09-12T12:25:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 --2- 172.21.15.137:0/617714803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc8101380 0x7febc810d8b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 --2- 172.21.15.137:0/617714803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81009e0 0x7febc8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 --2- 172.21.15.137:0/617714803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81069f0 0x7febc8106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 -- 172.21.15.137:0/617714803 >> 172.21.15.137:0/617714803 conn(0x7febc80fc6c0 msgr2=0x7febc80feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 -- 172.21.15.137:0/617714803 shutdown_connections 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.507+0000 7febcf60e640 1 -- 172.21.15.137:0/617714803 wait complete. 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 Processor -- start 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 -- start start 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81009e0 0x7febc81994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc8101380 0x7febc81999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc81069f0 0x7febc81a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febc8110e30 con 0x7febc81069f0 2024-09-12T12:25:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7febc8110cb0 con 0x7febc81009e0 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcf60e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7febc8110fb0 con 0x7febc8101380 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81009e0 0x7febc81994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcdbab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc81069f0 0x7febc81a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81009e0 0x7febc81994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:47844/0 (socket says 172.21.15.137:47844) 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 -- 172.21.15.137:0/2602373406 learned_addr learned my addr 172.21.15.137:0/2602373406 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcdbab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc81069f0 0x7febc81a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37598/0 (socket says 172.21.15.137:37598) 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febccba9640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc8101380 0x7febc81999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 -- 172.21.15.137:0/2602373406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc8101380 msgr2=0x7febc81999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc8101380 0x7febc81999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.512 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 -- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc81069f0 msgr2=0x7febc81a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc81069f0 0x7febc81a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcd3aa640 1 -- 172.21.15.137:0/2602373406 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc81a4a70 con 0x7febc81009e0 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.508+0000 7febcdbab640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc81069f0 0x7febc81a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.509+0000 7febcd3aa640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81009e0 0x7febc81994b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febc400e9f0 tx=0x7febc400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.509+0000 7febbe7fc640 1 -- 172.21.15.137:0/2602373406 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc400cd80 con 0x7febc81009e0 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.509+0000 7febbe7fc640 1 -- 172.21.15.137:0/2602373406 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc4010910 con 0x7febc81009e0 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.509+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc81a4c40 con 0x7febc81009e0 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.509+0000 7febbe7fc640 1 -- 172.21.15.137:0/2602373406 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4018990 con 0x7febc81009e0 2024-09-12T12:25:07.513 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.509+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc81a5160 con 0x7febc81009e0 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.511+0000 7febbe7fc640 1 -- 172.21.15.137:0/2602373406 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7febc4010ac0 con 0x7febc81009e0 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 Processor -- start 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 -- start start 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104caf0 0x5627f104ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104d410 0x5627f1057770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f105a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5627f0fb12f0 con 0x5627f1057cb0 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5627f0f59270 con 0x5627f104caf0 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febd2485740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5627f0fad2b0 con 0x5627f104d410 2024-09-12T12:25:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febbe7fc640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7febb40a2da0 0x7febb40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f105a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104d410 0x5627f1057770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febcdbab640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104caf0 0x5627f104ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f105a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37600/0 (socket says 172.21.15.137:37600) 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 -- 172.21.15.137:0/357384203 learned_addr learned my addr 172.21.15.137:0/357384203 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febcdbab640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7febb40a2da0 0x7febb40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febbe7fc640 1 -- 172.21.15.137:0/2602373406 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8528+0+0 (secure 0 0 0) 0x7febc40cf700 con 0x7febc81009e0 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 -- 172.21.15.137:0/357384203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104d410 msgr2=0x5627f1057770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104d410 0x5627f1057770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 -- 172.21.15.137:0/357384203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104caf0 msgr2=0x5627f104ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104caf0 0x5627f104ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febccba9640 1 -- 172.21.15.137:0/357384203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f0f58340 con 0x5627f1057cb0 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.512+0000 7febcdbab640 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104caf0 0x5627f104ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.513+0000 7febccba9640 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f105a0a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7febb800b740 tx=0x7febb800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9909e4724d2b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.513+0000 7febcdbab640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7febb40a2da0 0x7febb40a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7febc0009430 tx=0x7febc00317e0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.513+0000 7febb37fe640 1 -- 172.21.15.137:0/357384203 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8003dd0 con 0x5627f1057cb0 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.513+0000 7febb37fe640 1 -- 172.21.15.137:0/357384203 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb8015070 con 0x5627f1057cb0 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.513+0000 7febb37fe640 1 -- 172.21.15.137:0/357384203 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8004e50 con 0x5627f1057cb0 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- 172.21.15.137:0/357384203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 msgr2=0x5627f105a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f105a0a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7febb800b740 tx=0x7febb800bc00 comp rx=0 tx=0).stop 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- 172.21.15.137:0/357384203 shutdown_connections 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f105a0a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104d410 0x5627f1057770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 --2- 172.21.15.137:0/357384203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104caf0 0x5627f104ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- 172.21.15.137:0/357384203 >> 172.21.15.137:0/357384203 conn(0x5627f1049060 msgr2=0x5627f1049440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- 172.21.15.137:0/357384203 shutdown_connections 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- 172.21.15.137:0/357384203 wait complete. 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- start start 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 Processor -- start 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f1054b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1055090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f10555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5627f105ec90 con 0x5627f104d410 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5627f0f3a290 con 0x5627f1057cb0 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5627f105ee10 con 0x5627f104caf0 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.514+0000 7febd2485740 10 client.0 _ll_register_callbacks cb 0x5627f0f6b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f1054b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1055090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1055090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37608/0 (socket says 172.21.15.137:37608) 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f1054b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:42480/0 (socket says 172.21.15.137:42480) 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 -- 172.21.15.137:0/738776290 learned_addr learned my addr 172.21.15.137:0/738776290 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febcdbab640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f10555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 -- 172.21.15.137:0/738776290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 msgr2=0x5627f1054b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f1054b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 -- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 msgr2=0x5627f10555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f10555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 -- 172.21.15.137:0/738776290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f0cc4210 con 0x5627f104d410 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febcd3aa640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f1054b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febcdbab640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f10555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febccba9640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1055090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7febb8004650 tx=0x7febb8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987d3b1a29740d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febb0ff9640 1 -- 172.21.15.137:0/738776290 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8028060 con 0x5627f104d410 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb8015070 con 0x5627f104d410 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febb0ff9640 1 -- 172.21.15.137:0/738776290 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8528+0+0 (secure 0 0 0) 0x7febb8011440 con 0x5627f104d410 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febb0ff9640 1 -- 172.21.15.137:0/738776290 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7febb8011640 con 0x5627f104d410 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febd2485740 10 client.16863 fetch_fsmap learned FSMap version 38 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.515+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5627f0d90620 con 0x5627f104d410 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febb0ff9640 1 -- 172.21.15.137:0/738776290 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb8015070 con 0x5627f104d410 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febb0ff9640 1 -- 172.21.15.137:0/738776290 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7febb8022070 con 0x5627f104d410 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febd2485740 10 client.16863 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febd2485740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febd2485740 20 client.16863 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febd2485740 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f106bfb0 0x5627f106e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febd2485740 4 client.16863 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627f0fbea20 con 0x5627f106bfb0 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febcd3aa640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f106bfb0 0x5627f106e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febcd3aa640 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f106bfb0 0x5627f106e3a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a1685ac1128d80 server_cookie=fe531424875698b5 in_seq=0 out_seq=0 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.516+0000 7febb0ff9640 10 client.16863 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febb0ff9640 1 -- 172.21.15.137:0/738776290 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627f0fbea20 con 0x5627f106bfb0 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febb0ff9640 10 client.16863 handle_command_reply: tid=0 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 client.16863 shutdown 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 msgr2=0x5627f1055090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1055090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7febb8004650 tx=0x7febb8004430 comp rx=0 tx=0).stop 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 shutdown_connections 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f10555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f106bfb0 0x5627f106e3a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1055090 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febb0ff9640 0 client.16863 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 --2- 172.21.15.137:0/738776290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f1054b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 >> 172.21.15.137:0/738776290 conn(0x5627f1049060 msgr2=0x5627f104ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 shutdown_connections 2024-09-12T12:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 1 -- 172.21.15.137:0/738776290 wait complete. 2024-09-12T12:25:07.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.517+0000 7febd2485740 20 client.16863 trim_cache size 0 max 16384 2024-09-12T12:25:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.533+0000 7febd2485740 1 Processor -- start 2024-09-12T12:25:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.533+0000 7febd2485740 1 -- start start 2024-09-12T12:25:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febd2485740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febd2485740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104d410 0x5627f1073970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febd2485740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f1073eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febd2485740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febc81adec0 con 0x5627f1057cb0 2024-09-12T12:25:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febd2485740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7febc81ab790 con 0x5627f104d410 2024-09-12T12:25:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febd2485740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7febc81a95e0 con 0x5627f104caf0 2024-09-12T12:25:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febcdbab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f1073eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febcdbab640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f1073eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37622/0 (socket says 172.21.15.137:37622) 2024-09-12T12:25:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febcdbab640 1 -- 172.21.15.137:0/912842455 learned_addr learned my addr 172.21.15.137:0/912842455 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febccba9640 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104d410 0x5627f1073970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febccba9640 1 -- 172.21.15.137:0/912842455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 msgr2=0x5627f10480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febccba9640 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febccba9640 1 -- 172.21.15.137:0/912842455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 msgr2=0x5627f1073eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febccba9640 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f1073eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febccba9640 1 -- 172.21.15.137:0/912842455 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627f1048e90 con 0x5627f104d410 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.534+0000 7febcdbab640 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f1073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.535+0000 7febccba9640 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104d410 0x5627f1073970 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5627f1056100 tx=0x7febb8001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52454ade3d58dace server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.535+0000 7febb2ffd640 1 -- 172.21.15.137:0/912842455 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8019c80 con 0x5627f104d410 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.535+0000 7febb2ffd640 1 -- 172.21.15.137:0/912842455 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb8015070 con 0x5627f104d410 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.535+0000 7febb2ffd640 1 -- 172.21.15.137:0/912842455 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8011a40 con 0x5627f104d410 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 -- 172.21.15.137:0/912842455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104d410 msgr2=0x5627f1073970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104d410 0x5627f1073970 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5627f1056100 tx=0x7febb8001e50 comp rx=0 tx=0).stop 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 -- 172.21.15.137:0/912842455 shutdown_connections 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f1057cb0 0x5627f1073eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f104d410 0x5627f1073970 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 --2- 172.21.15.137:0/912842455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 -- 172.21.15.137:0/912842455 >> 172.21.15.137:0/912842455 conn(0x5627f1049060 msgr2=0x5627f10479a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 -- 172.21.15.137:0/912842455 shutdown_connections 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 -- 172.21.15.137:0/912842455 wait complete. 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 -- start start 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 Processor -- start 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.536+0000 7febd2485740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1051a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f1052140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febc81a9c50 con 0x5627f104d410 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7febc81ab0c0 con 0x5627f1057cb0 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7febc81aa320 con 0x5627f104caf0 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 10 client.0 _ll_register_callbacks cb 0x7febc0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:07.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1051a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febcd3aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10649c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:42506/0 (socket says 172.21.15.137:42506) 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1051a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37632/0 (socket says 172.21.15.137:37632) 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febcd3aa640 1 -- 172.21.15.137:0/3319050893 learned_addr learned my addr 172.21.15.137:0/3319050893 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febcdbab640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f1052140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 -- 172.21.15.137:0/3319050893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 msgr2=0x5627f10649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 -- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 msgr2=0x5627f1052140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f1052140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 -- 172.21.15.137:0/3319050893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627f10d5940 con 0x5627f104d410 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febcd3aa640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febcdbab640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f1052140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febccba9640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1051a00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x5627f1055f80 tx=0x7febb8019cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92c3c9c09d11a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febb0ff9640 1 -- 172.21.15.137:0/3319050893 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8003de0 con 0x5627f104d410 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febb0ff9640 1 -- 172.21.15.137:0/3319050893 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8528+0+0 (secure 0 0 0) 0x7febb8019630 con 0x5627f104d410 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb8015070 con 0x5627f104d410 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febb0ff9640 1 -- 172.21.15.137:0/3319050893 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7febb80117e0 con 0x5627f104d410 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.537+0000 7febd2485740 10 client.16869 fetch_fsmap learned FSMap version 38 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5627f10d6af0 con 0x5627f104d410 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febb0ff9640 1 -- 172.21.15.137:0/3319050893 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb8015070 con 0x5627f104d410 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febb0ff9640 1 -- 172.21.15.137:0/3319050893 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 38) ==== 8346+0+0 (secure 0 0 0) 0x7febb803b030 con 0x5627f104d410 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febd2485740 10 client.16869 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febd2485740 10 client.16869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febd2485740 20 client.16869 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febd2485740 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f10d70f0 0x5627f10d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febd2485740 4 client.16869 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:25:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627f0fb8d70 con 0x5627f10d70f0 2024-09-12T12:25:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febcd3aa640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f10d70f0 0x5627f10d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febb0ff9640 10 client.16869 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.538+0000 7febcd3aa640 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f10d70f0 0x5627f10d94e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd198ba25bd9e758 server_cookie=808567596a74ba19 in_seq=0 out_seq=0 2024-09-12T12:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.640+0000 7febb0ff9640 1 -- 172.21.15.137:0/3319050893 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627f0fb8d70 con 0x5627f10d70f0 2024-09-12T12:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.640+0000 7febb0ff9640 10 client.16869 handle_command_reply: tid=0 2024-09-12T12:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.640+0000 7febd2485740 1 client.16869 shutdown 2024-09-12T12:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 msgr2=0x5627f1051a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1051a00 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x5627f1055f80 tx=0x7febb8019cc0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 shutdown_connections 2024-09-12T12:25:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627f1057cb0 0x5627f1052140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5627f10d70f0 0x5627f10d94e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febb0ff9640 0 client.16869 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5627f104d410 0x5627f1051a00 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 --2- 172.21.15.137:0/3319050893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5627f104caf0 0x5627f10649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 >> 172.21.15.137:0/3319050893 conn(0x5627f1049060 msgr2=0x5627f1054af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 shutdown_connections 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 1 -- 172.21.15.137:0/3319050893 wait complete. 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.641+0000 7febd2485740 20 client.16869 trim_cache size 0 max 16384 2024-09-12T12:25:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:25:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7febb40a2da0 msgr2=0x7febb40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7febb40a2da0 0x7febb40a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7febc0009430 tx=0x7febc00317e0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81009e0 msgr2=0x7febc81994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81009e0 0x7febc81994b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febc400e9f0 tx=0x7febc400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 shutdown_connections 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7febb40a2da0 0x7febb40a5260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febc81069f0 0x7febc81a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc8101380 0x7febc81999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 --2- 172.21.15.137:0/2602373406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7febc81009e0 0x7febc81994b0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7febc400e9f0 tx=0x7febc400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.642+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 >> 172.21.15.137:0/2602373406 conn(0x7febc80fc6c0 msgr2=0x7febc80fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.643+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 shutdown_connections 2024-09-12T12:25:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:07.643+0000 7febcf60e640 1 -- 172.21.15.137:0/2602373406 wait complete. 2024-09-12T12:25:07.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960193 7.81 MB/sec execute 522 sec latency 84.822 ms 2024-09-12T12:25:08.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 -- 172.21.15.137:0/2275492639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 msgr2=0x7f2c041021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2275492639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 0x7f2c041021d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009900 tx=0x7f2c0002a410 comp rx=0 tx=0).stop 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 -- 172.21.15.137:0/2275492639 shutdown_connections 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2275492639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c04106f20 0x7f2c0410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2275492639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c041027a0 0x7f2c041067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2275492639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 0x7f2c041021d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 -- 172.21.15.137:0/2275492639 >> 172.21.15.137:0/2275492639 conn(0x7f2c040fd8f0 msgr2=0x7f2c040ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 -- 172.21.15.137:0/2275492639 shutdown_connections 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.898+0000 7f2c0c583640 1 -- 172.21.15.137:0/2275492639 wait complete. 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 Processor -- start 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 -- start start 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 0x7f2c04198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:08.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c041027a0 0x7f2c04198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c04106f20 0x7f2c0419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04111f80 con 0x7f2c04101df0 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04111e00 con 0x7f2c041027a0 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0c583640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04112100 con 0x7f2c04106f20 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.899+0000 7f2c0a31f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 0x7f2c04198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c0a31f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 0x7f2c04198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37638/0 (socket says 172.21.15.137:37638) 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c0a31f640 1 -- 172.21.15.137:0/2336393952 learned_addr learned my addr 172.21.15.137:0/2336393952 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c0ab20640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c04106f20 0x7f2c0419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c09b1e640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c041027a0 0x7f2c04198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c09b1e640 1 -- 172.21.15.137:0/2336393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c04106f20 msgr2=0x7f2c0419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c09b1e640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c04106f20 0x7f2c0419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c09b1e640 1 -- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 msgr2=0x7f2c04198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:08.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c09b1e640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 0x7f2c04198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c09b1e640 1 -- 172.21.15.137:0/2336393952 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0419d4b0 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c0ab20640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c04106f20 0x7f2c0419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c09b1e640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c041027a0 0x7f2c04198a40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e9f0 tx=0x7f2bf400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2bfb7fe640 1 -- 172.21.15.137:0/2336393952 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf400ccc0 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2bfb7fe640 1 -- 172.21.15.137:0/2336393952 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bf4005020 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0419d7a0 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2bfb7fe640 1 -- 172.21.15.137:0/2336393952 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40049e0 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.900+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c041a5080 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.902+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c041021d0 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.903+0000 7f2bfb7fe640 1 -- 172.21.15.137:0/2336393952 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2bf4005560 con 0x7f2c041027a0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.903+0000 7f2bfb7fe640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2bf00a2cf0 0x7f2bf00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.903+0000 7f2c0a31f640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2bf00a2cf0 0x7f2bf00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:08.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.903+0000 7f2bfb7fe640 1 -- 172.21.15.137:0/2336393952 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f2bf40cf3a0 con 0x7f2c041027a0 2024-09-12T12:25:08.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.904+0000 7f2c0a31f640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2bf00a2cf0 0x7f2bf00a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c000096c0 tx=0x7f2c00005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:08.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:07.908+0000 7f2bfb7fe640 1 -- 172.21.15.137:0/2336393952 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bf40051d0 con 0x7f2c041027a0 2024-09-12T12:25:08.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.122+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c0410a100 con 0x7f2c041027a0 2024-09-12T12:25:08.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.123+0000 7f2bfb7fe640 1 -- 172.21.15.137:0/2336393952 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11934 (secure 0 0 0) 0x7f2bf4084e20 con 0x7f2c041027a0 2024-09-12T12:25:08.124 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:25:08.124 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":39,"btime":"2024-09-12T12:25:07:764471+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:25:06.983730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:25:08.124 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 39 2024-09-12T12:25:08.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2bf00a2cf0 msgr2=0x7f2bf00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:08.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2bf00a2cf0 0x7f2bf00a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c000096c0 tx=0x7f2c00005a20 comp rx=0 tx=0).stop 2024-09-12T12:25:08.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c041027a0 msgr2=0x7f2c04198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:08.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c041027a0 0x7f2c04198a40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400e9f0 tx=0x7f2bf400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 shutdown_connections 2024-09-12T12:25:08.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2bf00a2cf0 0x7f2bf00a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2c04106f20 0x7f2c0419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c041027a0 0x7f2c04198a40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 --2- 172.21.15.137:0/2336393952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04101df0 0x7f2c04198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:08.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.125+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 >> 172.21.15.137:0/2336393952 conn(0x7f2c040fd8f0 msgr2=0x7f2c040fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:08.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.126+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 shutdown_connections 2024-09-12T12:25:08.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:08.126+0000 7f2c0c583640 1 -- 172.21.15.137:0/2336393952 wait complete. 2024-09-12T12:25:08.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.80 MB/sec execute 523 sec latency 740.972 ms 2024-09-12T12:25:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:08 smithi137 ceph-mon[30950]: pgmap v375: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-12T12:25:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:08 smithi137 ceph-mon[30950]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T12:25:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:08 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:25:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:08 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2336393952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:25:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:08 smithi171 ceph-mon[35098]: pgmap v375: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-12T12:25:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:08 smithi171 ceph-mon[35098]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T12:25:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:08 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:25:08.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:08 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2336393952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:25:08.981 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-12T12:25:08.981 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-12T12:25:09.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:08 smithi189 ceph-mon[35475]: pgmap v375: 97 pgs: 97 active+clean; 265 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 304 op/s 2024-09-12T12:25:09.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:08 smithi189 ceph-mon[35475]: osdmap e101: 12 total, 12 up, 12 in 2024-09-12T12:25:09.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:08 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:25:09.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:08 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2336393952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:25:09.232 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:25:09.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.79 MB/sec execute 524 sec latency 1741.000 ms 2024-09-12T12:25:09.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 -- 172.21.15.137:0/339489095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34106f10 msgr2=0x7faa3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 --2- 172.21.15.137:0/339489095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34106f10 0x7faa3410ea90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009ca0 tx=0x7faa2c02a8c0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 -- 172.21.15.137:0/339489095 shutdown_connections 2024-09-12T12:25:09.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 --2- 172.21.15.137:0/339489095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34106f10 0x7faa3410ea90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 --2- 172.21.15.137:0/339489095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa34102790 0x7faa341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 --2- 172.21.15.137:0/339489095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa34101de0 0x7faa341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 -- 172.21.15.137:0/339489095 >> 172.21.15.137:0/339489095 conn(0x7faa340fd8c0 msgr2=0x7faa340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:09.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 -- 172.21.15.137:0/339489095 shutdown_connections 2024-09-12T12:25:09.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.749+0000 7faa38b32640 1 -- 172.21.15.137:0/339489095 wait complete. 2024-09-12T12:25:09.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.750+0000 7faa38b32640 1 Processor -- start 2024-09-12T12:25:09.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.750+0000 7faa38b32640 1 -- start start 2024-09-12T12:25:09.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa38b32640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34101de0 0x7faa341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa38b32640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa34102790 0x7faa34198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa38b32640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa34106f10 0x7faa3419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa38b32640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa34111f70 con 0x7faa34101de0 2024-09-12T12:25:09.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa38b32640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faa34111df0 con 0x7faa34106f10 2024-09-12T12:25:09.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa38b32640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa341120f0 con 0x7faa34102790 2024-09-12T12:25:09.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34101de0 0x7faa341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34101de0 0x7faa341983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60682/0 (socket says 172.21.15.137:60682) 2024-09-12T12:25:09.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 -- 172.21.15.137:0/2315210812 learned_addr learned my addr 172.21.15.137:0/2315210812 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:09.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa31d9b640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa34102790 0x7faa34198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa32d9d640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa34106f10 0x7faa3419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 -- 172.21.15.137:0/2315210812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa34102790 msgr2=0x7faa34198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa34102790 0x7faa34198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 -- 172.21.15.137:0/2315210812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa34106f10 msgr2=0x7faa3419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa34106f10 0x7faa3419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 -- 172.21.15.137:0/2315210812 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3419d3a0 con 0x7faa34101de0 2024-09-12T12:25:09.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa31d9b640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa34102790 0x7faa34198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:09.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa32d9d640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa34106f10 0x7faa3419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:09.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa3259c640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34101de0 0x7faa341983f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faa2400d950 tx=0x7faa2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:09.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa237fe640 1 -- 172.21.15.137:0/2315210812 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24017030 con 0x7faa34101de0 2024-09-12T12:25:09.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3419d690 con 0x7faa34101de0 2024-09-12T12:25:09.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.751+0000 7faa237fe640 1 -- 172.21.15.137:0/2315210812 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa24004020 con 0x7faa34101de0 2024-09-12T12:25:09.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.752+0000 7faa237fe640 1 -- 172.21.15.137:0/2315210812 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240047a0 con 0x7faa34101de0 2024-09-12T12:25:09.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.752+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa341a4f70 con 0x7faa34101de0 2024-09-12T12:25:09.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa237fe640 1 -- 172.21.15.137:0/2315210812 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7faa240041d0 con 0x7faa34101de0 2024-09-12T12:25:09.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 Processor -- start 2024-09-12T12:25:09.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 -- start start 2024-09-12T12:25:09.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0eb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0e1c2a0 con 0x55f2b0eb6620 2024-09-12T12:25:09.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0dc3b10 con 0x55f2b0eb6f40 2024-09-12T12:25:09.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa3af0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0e18260 con 0x55f2b0ec5d50 2024-09-12T12:25:09.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.754+0000 7faa237fe640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa1c0a2cf0 0x7faa1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0eb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0eb6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60686/0 (socket says 172.21.15.137:60686) 2024-09-12T12:25:09.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 -- 172.21.15.137:0/3913518234 learned_addr learned my addr 172.21.15.137:0/3913518234 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:09.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa1c0a2cf0 0x7faa1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa31d9b640 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa237fe640 1 -- 172.21.15.137:0/2315210812 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7faa240ceed0 con 0x7faa34101de0 2024-09-12T12:25:09.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa3259c640 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 -- 172.21.15.137:0/3913518234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 msgr2=0x55f2b0ec8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 -- 172.21.15.137:0/3913518234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6f40 msgr2=0x55f2b0ec5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 -- 172.21.15.137:0/3913518234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b0b8db90 con 0x55f2b0eb6620 2024-09-12T12:25:09.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0eb6a00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faa2c006010 tx=0x7faa2c031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d1f8bcecfb5616 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:09.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa32d9d640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa1c0a2cf0 0x7faa1c0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009bf0 tx=0x7faa2c047000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:09.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa20ff9640 1 -- 172.21.15.137:0/3913518234 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c0305c0 con 0x55f2b0eb6620 2024-09-12T12:25:09.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa20ff9640 1 -- 172.21.15.137:0/3913518234 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2c036040 con 0x55f2b0eb6620 2024-09-12T12:25:09.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.755+0000 7faa20ff9640 1 -- 172.21.15.137:0/3913518234 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c049670 con 0x55f2b0eb6620 2024-09-12T12:25:09.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 -- 172.21.15.137:0/3913518234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 msgr2=0x55f2b0eb6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0eb6a00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faa2c006010 tx=0x7faa2c031870 comp rx=0 tx=0).stop 2024-09-12T12:25:09.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 -- 172.21.15.137:0/3913518234 shutdown_connections 2024-09-12T12:25:09.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 --2- 172.21.15.137:0/3913518234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0eb6a00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 -- 172.21.15.137:0/3913518234 >> 172.21.15.137:0/3913518234 conn(0x55f2b0eb2cf0 msgr2=0x55f2b0eb30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:09.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 -- 172.21.15.137:0/3913518234 shutdown_connections 2024-09-12T12:25:09.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.756+0000 7faa3af0b740 1 -- 172.21.15.137:0/3913518234 wait complete. 2024-09-12T12:25:09.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 -- start start 2024-09-12T12:25:09.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 Processor -- start 2024-09-12T12:25:09.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0ec88c0 con 0x55f2b0eb6f40 2024-09-12T12:25:09.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0da46e0 con 0x55f2b0ec5d50 2024-09-12T12:25:09.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f2b0ec8a40 con 0x55f2b0eb6620 2024-09-12T12:25:09.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3af0b740 10 client.0 _ll_register_callbacks cb 0x55f2b0dc8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:09.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60688/0 (socket says 172.21.15.137:60688) 2024-09-12T12:25:09.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 -- 172.21.15.137:0/1682827479 learned_addr learned my addr 172.21.15.137:0/1682827479 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:09.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa3259c640 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa32d9d640 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 -- 172.21.15.137:0/1682827479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6620 msgr2=0x55f2b0ec2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 -- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 msgr2=0x55f2b0ec37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 -- 172.21.15.137:0/1682827479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b0ed1f40 con 0x55f2b0eb6f40 2024-09-12T12:25:09.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa31d9b640 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec32a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7faa280106e0 tx=0x7faa28010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab82f26e95eb5c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:09.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.757+0000 7faa19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:09.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa19ffb640 1 -- 172.21.15.137:0/1682827479 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2801d060 con 0x55f2b0eb6f40 2024-09-12T12:25:09.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa19ffb640 1 -- 172.21.15.137:0/1682827479 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7faa2800e8f0 con 0x55f2b0eb6f40 2024-09-12T12:25:09.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2c036040 con 0x55f2b0eb6f40 2024-09-12T12:25:09.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa19ffb640 1 -- 172.21.15.137:0/1682827479 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faa2800eaf0 con 0x55f2b0eb6f40 2024-09-12T12:25:09.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 10 client.16887 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:09.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f2b0ed4930 con 0x55f2b0eb6f40 2024-09-12T12:25:09.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa19ffb640 1 -- 172.21.15.137:0/1682827479 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2c036040 con 0x55f2b0eb6f40 2024-09-12T12:25:09.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa19ffb640 1 -- 172.21.15.137:0/1682827479 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7faa28020390 con 0x55f2b0eb6f40 2024-09-12T12:25:09.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 10 client.16887 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:09.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 10 client.16887 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:25:09.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 20 client.16887 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:09.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0ed5e90 0x55f2b0ed82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 4 client.16887 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:09.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2b0e299d0 con 0x55f2b0ed5e90 2024-09-12T12:25:09.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.758+0000 7faa3259c640 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0ed5e90 0x55f2b0ed82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.759+0000 7faa19ffb640 10 client.16887 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:09.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.759+0000 7faa3259c640 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0ed5e90 0x55f2b0ed82b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c375c791c6cfb8 server_cookie=bfcee6afd1dd90d0 in_seq=0 out_seq=0 2024-09-12T12:25:09.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.759+0000 7faa19ffb640 1 -- 172.21.15.137:0/1682827479 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2b0e299d0 con 0x55f2b0ed5e90 2024-09-12T12:25:09.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.759+0000 7faa19ffb640 10 client.16887 handle_command_reply: tid=0 2024-09-12T12:25:09.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.759+0000 7faa3af0b740 1 client.16887 shutdown 2024-09-12T12:25:09.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 msgr2=0x55f2b0ec32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec32a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7faa280106e0 tx=0x7faa28010ae0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 shutdown_connections 2024-09-12T12:25:09.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ec37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0ed5e90 0x55f2b0ed82b0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec32a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 --2- 172.21.15.137:0/1682827479 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa19ffb640 0 client.16887 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:09.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 >> 172.21.15.137:0/1682827479 conn(0x55f2b0eb2cf0 msgr2=0x55f2b0eb8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:09.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 shutdown_connections 2024-09-12T12:25:09.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 1 -- 172.21.15.137:0/1682827479 wait complete. 2024-09-12T12:25:09.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.760+0000 7faa3af0b740 20 client.16887 trim_cache size 0 max 16384 2024-09-12T12:25:09.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 Processor -- start 2024-09-12T12:25:09.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 -- start start 2024-09-12T12:25:09.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0eddb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa341adf10 con 0x55f2b0eb6f40 2024-09-12T12:25:09.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faa341ab7b0 con 0x55f2b0eb6620 2024-09-12T12:25:09.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.776+0000 7faa3af0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa341a9600 con 0x55f2b0ec5d50 2024-09-12T12:25:09.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60694/0 (socket says 172.21.15.137:60694) 2024-09-12T12:25:09.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 -- 172.21.15.137:0/4106841915 learned_addr learned my addr 172.21.15.137:0/4106841915 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:09.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa3259c640 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 -- 172.21.15.137:0/4106841915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 msgr2=0x55f2b0eddb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa32d9d640 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0eddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0eddb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 -- 172.21.15.137:0/4106841915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6620 msgr2=0x55f2b0ec4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 -- 172.21.15.137:0/4106841915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2b0eb2580 con 0x55f2b0eb6f40 2024-09-12T12:25:09.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa3259c640 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:09.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa32d9d640 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0eddb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:09.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa31d9b640 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec3630 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7faa280106b0 tx=0x7faa28005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9affbec2ac8d57e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:09.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa1bfff640 1 -- 172.21.15.137:0/4106841915 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2802d020 con 0x55f2b0eb6f40 2024-09-12T12:25:09.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa1bfff640 1 -- 172.21.15.137:0/4106841915 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa28020a20 con 0x55f2b0eb6f40 2024-09-12T12:25:09.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.777+0000 7faa1bfff640 1 -- 172.21.15.137:0/4106841915 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2801d030 con 0x55f2b0eb6f40 2024-09-12T12:25:09.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 -- 172.21.15.137:0/4106841915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 msgr2=0x55f2b0ec3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec3630 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7faa280106b0 tx=0x7faa28005df0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 -- 172.21.15.137:0/4106841915 shutdown_connections 2024-09-12T12:25:09.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0ec5d50 0x55f2b0eddb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6f40 0x55f2b0ec3630 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 --2- 172.21.15.137:0/4106841915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0eb6620 0x55f2b0ec4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 -- 172.21.15.137:0/4106841915 >> 172.21.15.137:0/4106841915 conn(0x55f2b0eb2cf0 msgr2=0x55f2b0eb1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:09.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 -- 172.21.15.137:0/4106841915 shutdown_connections 2024-09-12T12:25:09.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 -- 172.21.15.137:0/4106841915 wait complete. 2024-09-12T12:25:09.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 -- start start 2024-09-12T12:25:09.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.778+0000 7faa3af0b740 1 Processor -- start 2024-09-12T12:25:09.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0ece310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6f40 0x55f2b0eca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3af0b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ecab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3af0b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faa341a9c70 con 0x55f2b0eb6620 2024-09-12T12:25:09.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3af0b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faa341ab0e0 con 0x55f2b0ec5d50 2024-09-12T12:25:09.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3af0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faa341aa340 con 0x55f2b0eb6f40 2024-09-12T12:25:09.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3af0b740 10 client.0 _ll_register_callbacks cb 0x7faa2800beb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:09.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0ece310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0ece310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60698/0 (socket says 172.21.15.137:60698) 2024-09-12T12:25:09.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 -- 172.21.15.137:0/2883640367 learned_addr learned my addr 172.21.15.137:0/2883640367 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:09.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa32d9d640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ecab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa31d9b640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6f40 0x55f2b0eca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 -- 172.21.15.137:0/2883640367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6f40 msgr2=0x55f2b0eca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6f40 0x55f2b0eca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 -- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 msgr2=0x55f2b0ecab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:09.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ecab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 -- 172.21.15.137:0/2883640367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2b0ecb280 con 0x55f2b0eb6620 2024-09-12T12:25:09.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa32d9d640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ecab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:09.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa31d9b640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6f40 0x55f2b0eca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:09.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa3259c640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0ece310 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faa24009fd0 tx=0x7faa24097050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e158c6e396d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:09.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:09.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa19ffb640 1 -- 172.21.15.137:0/2883640367 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2409b410 con 0x55f2b0eb6620 2024-09-12T12:25:09.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.779+0000 7faa19ffb640 1 -- 172.21.15.137:0/2883640367 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7faa24097a80 con 0x55f2b0eb6620 2024-09-12T12:25:09.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa28020a20 con 0x55f2b0eb6620 2024-09-12T12:25:09.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa19ffb640 1 -- 172.21.15.137:0/2883640367 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7faa2409f7c0 con 0x55f2b0eb6620 2024-09-12T12:25:09.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 10 client.16893 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:09.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f2b0f419b0 con 0x55f2b0eb6620 2024-09-12T12:25:09.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa19ffb640 1 -- 172.21.15.137:0/2883640367 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa28020a20 con 0x55f2b0eb6620 2024-09-12T12:25:09.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa19ffb640 1 -- 172.21.15.137:0/2883640367 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7faa240662a0 con 0x55f2b0eb6620 2024-09-12T12:25:09.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 10 client.16893 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:09.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 10 client.16893 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:25:09.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 20 client.16893 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:09.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0f420f0 0x55f2b0f44530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:09.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 4 client.16893 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-12T12:25:09.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x55f2b0e23d20 con 0x55f2b0f420f0 2024-09-12T12:25:09.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.780+0000 7faa31d9b640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0f420f0 0x55f2b0f44530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:09.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.781+0000 7faa31d9b640 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0f420f0 0x55f2b0f44530 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf9ce45aac539a6 server_cookie=94fce8687c0b0c13 in_seq=0 out_seq=0 2024-09-12T12:25:09.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.781+0000 7faa19ffb640 10 client.16893 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.872+0000 7faa19ffb640 1 -- 172.21.15.137:0/2883640367 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f2b0e23d20 con 0x55f2b0f420f0 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.872+0000 7faa19ffb640 10 client.16893 handle_command_reply: tid=0 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.872+0000 7faa3af0b740 1 client.16893 shutdown 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 msgr2=0x55f2b0ece310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0ece310 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faa24009fd0 tx=0x7faa24097050 comp rx=0 tx=0).stop 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 shutdown_connections 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f2b0ec5d50 0x55f2b0ecab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f2b0eb6f40 0x55f2b0eca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55f2b0f420f0 0x55f2b0f44530 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 --2- 172.21.15.137:0/2883640367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b0eb6620 0x55f2b0ece310 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa19ffb640 0 client.16893 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 >> 172.21.15.137:0/2883640367 conn(0x55f2b0eb2cf0 msgr2=0x55f2b0eb1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 shutdown_connections 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 1 -- 172.21.15.137:0/2883640367 wait complete. 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.873+0000 7faa3af0b740 20 client.16893 trim_cache size 0 max 16384 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:25:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 39, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 46, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "7c925699": { 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 46, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 147.2, 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:25:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:09.877 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa1c0a2cf0 msgr2=0x7faa1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa1c0a2cf0 0x7faa1c0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009bf0 tx=0x7faa2c047000 comp rx=0 tx=0).stop 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34101de0 msgr2=0x7faa341983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34101de0 0x7faa341983f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7faa2400d950 tx=0x7faa2400de10 comp rx=0 tx=0).stop 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 shutdown_connections 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faa1c0a2cf0 0x7faa1c0a51b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa34106f10 0x7faa3419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa34102790 0x7faa34198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 --2- 172.21.15.137:0/2315210812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa34101de0 0x7faa341983f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:09.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 >> 172.21.15.137:0/2315210812 conn(0x7faa340fd8c0 msgr2=0x7faa340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:09.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 shutdown_connections 2024-09-12T12:25:09.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:09.874+0000 7faa38b32640 1 -- 172.21.15.137:0/2315210812 wait complete. 2024-09-12T12:25:10.591 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7c925699', quiesce took 3.6 seconds. Will release after: 23.599999999999998 2024-09-12T12:25:10.759 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.77 MB/sec execute 525 sec latency 2741.388 ms 2024-09-12T12:25:10.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:10 smithi137 ceph-mon[30950]: pgmap v377: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 415 op/s 2024-09-12T12:25:10.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:10 smithi171 ceph-mon[35098]: pgmap v377: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 415 op/s 2024-09-12T12:25:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:10 smithi189 ceph-mon[35475]: pgmap v377: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 415 op/s 2024-09-12T12:25:11.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.76 MB/sec execute 526 sec latency 3741.478 ms 2024-09-12T12:25:12.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.74 MB/sec execute 527 sec latency 4741.585 ms 2024-09-12T12:25:12.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:12 smithi171 ceph-mon[35098]: pgmap v378: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 378 op/s 2024-09-12T12:25:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:12 smithi137 ceph-mon[30950]: pgmap v378: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 378 op/s 2024-09-12T12:25:13.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:12 smithi189 ceph-mon[35475]: pgmap v378: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 378 op/s 2024-09-12T12:25:13.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.73 MB/sec execute 528 sec latency 5741.690 ms 2024-09-12T12:25:14.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.71 MB/sec execute 529 sec latency 6741.816 ms 2024-09-12T12:25:14.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:14 smithi171 ceph-mon[35098]: pgmap v379: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-09-12T12:25:15.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:14 smithi137 ceph-mon[30950]: pgmap v379: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-09-12T12:25:15.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:14 smithi189 ceph-mon[35475]: pgmap v379: 97 pgs: 97 active+clean; 259 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-09-12T12:25:15.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.70 MB/sec execute 530 sec latency 7741.916 ms 2024-09-12T12:25:16.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.68 MB/sec execute 531 sec latency 8742.011 ms 2024-09-12T12:25:16.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:16 smithi171 ceph-mon[35098]: pgmap v380: 97 pgs: 97 active+clean; 264 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-12T12:25:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:16 smithi137 ceph-mon[30950]: pgmap v380: 97 pgs: 97 active+clean; 264 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-12T12:25:17.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:16 smithi189 ceph-mon[35475]: pgmap v380: 97 pgs: 97 active+clean; 264 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-12T12:25:17.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.67 MB/sec execute 532 sec latency 9742.120 ms 2024-09-12T12:25:18.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.66 MB/sec execute 533 sec latency 10742.168 ms 2024-09-12T12:25:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:18 smithi171 ceph-mon[35098]: pgmap v381: 97 pgs: 97 active+clean; 264 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-12T12:25:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:18 smithi137 ceph-mon[30950]: pgmap v381: 97 pgs: 97 active+clean; 264 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-12T12:25:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:18 smithi189 ceph-mon[35475]: pgmap v381: 97 pgs: 97 active+clean; 264 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-12T12:25:19.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.64 MB/sec execute 534 sec latency 11742.231 ms 2024-09-12T12:25:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 -- 172.21.15.137:0/2195147407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 msgr2=0x7f1520103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2195147407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f1520103100 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1510009900 tx=0x7f151002a480 comp rx=0 tx=0).stop 2024-09-12T12:25:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 -- 172.21.15.137:0/2195147407 shutdown_connections 2024-09-12T12:25:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2195147407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f152010c0f0 0x7f152010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2195147407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1520103640 0x7f152010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2195147407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f1520103100 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 -- 172.21.15.137:0/2195147407 >> 172.21.15.137:0/2195147407 conn(0x7f15200fc6b0 msgr2=0x7f15200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 -- 172.21.15.137:0/2195147407 shutdown_connections 2024-09-12T12:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.741+0000 7f1525ba6640 1 -- 172.21.15.137:0/2195147407 wait complete. 2024-09-12T12:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 Processor -- start 2024-09-12T12:25:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 -- start start 2024-09-12T12:25:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f15201983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1520103640 0x7f15201988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f152010c0f0 0x7f152019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1520111ef0 con 0x7f1520102d20 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1520111d70 con 0x7f152010c0f0 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f1525ba6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1520112070 con 0x7f1520103640 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f151f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f15201983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f151f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f15201983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34058/0 (socket says 172.21.15.137:34058) 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f151f7fe640 1 -- 172.21.15.137:0/2621253105 learned_addr learned my addr 172.21.15.137:0/2621253105 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f151effd640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1520103640 0x7f15201988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.742+0000 7f151ffff640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f152010c0f0 0x7f152019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151f7fe640 1 -- 172.21.15.137:0/2621253105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1520103640 msgr2=0x7f15201988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1520103640 0x7f15201988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151f7fe640 1 -- 172.21.15.137:0/2621253105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f152010c0f0 msgr2=0x7f152019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f152010c0f0 0x7f152019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151f7fe640 1 -- 172.21.15.137:0/2621253105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152019d390 con 0x7f1520102d20 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151effd640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1520103640 0x7f15201988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f15201983a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1510006010 tx=0x7f15100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151cff9640 1 -- 172.21.15.137:0/2621253105 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151003f030 con 0x7f1520102d20 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151cff9640 1 -- 172.21.15.137:0/2621253105 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1510003ce0 con 0x7f1520102d20 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152019d620 con 0x7f1520102d20 2024-09-12T12:25:20.747 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f151cff9640 1 -- 172.21.15.137:0/2621253105 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510035070 con 0x7f1520102d20 2024-09-12T12:25:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.743+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15201a4f10 con 0x7f1520102d20 2024-09-12T12:25:20.754 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.745+0000 7f151cff9640 1 -- 172.21.15.137:0/2621253105 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1510042030 con 0x7f1520102d20 2024-09-12T12:25:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 Processor -- start 2024-09-12T12:25:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 -- start start 2024-09-12T12:25:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151cff9640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15080a2da0 0x7f15080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc1410 0x5586f7ccb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5586f7c252f0 con 0x5586f7ccbcb0 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5586f7bcd270 con 0x5586f7cc0af0 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f1528a20740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586f7c212b0 con 0x5586f7cc1410 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151ffff640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15080a2da0 0x7f15080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151cff9640 1 -- 172.21.15.137:0/2621253105 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f15100efb50 con 0x7f1520102d20 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34074/0 (socket says 172.21.15.137:34074) 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151ffff640 1 -- 172.21.15.137:0/298762388 learned_addr learned my addr 172.21.15.137:0/298762388 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151effd640 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc1410 0x5586f7ccb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.746+0000 7f151f7fe640 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151ffff640 1 -- 172.21.15.137:0/298762388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc1410 msgr2=0x5586f7ccb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151ffff640 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc1410 0x5586f7ccb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151ffff640 1 -- 172.21.15.137:0/298762388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 msgr2=0x5586f7cc0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151ffff640 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151ffff640 1 -- 172.21.15.137:0/298762388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f7bcc340 con 0x5586f7ccbcb0 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151effd640 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc1410 0x5586f7ccb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151ffff640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15080a2da0 0x7f15080a5260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1518013b30 tx=0x7f151800bd10 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f151ffff640 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cce0a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1518010d30 tx=0x7f151801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27d99683dd24a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f1505ffb640 1 -- 172.21.15.137:0/298762388 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518021070 con 0x5586f7ccbcb0 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f1505ffb640 1 -- 172.21.15.137:0/298762388 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f151800ee50 con 0x5586f7ccbcb0 2024-09-12T12:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.747+0000 7f1505ffb640 1 -- 172.21.15.137:0/298762388 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518004440 con 0x5586f7ccbcb0 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 -- 172.21.15.137:0/298762388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 msgr2=0x5586f7cce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cce0a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1518010d30 tx=0x7f151801a000 comp rx=0 tx=0).stop 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 -- 172.21.15.137:0/298762388 shutdown_connections 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cce0a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc1410 0x5586f7ccb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 --2- 172.21.15.137:0/298762388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 -- 172.21.15.137:0/298762388 >> 172.21.15.137:0/298762388 conn(0x5586f7cbd060 msgr2=0x5586f7cbd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 -- 172.21.15.137:0/298762388 shutdown_connections 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 -- 172.21.15.137:0/298762388 wait complete. 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.748+0000 7f1528a20740 1 -- start start 2024-09-12T12:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 1 Processor -- start 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cc8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7cc9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7ccbcb0 0x5586f7cc9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5586f7cce880 con 0x5586f7cc1410 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5586f7bae290 con 0x5586f7ccbcb0 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586f7ccea00 con 0x5586f7cc0af0 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f1528a20740 10 client.0 _ll_register_callbacks cb 0x5586f7bdf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7cc9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7cc9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34084/0 (socket says 172.21.15.137:34084) 2024-09-12T12:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 -- 172.21.15.137:0/2004218332 learned_addr learned my addr 172.21.15.137:0/2004218332 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cc8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 -- 172.21.15.137:0/2004218332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 msgr2=0x5586f7cc8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cc8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 -- 172.21.15.137:0/2004218332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7ccbcb0 msgr2=0x5586f7cc9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7ccbcb0 0x5586f7cc9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 -- 172.21.15.137:0/2004218332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f7938210 con 0x5586f7cc1410 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cc8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f151effd640 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7cc9110 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1514012480 tx=0x7f1514012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d18cd007a4e513 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f14feffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.749+0000 7f14feffd640 1 -- 172.21.15.137:0/2004218332 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514005220 con 0x5586f7cc1410 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f14feffd640 1 -- 172.21.15.137:0/2004218332 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f15140053c0 con 0x5586f7cc1410 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f151800ee50 con 0x5586f7cc1410 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f14feffd640 1 -- 172.21.15.137:0/2004218332 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1514005600 con 0x5586f7cc1410 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 10 client.16905 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586f7a04620 con 0x5586f7cc1410 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f14feffd640 1 -- 172.21.15.137:0/2004218332 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f151800ee50 con 0x5586f7cc1410 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f14feffd640 1 -- 172.21.15.137:0/2004218332 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f1514021020 con 0x5586f7cc1410 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 10 client.16905 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 20 client.16905 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7cdbe30 0x5586f7cde2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 4 client.16905 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586f7c32a20 con 0x5586f7cdbe30 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.750+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7cdbe30 0x5586f7cde2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.751+0000 7f14feffd640 10 client.16905 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.751+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7cdbe30 0x5586f7cde2a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58eb9fe44726a9d server_cookie=cc510c9c08e4db29 in_seq=0 out_seq=0 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f14feffd640 1 -- 172.21.15.137:0/2004218332 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586f7c32a20 con 0x5586f7cdbe30 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f14feffd640 10 client.16905 handle_command_reply: tid=0 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 client.16905 shutdown 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 msgr2=0x5586f7cc9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7cc9110 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1514012480 tx=0x7f1514012940 comp rx=0 tx=0).stop 2024-09-12T12:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 shutdown_connections 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7ccbcb0 0x5586f7cc9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7cdbe30 0x5586f7cde2a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7cc9110 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 --2- 172.21.15.137:0/2004218332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cc8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f14feffd640 0 client.16905 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 >> 172.21.15.137:0/2004218332 conn(0x5586f7cbd060 msgr2=0x5586f7cc2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 shutdown_connections 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 1 -- 172.21.15.137:0/2004218332 wait complete. 2024-09-12T12:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.752+0000 7f1528a20740 20 client.16905 trim_cache size 0 max 16384 2024-09-12T12:25:20.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.63 MB/sec execute 535 sec latency 12742.278 ms 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 Processor -- start 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 -- start start 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7ce3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7ccbcb0 0x5586f7ce40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15201b0050 con 0x5586f7cc1410 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15201ad8f0 con 0x5586f7cc0af0 2024-09-12T12:25:20.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.766+0000 7f1528a20740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15201ab760 con 0x5586f7ccbcb0 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7ce3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7ce3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34100/0 (socket says 172.21.15.137:34100) 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 -- 172.21.15.137:0/4174705750 learned_addr learned my addr 172.21.15.137:0/4174705750 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151f7fe640 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151ffff640 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7ccbcb0 0x5586f7ce40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 -- 172.21.15.137:0/4174705750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7ccbcb0 msgr2=0x5586f7ce40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7ccbcb0 0x5586f7ce40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 -- 172.21.15.137:0/4174705750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 msgr2=0x5586f7cc9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 -- 172.21.15.137:0/4174705750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f7cc0920 con 0x5586f7cc1410 2024-09-12T12:25:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151f7fe640 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f151effd640 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7ce3bb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1514000c00 tx=0x7f151401a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9f2457339cce13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f15057fa640 1 -- 172.21.15.137:0/4174705750 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140051b0 con 0x5586f7cc1410 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f15057fa640 1 -- 172.21.15.137:0/4174705750 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1514004b60 con 0x5586f7cc1410 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.767+0000 7f15057fa640 1 -- 172.21.15.137:0/4174705750 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151401ae00 con 0x5586f7cc1410 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 -- 172.21.15.137:0/4174705750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 msgr2=0x5586f7ce3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7ce3bb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1514000c00 tx=0x7f151401a650 comp rx=0 tx=0).stop 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 -- 172.21.15.137:0/4174705750 shutdown_connections 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7ccbcb0 0x5586f7ce40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7cc1410 0x5586f7ce3bb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 --2- 172.21.15.137:0/4174705750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc0af0 0x5586f7cc9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 -- 172.21.15.137:0/4174705750 >> 172.21.15.137:0/4174705750 conn(0x5586f7cbd060 msgr2=0x5586f7ccac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 -- 172.21.15.137:0/4174705750 shutdown_connections 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 -- 172.21.15.137:0/4174705750 wait complete. 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 -- start start 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.768+0000 7f1528a20740 1 Processor -- start 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cd3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc1410 0x5586f7cd4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cd0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f15201abdb0 con 0x5586f7ccbcb0 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15201ad220 con 0x5586f7cc1410 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15201ac480 con 0x5586f7cc0af0 2024-09-12T12:25:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 10 client.0 _ll_register_callbacks cb 0x7f151400b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cd0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cd0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34106/0 (socket says 172.21.15.137:34106) 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 -- 172.21.15.137:0/2166747831 learned_addr learned my addr 172.21.15.137:0/2166747831 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151effd640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc1410 0x5586f7cd4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cd3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 -- 172.21.15.137:0/2166747831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 msgr2=0x5586f7cd3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cd3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 -- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc1410 msgr2=0x5586f7cd4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc1410 0x5586f7cd4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 -- 172.21.15.137:0/2166747831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f7cbe1f0 con 0x5586f7ccbcb0 2024-09-12T12:25:20.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151effd640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc1410 0x5586f7cd4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cd3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f151ffff640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cd0ce0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f15180107a0 tx=0x7f151800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b83d9a24c9f032 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f14feffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f14feffd640 1 -- 172.21.15.137:0/2166747831 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518021040 con 0x5586f7ccbcb0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f14feffd640 1 -- 172.21.15.137:0/2166747831 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f1518005050 con 0x5586f7ccbcb0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.769+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1514004b60 con 0x5586f7ccbcb0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f14feffd640 1 -- 172.21.15.137:0/2166747831 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1518004e10 con 0x5586f7ccbcb0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 10 client.16911 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586f7d47270 con 0x5586f7ccbcb0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f14feffd640 1 -- 172.21.15.137:0/2166747831 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1514004b60 con 0x5586f7ccbcb0 2024-09-12T12:25:20.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f14feffd640 1 -- 172.21.15.137:0/2166747831 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f1518035070 con 0x5586f7ccbcb0 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 10 client.16911 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 20 client.16911 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7d47800 0x5586f7d49c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 4 client.16911 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586f7c2cd70 con 0x5586f7d47800 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.770+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7d47800 0x5586f7d49c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.771+0000 7f151f7fe640 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7d47800 0x5586f7d49c20 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e53be04deafcb5 server_cookie=e6a8d9b31439457a in_seq=0 out_seq=0 2024-09-12T12:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.771+0000 7f14feffd640 10 client.16911 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f14feffd640 1 -- 172.21.15.137:0/2166747831 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5586f7c2cd70 con 0x5586f7d47800 2024-09-12T12:25:20.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f14feffd640 10 client.16911 handle_command_reply: tid=0 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 client.16911 shutdown 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 msgr2=0x5586f7cd0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cd0ce0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f15180107a0 tx=0x7f151800eeb0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 shutdown_connections 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586f7d47800 0x5586f7d49c20 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586f7ccbcb0 0x5586f7cd0ce0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f14feffd640 0 client.16911 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586f7cc1410 0x5586f7cd4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 --2- 172.21.15.137:0/2166747831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586f7cc0af0 0x5586f7cd3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 >> 172.21.15.137:0/2166747831 conn(0x5586f7cbd060 msgr2=0x5586f7cbb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 shutdown_connections 2024-09-12T12:25:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 1 -- 172.21.15.137:0/2166747831 wait complete. 2024-09-12T12:25:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.802+0000 7f1528a20740 20 client.16911 trim_cache size 0 max 16384 2024-09-12T12:25:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:25:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:25:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:25:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:25:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15080a2da0 msgr2=0x7f15080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15080a2da0 0x7f15080a5260 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1518013b30 tx=0x7f151800bd10 comp rx=0 tx=0).stop 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 msgr2=0x7f15201983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f15201983a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1510006010 tx=0x7f15100042f0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 shutdown_connections 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15080a2da0 0x7f15080a5260 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f152010c0f0 0x7f152019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1520103640 0x7f15201988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 --2- 172.21.15.137:0/2621253105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520102d20 0x7f15201983a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 >> 172.21.15.137:0/2621253105 conn(0x7f15200fc6b0 msgr2=0x7f15200fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 shutdown_connections 2024-09-12T12:25:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:20.803+0000 7f1525ba6640 1 -- 172.21.15.137:0/2621253105 wait complete. 2024-09-12T12:25:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:20 smithi171 ceph-mon[35098]: pgmap v382: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 4.4 MiB/s wr, 210 op/s 2024-09-12T12:25:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:20 smithi137 ceph-mon[30950]: pgmap v382: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 4.4 MiB/s wr, 210 op/s 2024-09-12T12:25:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:20 smithi189 ceph-mon[35475]: pgmap v382: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 4.4 MiB/s wr, 210 op/s 2024-09-12T12:25:21.760 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.61 MB/sec execute 536 sec latency 13742.318 ms 2024-09-12T12:25:22.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.60 MB/sec execute 537 sec latency 14742.410 ms 2024-09-12T12:25:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:22 smithi171 ceph-mon[35098]: pgmap v383: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.0 MiB/s wr, 72 op/s 2024-09-12T12:25:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:22 smithi137 ceph-mon[30950]: pgmap v383: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.0 MiB/s wr, 72 op/s 2024-09-12T12:25:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:22 smithi189 ceph-mon[35475]: pgmap v383: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.0 MiB/s wr, 72 op/s 2024-09-12T12:25:23.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.58 MB/sec execute 538 sec latency 15742.512 ms 2024-09-12T12:25:24.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.57 MB/sec execute 539 sec latency 16742.614 ms 2024-09-12T12:25:24.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:24 smithi171 ceph-mon[35098]: pgmap v384: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.0 MiB/s wr, 72 op/s 2024-09-12T12:25:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:24 smithi137 ceph-mon[30950]: pgmap v384: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.0 MiB/s wr, 72 op/s 2024-09-12T12:25:25.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:24 smithi189 ceph-mon[35475]: pgmap v384: 97 pgs: 97 active+clean; 277 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 3.0 MiB/s wr, 72 op/s 2024-09-12T12:25:25.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.56 MB/sec execute 540 sec latency 17742.738 ms 2024-09-12T12:25:26.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.54 MB/sec execute 541 sec latency 18742.837 ms 2024-09-12T12:25:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:26 smithi171 ceph-mon[35098]: pgmap v385: 97 pgs: 97 active+clean; 273 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 3.7 MiB/s wr, 110 op/s 2024-09-12T12:25:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:26 smithi137 ceph-mon[30950]: pgmap v385: 97 pgs: 97 active+clean; 273 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 3.7 MiB/s wr, 110 op/s 2024-09-12T12:25:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:26 smithi189 ceph-mon[35475]: pgmap v385: 97 pgs: 97 active+clean; 273 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 3.7 MiB/s wr, 110 op/s 2024-09-12T12:25:27.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.53 MB/sec execute 542 sec latency 19742.945 ms 2024-09-12T12:25:27.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:25:27.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:27.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:27.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:27.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:27.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:25:27.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:25:27.973 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:27 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:25:28.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:25:28.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:25:28.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:27 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:25:28.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:27 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:25:28.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.51 MB/sec execute 543 sec latency 20743.022 ms 2024-09-12T12:25:29.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:29 smithi171 ceph-mon[35098]: pgmap v386: 97 pgs: 97 active+clean; 273 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.5 MiB/s wr, 71 op/s 2024-09-12T12:25:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:29 smithi137 ceph-mon[30950]: pgmap v386: 97 pgs: 97 active+clean; 273 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.5 MiB/s wr, 71 op/s 2024-09-12T12:25:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:29 smithi189 ceph-mon[35475]: pgmap v386: 97 pgs: 97 active+clean; 273 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 2.5 MiB/s wr, 71 op/s 2024-09-12T12:25:29.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.50 MB/sec execute 544 sec latency 21743.129 ms 2024-09-12T12:25:30.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.49 MB/sec execute 545 sec latency 22743.230 ms 2024-09-12T12:25:31.393 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:31 smithi189 ceph-mon[35475]: pgmap v387: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-09-12T12:25:31.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:31 smithi171 ceph-mon[35098]: pgmap v387: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-09-12T12:25:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:31 smithi137 ceph-mon[30950]: pgmap v387: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 3.2 MiB/s wr, 100 op/s 2024-09-12T12:25:31.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.47 MB/sec execute 546 sec latency 23743.289 ms 2024-09-12T12:25:32.761 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.46 MB/sec execute 547 sec latency 24743.383 ms 2024-09-12T12:25:33.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:33 smithi171 ceph-mon[35098]: pgmap v388: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-12T12:25:33.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:33 smithi137 ceph-mon[30950]: pgmap v388: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-12T12:25:33.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:33 smithi189 ceph-mon[35475]: pgmap v388: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-12T12:25:33.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.45 MB/sec execute 548 sec latency 25743.432 ms 2024-09-12T12:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 -- 172.21.15.137:0/2127349156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c1069d0 msgr2=0x7f264c106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 --2- 172.21.15.137:0/2127349156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c1069d0 0x7f264c106db0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2634009930 tx=0x7f263402a420 comp rx=0 tx=0).stop 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 -- 172.21.15.137:0/2127349156 shutdown_connections 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 --2- 172.21.15.137:0/2127349156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f264c101360 0x7f264c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 --2- 172.21.15.137:0/2127349156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264c1009c0 0x7f264c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 --2- 172.21.15.137:0/2127349156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c1069d0 0x7f264c106db0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 -- 172.21.15.137:0/2127349156 >> 172.21.15.137:0/2127349156 conn(0x7f264c0fc700 msgr2=0x7f264c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 -- 172.21.15.137:0/2127349156 shutdown_connections 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 -- 172.21.15.137:0/2127349156 wait complete. 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 Processor -- start 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.900+0000 7f2652c8d640 1 -- start start 2024-09-12T12:25:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2652c8d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f264c1009c0 0x7f264c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2652c8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c101360 0x7f264c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2652c8d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264c1069d0 0x7f264c1a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2652c8d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f264c110e10 con 0x7f264c101360 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2652c8d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f264c110c90 con 0x7f264c1009c0 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2652c8d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f264c110f90 con 0x7f264c1069d0 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c101360 0x7f264c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c101360 0x7f264c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35814/0 (socket says 172.21.15.137:35814) 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 -- 172.21.15.137:0/4143572626 learned_addr learned my addr 172.21.15.137:0/4143572626 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2650a29640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f264c1009c0 0x7f264c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f265122a640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264c1069d0 0x7f264c1a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 -- 172.21.15.137:0/4143572626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264c1069d0 msgr2=0x7f264c1a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264c1069d0 0x7f264c1a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 -- 172.21.15.137:0/4143572626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f264c1009c0 msgr2=0x7f264c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f264c1009c0 0x7f264c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 -- 172.21.15.137:0/4143572626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c1a47a0 con 0x7f264c101360 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2650a29640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f264c1009c0 0x7f264c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f265122a640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264c1069d0 0x7f264c1a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f264bfff640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c101360 0x7f264c199720 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e9f0 tx=0x7f263c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2649ffb640 1 -- 172.21.15.137:0/4143572626 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c00cd80 con 0x7f264c101360 2024-09-12T12:25:33.906 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.901+0000 7f2649ffb640 1 -- 172.21.15.137:0/4143572626 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263c005500 con 0x7f264c101360 2024-09-12T12:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.902+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264c1a4970 con 0x7f264c101360 2024-09-12T12:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.902+0000 7f2649ffb640 1 -- 172.21.15.137:0/4143572626 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c010690 con 0x7f264c101360 2024-09-12T12:25:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.902+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264c1a4e90 con 0x7f264c101360 2024-09-12T12:25:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2649ffb640 1 -- 172.21.15.137:0/4143572626 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f263c0046c0 con 0x7f264c101360 2024-09-12T12:25:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 Processor -- start 2024-09-12T12:25:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 -- start start 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b2002ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b20100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589b1f672f0 con 0x5589b2003410 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5589b1f0f270 con 0x5589b2002af0 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2655a28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589b1f632b0 con 0x5589b200dcb0 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2649ffb640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26380a2e20 0x7f26380a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2650a29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2650a29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35816/0 (socket says 172.21.15.137:35816) 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f2650a29640 1 -- 172.21.15.137:0/3492798025 learned_addr learned my addr 172.21.15.137:0/3492798025 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.905+0000 7f265122a640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26380a2e20 0x7f26380a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f264bfff640 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b20100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2649ffb640 1 -- 172.21.15.137:0/4143572626 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f263c0cf920 con 0x7f264c101360 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f265122a640 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b2002ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2650a29640 1 -- 172.21.15.137:0/3492798025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 msgr2=0x5589b20100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2650a29640 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b20100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2650a29640 1 -- 172.21.15.137:0/3492798025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 msgr2=0x5589b2002ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2650a29640 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b2002ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2650a29640 1 -- 172.21.15.137:0/3492798025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b1f0e340 con 0x5589b2003410 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f264bfff640 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b20100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f265122a640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26380a2e20 0x7f26380a52e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2644005c40 tx=0x7f2644009cd0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2650a29640 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200d770 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2634009930 tx=0x7f2634005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fdbd62ab42cc23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2642ffd640 1 -- 172.21.15.137:0/3492798025 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263403f070 con 0x5589b2003410 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2642ffd640 1 -- 172.21.15.137:0/3492798025 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2634004030 con 0x5589b2003410 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.906+0000 7f2642ffd640 1 -- 172.21.15.137:0/3492798025 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634035070 con 0x5589b2003410 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 -- 172.21.15.137:0/3492798025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 msgr2=0x5589b200d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200d770 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2634009930 tx=0x7f2634005e60 comp rx=0 tx=0).stop 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 -- 172.21.15.137:0/3492798025 shutdown_connections 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b20100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200d770 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 --2- 172.21.15.137:0/3492798025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b2002ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 -- 172.21.15.137:0/3492798025 >> 172.21.15.137:0/3492798025 conn(0x5589b1fff060 msgr2=0x5589b1fff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 -- 172.21.15.137:0/3492798025 shutdown_connections 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 -- 172.21.15.137:0/3492798025 wait complete. 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 -- start start 2024-09-12T12:25:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.907+0000 7f2655a28740 1 Processor -- start 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2655a28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b200ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2655a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2655a28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b200b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2655a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589b2010880 con 0x5589b2003410 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2655a28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5589b1ef0290 con 0x5589b2002af0 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2655a28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5589b2010a00 con 0x5589b200dcb0 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2655a28740 10 client.0 _ll_register_callbacks cb 0x5589b1f21140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35828/0 (socket says 172.21.15.137:35828) 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 -- 172.21.15.137:0/1140507261 learned_addr learned my addr 172.21.15.137:0/1140507261 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2650a29640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b200ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 -- 172.21.15.137:0/1140507261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 msgr2=0x5589b200b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f265122a640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b200b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b200b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 -- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 msgr2=0x5589b200ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b200ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f264bfff640 1 -- 172.21.15.137:0/1140507261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b1c7a210 con 0x5589b2003410 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f265122a640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b200b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.908+0000 7f2650a29640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b200ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f264bfff640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200b070 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f263c00cfd0 tx=0x7f263c00a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1c480899458c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f262bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f262bfff640 1 -- 172.21.15.137:0/1140507261 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c09be60 con 0x5589b2003410 2024-09-12T12:25:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f262bfff640 1 -- 172.21.15.137:0/1140507261 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f263c09c0c0 con 0x5589b2003410 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2634004030 con 0x5589b2003410 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f262bfff640 1 -- 172.21.15.137:0/1140507261 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f263c060030 con 0x5589b2003410 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 10 client.16923 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5589b1d46620 con 0x5589b2003410 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f262bfff640 1 -- 172.21.15.137:0/1140507261 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2634004030 con 0x5589b2003410 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f262bfff640 1 -- 172.21.15.137:0/1140507261 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f263c0a4640 con 0x5589b2003410 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 10 client.16923 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 20 client.16923 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b201dbe0 0x5589b201ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 4 client.16923 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.909+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b1f74a20 con 0x5589b201dbe0 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.910+0000 7f2650a29640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b201dbe0 0x5589b201ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.910+0000 7f262bfff640 10 client.16923 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.910+0000 7f2650a29640 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b201dbe0 0x5589b201ffd0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1723f80d1854b425 server_cookie=7895a38dedf4b4fb in_seq=0 out_seq=0 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f262bfff640 1 -- 172.21.15.137:0/1140507261 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589b1f74a20 con 0x5589b201dbe0 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f262bfff640 10 client.16923 handle_command_reply: tid=0 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 client.16923 shutdown 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 msgr2=0x5589b200b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200b070 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f263c00cfd0 tx=0x7f263c00a440 comp rx=0 tx=0).stop 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 shutdown_connections 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b200dcb0 0x5589b200b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b201dbe0 0x5589b201ffd0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b200b070 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 --2- 172.21.15.137:0/1140507261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b200ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f262bfff640 0 client.16923 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 >> 172.21.15.137:0/1140507261 conn(0x5589b1fff060 msgr2=0x5589b2004dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 shutdown_connections 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 1 -- 172.21.15.137:0/1140507261 wait complete. 2024-09-12T12:25:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.911+0000 7f2655a28740 20 client.16923 trim_cache size 0 max 16384 2024-09-12T12:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 Processor -- start 2024-09-12T12:25:33.932 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 -- start start 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b20055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2003410 0x5589b200b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b200dcb0 0x5589b20258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f264c1afee0 con 0x5589b200dcb0 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f264c1ad780 con 0x5589b2002af0 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2655a28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f264c1ab5f0 con 0x5589b2003410 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b200dcb0 0x5589b20258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b200dcb0 0x5589b20258c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35832/0 (socket says 172.21.15.137:35832) 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 -- 172.21.15.137:0/267238455 learned_addr learned my addr 172.21.15.137:0/267238455 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f2650a29640 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b20055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f264bfff640 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2003410 0x5589b200b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 -- 172.21.15.137:0/267238455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2003410 msgr2=0x5589b200b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2003410 0x5589b200b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 -- 172.21.15.137:0/267238455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 msgr2=0x5589b20055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b20055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 -- 172.21.15.137:0/267238455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b1ffee90 con 0x5589b200dcb0 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.926+0000 7f265122a640 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b200dcb0 0x5589b20258c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2644009f10 tx=0x7f2644004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445db83bb85f006c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f26427fc640 1 -- 172.21.15.137:0/267238455 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644025070 con 0x5589b200dcb0 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f26427fc640 1 -- 172.21.15.137:0/267238455 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2644012e50 con 0x5589b200dcb0 2024-09-12T12:25:33.934 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f26427fc640 1 -- 172.21.15.137:0/267238455 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644021c80 con 0x5589b200dcb0 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f2655a28740 1 -- 172.21.15.137:0/267238455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b200dcb0 msgr2=0x5589b20258c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f2655a28740 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b200dcb0 0x5589b20258c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2644009f10 tx=0x7f2644004ab0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f2655a28740 1 -- 172.21.15.137:0/267238455 shutdown_connections 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f2655a28740 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b200dcb0 0x5589b20258c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f2655a28740 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2003410 0x5589b200b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f2655a28740 1 --2- 172.21.15.137:0/267238455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b2002af0 0x5589b20055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.927+0000 7f2655a28740 1 -- 172.21.15.137:0/267238455 >> 172.21.15.137:0/267238455 conn(0x5589b1fff060 msgr2=0x5589b200cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 -- 172.21.15.137:0/267238455 shutdown_connections 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 -- 172.21.15.137:0/267238455 wait complete. 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 -- start start 2024-09-12T12:25:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 Processor -- start 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2002af0 0x5589b2016350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b2011670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b200dcb0 0x5589b2011bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f264c1abc40 con 0x5589b2003410 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f264c1ad0b0 con 0x5589b200dcb0 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f264c1ac310 con 0x5589b2002af0 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2655a28740 10 client.0 _ll_register_callbacks cb 0x7f263c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f264bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b2011670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f264bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b2011670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35846/0 (socket says 172.21.15.137:35846) 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f264bfff640 1 -- 172.21.15.137:0/723969722 learned_addr learned my addr 172.21.15.137:0/723969722 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.928+0000 7f2650a29640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2002af0 0x5589b2016350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f265122a640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b200dcb0 0x5589b2011bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f264bfff640 1 -- 172.21.15.137:0/723969722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2002af0 msgr2=0x5589b2016350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f264bfff640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2002af0 0x5589b2016350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f264bfff640 1 -- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b200dcb0 msgr2=0x5589b2011bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f264bfff640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b200dcb0 0x5589b2011bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f264bfff640 1 -- 172.21.15.137:0/723969722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b20123c0 con 0x5589b2003410 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f265122a640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b200dcb0 0x5589b2011bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f264bfff640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b2011670 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f263c0a45a0 tx=0x7f263c0a4a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102a21673b45af01 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f262bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f262bfff640 1 -- 172.21.15.137:0/723969722 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c098100 con 0x5589b2003410 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f262bfff640 1 -- 172.21.15.137:0/723969722 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f263c097bb0 con 0x5589b2003410 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2644012e50 con 0x5589b2003410 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f262bfff640 1 -- 172.21.15.137:0/723969722 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f263c09b7c0 con 0x5589b2003410 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f2655a28740 10 client.16929 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5589b2088810 con 0x5589b2003410 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f262bfff640 1 -- 172.21.15.137:0/723969722 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2644012e50 con 0x5589b2003410 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.929+0000 7f262bfff640 1 -- 172.21.15.137:0/723969722 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f263c0ab050 con 0x5589b2003410 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2655a28740 10 client.16929 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2655a28740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2655a28740 20 client.16929 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2655a28740 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b2088fc0 0x5589b208b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2655a28740 4 client.16929 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:25:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589b1f6ed70 con 0x5589b2088fc0 2024-09-12T12:25:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2650a29640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b2088fc0 0x5589b208b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f262bfff640 10 client.16929 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.930+0000 7f2650a29640 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b2088fc0 0x5589b208b3e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b76509b5b85e54 server_cookie=a4d02fea2cb54de3 in_seq=0 out_seq=0 2024-09-12T12:25:33.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f262bfff640 1 -- 172.21.15.137:0/723969722 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589b1f6ed70 con 0x5589b2088fc0 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f262bfff640 10 client.16929 handle_command_reply: tid=0 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 client.16929 shutdown 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 msgr2=0x5589b2011670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b2011670 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f263c0a45a0 tx=0x7f263c0a4a00 comp rx=0 tx=0).stop 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 shutdown_connections 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5589b200dcb0 0x5589b2011bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589b2088fc0 0x5589b208b3e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5589b2003410 0x5589b2011670 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 --2- 172.21.15.137:0/723969722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589b2002af0 0x5589b2016350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f262bfff640 0 client.16929 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 >> 172.21.15.137:0/723969722 conn(0x5589b1fff060 msgr2=0x5589b200c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 shutdown_connections 2024-09-12T12:25:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 1 -- 172.21.15.137:0/723969722 wait complete. 2024-09-12T12:25:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2655a28740 20 client.16929 trim_cache size 0 max 16384 2024-09-12T12:25:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:25:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:25:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:25:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26380a2e20 msgr2=0x7f26380a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26380a2e20 0x7f26380a52e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2644005c40 tx=0x7f2644009cd0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c101360 msgr2=0x7f264c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c101360 0x7f264c199720 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f263c00e9f0 tx=0x7f263c00edf0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 shutdown_connections 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26380a2e20 0x7f26380a52e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f264c1069d0 0x7f264c1a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f264c101360 0x7f264c199720 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 --2- 172.21.15.137:0/4143572626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f264c1009c0 0x7f264c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 >> 172.21.15.137:0/4143572626 conn(0x7f264c0fc700 msgr2=0x7f264c0fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 shutdown_connections 2024-09-12T12:25:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:33.961+0000 7f2652c8d640 1 -- 172.21.15.137:0/4143572626 wait complete. 2024-09-12T12:25:34.192 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:25:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:25:34.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.43 MB/sec execute 549 sec latency 26743.463 ms 2024-09-12T12:25:34.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 -- 172.21.15.137:0/2523707226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8106f10 msgr2=0x7f26a810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 --2- 172.21.15.137:0/2523707226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8106f10 0x7f26a810ea90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f26a0009ca0 tx=0x7f26a002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 -- 172.21.15.137:0/2523707226 shutdown_connections 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 --2- 172.21.15.137:0/2523707226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8106f10 0x7f26a810ea90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 --2- 172.21.15.137:0/2523707226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8102790 0x7f26a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 --2- 172.21.15.137:0/2523707226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26a8101de0 0x7f26a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 -- 172.21.15.137:0/2523707226 >> 172.21.15.137:0/2523707226 conn(0x7f26a80fd8c0 msgr2=0x7f26a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 -- 172.21.15.137:0/2523707226 shutdown_connections 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.945+0000 7f26af02f640 1 -- 172.21.15.137:0/2523707226 wait complete. 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 Processor -- start 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 -- start start 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26a8101de0 0x7f26a81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:34.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8102790 0x7f26a8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8106f10 0x7f26a819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8111f70 con 0x7f26a8102790 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f26a8111df0 con 0x7f26a8101de0 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26af02f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26a81120f0 con 0x7f26a8106f10 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.946+0000 7f26ad5cc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8106f10 0x7f26a819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8102790 0x7f26a8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26ad5cc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8106f10 0x7f26a819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33232/0 (socket says 172.21.15.137:33232) 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26ad5cc640 1 -- 172.21.15.137:0/2922397910 learned_addr learned my addr 172.21.15.137:0/2922397910 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26acdcb640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26a8101de0 0x7f26a81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a7fff640 1 -- 172.21.15.137:0/2922397910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8106f10 msgr2=0x7f26a819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a7fff640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8106f10 0x7f26a819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a7fff640 1 -- 172.21.15.137:0/2922397910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26a8101de0 msgr2=0x7f26a81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:34.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a7fff640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26a8101de0 0x7f26a81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a7fff640 1 -- 172.21.15.137:0/2922397910 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a819d3a0 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26ad5cc640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8106f10 0x7f26a819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a7fff640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8102790 0x7f26a8198930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f269c00d720 tx=0x7f269c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a5ffb640 1 -- 172.21.15.137:0/2922397910 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c017070 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a5ffb640 1 -- 172.21.15.137:0/2922397910 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269c00de30 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a819d690 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.947+0000 7f26a5ffb640 1 -- 172.21.15.137:0/2922397910 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c0047a0 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.948+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a81a4f70 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.949+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a81021c0 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.950+0000 7f26a5ffb640 1 -- 172.21.15.137:0/2922397910 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f269c0099d0 con 0x7f26a8102790 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.951+0000 7f26a5ffb640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26900a2cf0 0x7f26900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.951+0000 7f26acdcb640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26900a2cf0 0x7f26900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:34.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.951+0000 7f26a5ffb640 1 -- 172.21.15.137:0/2922397910 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f269c0cef90 con 0x7f26a8102790 2024-09-12T12:25:34.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.951+0000 7f26acdcb640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26900a2cf0 0x7f26900a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26980058c0 tx=0x7f269800a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:34.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:34.955+0000 7f26a5ffb640 1 -- 172.21.15.137:0/2922397910 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f269c020020 con 0x7f26a8102790 2024-09-12T12:25:35.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.155+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26a810a120 con 0x7f26a8102790 2024-09-12T12:25:35.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.156+0000 7f26a5ffb640 1 -- 172.21.15.137:0/2922397910 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11934 (secure 0 0 0) 0x7f269c09bbd0 con 0x7f26a8102790 2024-09-12T12:25:35.155 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:25:35.156 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":39,"btime":"2024-09-12T12:25:07:764471+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:25:06.983730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:25:35.156 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 39 2024-09-12T12:25:35.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26900a2cf0 msgr2=0x7f26900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26900a2cf0 0x7f26900a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f26980058c0 tx=0x7f269800a2d0 comp rx=0 tx=0).stop 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8102790 msgr2=0x7f26a8198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8102790 0x7f26a8198930 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f269c00d720 tx=0x7f269c00dbe0 comp rx=0 tx=0).stop 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 shutdown_connections 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f26900a2cf0 0x7f26900a51b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26a8106f10 0x7f26a819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26a8102790 0x7f26a8198930 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 --2- 172.21.15.137:0/2922397910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26a8101de0 0x7f26a81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 >> 172.21.15.137:0/2922397910 conn(0x7f26a80fd8c0 msgr2=0x7f26a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 shutdown_connections 2024-09-12T12:25:35.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:35.158+0000 7f26af02f640 1 -- 172.21.15.137:0/2922397910 wait complete. 2024-09-12T12:25:35.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:35 smithi171 ceph-mon[35098]: pgmap v389: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-12T12:25:35.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:35 smithi137 ceph-mon[30950]: pgmap v389: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-12T12:25:35.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:35 smithi189 ceph-mon[35475]: pgmap v389: 97 pgs: 97 active+clean; 272 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-12T12:25:35.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.42 MB/sec execute 550 sec latency 27743.497 ms 2024-09-12T12:25:35.958 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 7c925699 --release --await' 2024-09-12T12:25:35.958 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 7c925699 --release --await 2024-09-12T12:25:36.205 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:25:36.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:36 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2922397910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:25:36.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:36 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2922397910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:25:36.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:36 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2922397910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:25:36.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 960781 7.41 MB/sec execute 551 sec latency 28743.579 ms 2024-09-12T12:25:37.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.930+0000 7f524bafa640 1 -- 172.21.15.137:0/2511298162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244106a80 msgr2=0x7f5244106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.930+0000 7f524bafa640 1 --2- 172.21.15.137:0/2511298162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244106a80 0x7f5244106e60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5234009930 tx=0x7f523402a420 comp rx=0 tx=0).stop 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.930+0000 7f524bafa640 1 -- 172.21.15.137:0/2511298162 shutdown_connections 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.930+0000 7f524bafa640 1 --2- 172.21.15.137:0/2511298162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5244101420 0x7f524410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.930+0000 7f524bafa640 1 --2- 172.21.15.137:0/2511298162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244100a80 0x7f5244100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.930+0000 7f524bafa640 1 --2- 172.21.15.137:0/2511298162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244106a80 0x7f5244106e60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.930+0000 7f524bafa640 1 -- 172.21.15.137:0/2511298162 >> 172.21.15.137:0/2511298162 conn(0x7f52440fc7a0 msgr2=0x7f52440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.931+0000 7f524bafa640 1 -- 172.21.15.137:0/2511298162 shutdown_connections 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.931+0000 7f524bafa640 1 -- 172.21.15.137:0/2511298162 wait complete. 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 Processor -- start 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 -- start start 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5244100a80 0x7f5244070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244101420 0x7f5244071230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244106a80 0x7f5244071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f524410fe40 con 0x7f5244101420 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f524410fcc0 con 0x7f5244100a80 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524bafa640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f524410ffc0 con 0x7f5244106a80 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524a097640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244106a80 0x7f5244071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524a097640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244106a80 0x7f5244071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33246/0 (socket says 172.21.15.137:33246) 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f524a097640 1 -- 172.21.15.137:0/1725252850 learned_addr learned my addr 172.21.15.137:0/1725252850 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f5249896640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5244100a80 0x7f5244070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.932+0000 7f5249095640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244101420 0x7f5244071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.933+0000 7f5249095640 1 -- 172.21.15.137:0/1725252850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244106a80 msgr2=0x7f5244071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.933+0000 7f5249095640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244106a80 0x7f5244071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.933+0000 7f5249095640 1 -- 172.21.15.137:0/1725252850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5244100a80 msgr2=0x7f5244070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.933+0000 7f5249095640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5244100a80 0x7f5244070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.933+0000 7f5249095640 1 -- 172.21.15.137:0/1725252850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524406fe00 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.933+0000 7f524a097640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244106a80 0x7f5244071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.933+0000 7f5249095640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244101420 0x7f5244071230 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5238009fd0 tx=0x7f523800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.934+0000 7f5242ffd640 1 -- 172.21.15.137:0/1725252850 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238004b20 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.934+0000 7f5242ffd640 1 -- 172.21.15.137:0/1725252850 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5238004cc0 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.934+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52440700f0 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.934+0000 7f5242ffd640 1 -- 172.21.15.137:0/1725252850 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52380107b0 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.934+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5244070630 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.936+0000 7f5242ffd640 1 -- 172.21.15.137:0/1725252850 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5238005020 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 Processor -- start 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f5242ffd640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f522c0a2d70 0x7f522c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524a097640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f522c0a2d70 0x7f522c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f5242ffd640 1 -- 172.21.15.137:0/1725252850 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f52380cfb90 con 0x7f5244101420 2024-09-12T12:25:37.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 -- start start 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc755440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc762630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5559dc6ba620 con 0x5559dc755980 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5559dc661e50 con 0x5559dc760240 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f524ded3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5559dc6b65e0 con 0x5559dc755060 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.937+0000 7f5249896640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc755440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f522c0a2d70 0x7f522c0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f523c005fd0 tx=0x7f523c00b420 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f5249095640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc762630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f5249896640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc755440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33256/0 (socket says 172.21.15.137:33256) 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f5249896640 1 -- 172.21.15.137:0/631512598 learned_addr learned my addr 172.21.15.137:0/631512598 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:37.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 -- 172.21.15.137:0/631512598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 msgr2=0x5559dc755440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc755440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 -- 172.21.15.137:0/631512598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 msgr2=0x5559dc75fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 -- 172.21.15.137:0/631512598 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559dc652d40 con 0x5559dc760240 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f5249095640 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f524a097640 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc762630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f523c00bfd0 tx=0x7f523c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ce680100e2ed5c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f522bfff640 1 -- 172.21.15.137:0/631512598 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c004220 con 0x5559dc760240 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f522bfff640 1 -- 172.21.15.137:0/631512598 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523c0043c0 con 0x5559dc760240 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.938+0000 7f522bfff640 1 -- 172.21.15.137:0/631512598 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c004a00 con 0x5559dc760240 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 -- 172.21.15.137:0/631512598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 msgr2=0x5559dc762630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc762630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f523c00bfd0 tx=0x7f523c013000 comp rx=0 tx=0).stop 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 -- 172.21.15.137:0/631512598 shutdown_connections 2024-09-12T12:25:37.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc762630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 --2- 172.21.15.137:0/631512598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc755440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 -- 172.21.15.137:0/631512598 >> 172.21.15.137:0/631512598 conn(0x5559dc7515f0 msgr2=0x5559dc7519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 -- 172.21.15.137:0/631512598 shutdown_connections 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 -- 172.21.15.137:0/631512598 wait complete. 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.939+0000 7f524ded3740 1 -- start start 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 1 Processor -- start 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc75d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc75dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5559dc762e10 con 0x5559dc755980 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5559dc642f10 con 0x5559dc760240 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5559dc762f90 con 0x5559dc755060 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524ded3740 10 client.0 _ll_register_callbacks cb 0x5559dc666db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35898/0 (socket says 172.21.15.137:35898) 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 -- 172.21.15.137:0/1081339604 learned_addr learned my addr 172.21.15.137:0/1081339604 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f524a097640 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc75dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249896640 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc75d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 -- 172.21.15.137:0/1081339604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 msgr2=0x5559dc75d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc75d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 -- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 msgr2=0x5559dc75dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc75dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.940+0000 7f5249095640 1 -- 172.21.15.137:0/1081339604 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559dc3c9580 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f5249095640 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75d680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f523809c000 tx=0x7f523809c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151d6a70a7f8ad54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f52297fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f52297fa640 1 -- 172.21.15.137:0/1081339604 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5238005590 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f52297fa640 1 -- 172.21.15.137:0/1081339604 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f52380a00d0 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f523c0043c0 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f52297fa640 1 -- 172.21.15.137:0/1081339604 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f52380986e0 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 10 client.16947 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5559dc493960 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f52297fa640 1 -- 172.21.15.137:0/1081339604 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523c0043c0 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f52297fa640 1 -- 172.21.15.137:0/1081339604 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f52380a4850 con 0x5559dc755980 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 10 client.16947 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:37.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 10 client.16947 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 20 client.16947 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc772200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 4 client.16947 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.941+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559dc6c7b90 con 0x5559dc76fdc0 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.942+0000 7f5249896640 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc772200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.942+0000 7f52297fa640 10 client.16947 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.942+0000 7f5249896640 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc772200 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=186616a04b170693 server_cookie=54572ce27d2c3e15 in_seq=0 out_seq=0 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f52297fa640 1 -- 172.21.15.137:0/1081339604 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559dc6c7b90 con 0x5559dc76fdc0 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f52297fa640 10 client.16947 handle_command_reply: tid=0 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 client.16947 shutdown 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 msgr2=0x5559dc75d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75d680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f523809c000 tx=0x7f523809c8a0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 shutdown_connections 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc760240 0x5559dc75dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc772200 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f52297fa640 0 client.16947 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc75d680 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 --2- 172.21.15.137:0/1081339604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc755060 0x5559dc75d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 >> 172.21.15.137:0/1081339604 conn(0x5559dc7515f0 msgr2=0x5559dc757320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 shutdown_connections 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 1 -- 172.21.15.137:0/1081339604 wait complete. 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.943+0000 7f524ded3740 20 client.16947 trim_cache size 0 max 16384 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.960+0000 7f524ded3740 1 Processor -- start 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.960+0000 7f524ded3740 1 -- start start 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.960+0000 7f524ded3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755060 0x5559dc777ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f524ded3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc778010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f524ded3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc75a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f524ded3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52441babd0 con 0x5559dc755980 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f524ded3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b8470 con 0x5559dc755060 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f524ded3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b62e0 con 0x5559dc760240 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc778010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc778010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35906/0 (socket says 172.21.15.137:35906) 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 -- 172.21.15.137:0/3325571302 learned_addr learned my addr 172.21.15.137:0/3325571302 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f524a097640 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc75a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249896640 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755060 0x5559dc777ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 -- 172.21.15.137:0/3325571302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 msgr2=0x5559dc75a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc75a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 -- 172.21.15.137:0/3325571302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755060 msgr2=0x5559dc777ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755060 0x5559dc777ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 -- 172.21.15.137:0/3325571302 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559dc752780 con 0x5559dc755980 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f5249095640 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc778010 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f52380984d0 tx=0x7f52380a05a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae4f3018e8bc603 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f522b7fe640 1 -- 172.21.15.137:0/3325571302 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523809b7e0 con 0x5559dc755980 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f522b7fe640 1 -- 172.21.15.137:0/3325571302 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5238005590 con 0x5559dc755980 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.961+0000 7f522b7fe640 1 -- 172.21.15.137:0/3325571302 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523809c2d0 con 0x5559dc755980 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 -- 172.21.15.137:0/3325571302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 msgr2=0x5559dc778010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc778010 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f52380984d0 tx=0x7f52380a05a0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 -- 172.21.15.137:0/3325571302 shutdown_connections 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc75a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755980 0x5559dc778010 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 --2- 172.21.15.137:0/3325571302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755060 0x5559dc777ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 -- 172.21.15.137:0/3325571302 >> 172.21.15.137:0/3325571302 conn(0x5559dc7515f0 msgr2=0x5559dc75ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 -- 172.21.15.137:0/3325571302 shutdown_connections 2024-09-12T12:25:37.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 -- 172.21.15.137:0/3325571302 wait complete. 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 -- start start 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.962+0000 7f524ded3740 1 Processor -- start 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755060 0x5559dc764190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755980 0x5559dc7646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc764c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b6930 con 0x5559dc755060 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b7da0 con 0x5559dc755980 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f52441b7000 con 0x5559dc760240 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 10 client.0 _ll_register_callbacks cb 0x7f523809ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755060 0x5559dc764190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755060 0x5559dc764190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35910/0 (socket says 172.21.15.137:35910) 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 -- 172.21.15.137:0/1184963526 learned_addr learned my addr 172.21.15.137:0/1184963526 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:37.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249095640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755980 0x5559dc7646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524a097640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc764c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 -- 172.21.15.137:0/1184963526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 msgr2=0x5559dc764c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc764c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 -- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755980 msgr2=0x5559dc7646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755980 0x5559dc7646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 -- 172.21.15.137:0/1184963526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559dc7d9cf0 con 0x5559dc755060 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249095640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755980 0x5559dc7646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f5249896640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755060 0x5559dc764190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f523402a940 tx=0x7f5234031b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28246b373d64cf5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f52297fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f52297fa640 1 -- 172.21.15.137:0/1184963526 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234035040 con 0x5559dc755060 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f52297fa640 1 -- 172.21.15.137:0/1184963526 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f5234005040 con 0x5559dc755060 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5238005590 con 0x5559dc755060 2024-09-12T12:25:37.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f52297fa640 1 -- 172.21.15.137:0/1184963526 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5234005240 con 0x5559dc755060 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 10 client.16953 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.963+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5559dc7db070 con 0x5559dc755060 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f52297fa640 1 -- 172.21.15.137:0/1184963526 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5238005590 con 0x5559dc755060 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f52297fa640 1 -- 172.21.15.137:0/1184963526 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7f5234058020 con 0x5559dc755060 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f524ded3740 10 client.16953 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f524ded3740 10 client.16953 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f524ded3740 20 client.16953 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f524ded3740 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc7dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f524ded3740 4 client.16953 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7c925699", "release": true, "await": true}] 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7c925699", "release": true, "await": true}) -- 0x5559dc6c20a0 con 0x5559dc76fdc0 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.964+0000 7f5249095640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc7dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.965+0000 7f5249095640 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc7dd880 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f79480d08efd978 server_cookie=17207be1cec8ceb4 in_seq=0 out_seq=0 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.965+0000 7f52297fa640 10 client.16953 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:37.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f52297fa640 1 -- 172.21.15.137:0/1184963526 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5559dc6c20a0 con 0x5559dc76fdc0 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f52297fa640 10 client.16953 handle_command_reply: tid=0 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 client.16953 shutdown 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755060 msgr2=0x5559dc764190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755060 0x5559dc764190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f523402a940 tx=0x7f5234031b40 comp rx=0 tx=0).stop 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 shutdown_connections 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559dc760240 0x5559dc764c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5559dc755980 0x5559dc7646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5559dc76fdc0 0x5559dc7dd880 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 --2- 172.21.15.137:0/1184963526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5559dc755060 0x5559dc764190 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f52297fa640 0 client.16953 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 >> 172.21.15.137:0/1184963526 conn(0x5559dc7515f0 msgr2=0x5559dc756ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:37.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 shutdown_connections 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 1 -- 172.21.15.137:0/1184963526 wait complete. 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524ded3740 20 client.16953 trim_cache size 0 max 16384 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 39, 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 48, 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout: "7c925699": { 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 48, 2024-09-12T12:25:37.025 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 147.2, 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:37.026 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f522c0a2d70 msgr2=0x7f522c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524bafa640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f522c0a2d70 0x7f522c0a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f523c005fd0 tx=0x7f523c00b420 comp rx=0 tx=0).stop 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244101420 msgr2=0x7f5244071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.970+0000 7f524bafa640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244101420 0x7f5244071230 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5238009fd0 tx=0x7f523800ef90 comp rx=0 tx=0).stop 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 shutdown_connections 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f522c0a2d70 0x7f522c0a5230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5244106a80 0x7f5244071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5244101420 0x7f5244071230 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 --2- 172.21.15.137:0/1725252850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5244100a80 0x7f5244070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 >> 172.21.15.137:0/1725252850 conn(0x7f52440fc7a0 msgr2=0x7f52440fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 shutdown_connections 2024-09-12T12:25:37.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:25:36.971+0000 7f524bafa640 1 -- 172.21.15.137:0/1725252850 wait complete. 2024-09-12T12:25:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:37 smithi137 ceph-mon[30950]: pgmap v390: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.2 MiB/s wr, 99 op/s 2024-09-12T12:25:37.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:37 smithi171 ceph-mon[35098]: pgmap v390: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.2 MiB/s wr, 99 op/s 2024-09-12T12:25:37.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:37 smithi189 ceph-mon[35475]: pgmap v390: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.2 MiB/s wr, 99 op/s 2024-09-12T12:25:37.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 962911 7.41 MB/sec execute 552 sec latency 28954.092 ms 2024-09-12T12:25:37.894 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7c925699', total seconds elapsed: 30.9 2024-09-12T12:25:38.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 964245 7.41 MB/sec execute 553 sec latency 15.334 ms 2024-09-12T12:25:39.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:39 smithi171 ceph-mon[35098]: pgmap v391: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-09-12T12:25:39.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:39 smithi137 ceph-mon[30950]: pgmap v391: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-09-12T12:25:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:39 smithi189 ceph-mon[35475]: pgmap v391: 97 pgs: 97 active+clean; 271 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-09-12T12:25:39.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 966603 7.42 MB/sec execute 554 sec latency 14.597 ms 2024-09-12T12:25:40.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 967901 7.42 MB/sec execute 555 sec latency 21.971 ms 2024-09-12T12:25:41.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:41 smithi171 ceph-mon[35098]: pgmap v392: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-12T12:25:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:41 smithi137 ceph-mon[30950]: pgmap v392: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-12T12:25:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:41 smithi189 ceph-mon[35475]: pgmap v392: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 137 op/s 2024-09-12T12:25:41.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 970263 7.43 MB/sec execute 556 sec latency 14.610 ms 2024-09-12T12:25:42.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 971839 7.43 MB/sec execute 557 sec latency 23.242 ms 2024-09-12T12:25:43.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:43 smithi171 ceph-mon[35098]: pgmap v393: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-12T12:25:43.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:43 smithi137 ceph-mon[30950]: pgmap v393: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-12T12:25:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:43 smithi189 ceph-mon[35475]: pgmap v393: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-12T12:25:43.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 973879 7.44 MB/sec execute 558 sec latency 15.249 ms 2024-09-12T12:25:44.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 975329 7.44 MB/sec execute 559 sec latency 15.711 ms 2024-09-12T12:25:45.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:45 smithi171 ceph-mon[35098]: pgmap v394: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-12T12:25:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:45 smithi137 ceph-mon[30950]: pgmap v394: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-12T12:25:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:45 smithi189 ceph-mon[35475]: pgmap v394: 97 pgs: 97 active+clean; 275 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-12T12:25:45.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 977398 7.45 MB/sec execute 560 sec latency 16.478 ms 2024-09-12T12:25:46.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 978996 7.44 MB/sec execute 561 sec latency 22.814 ms 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 -- 172.21.15.137:0/225657244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 msgr2=0x7fa820100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 --2- 172.21.15.137:0/225657244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa820100e10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009930 tx=0x7fa80c02a420 comp rx=0 tx=0).stop 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 -- 172.21.15.137:0/225657244 shutdown_connections 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 --2- 172.21.15.137:0/225657244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa820101350 0x7fa82010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 --2- 172.21.15.137:0/225657244 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa820100e10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 --2- 172.21.15.137:0/225657244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201069c0 0x7fa820106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 -- 172.21.15.137:0/225657244 >> 172.21.15.137:0/225657244 conn(0x7fa820077990 msgr2=0x7fa8200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 -- 172.21.15.137:0/225657244 shutdown_connections 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.051+0000 7fa8271fd640 1 -- 172.21.15.137:0/225657244 wait complete. 2024-09-12T12:25:47.054 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 Processor -- start 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 -- start start 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa8201972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa820101350 0x7fa820197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201069c0 0x7fa82019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa820110e00 con 0x7fa8201009b0 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa820110c80 con 0x7fa820101350 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa8271fd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa820110f80 con 0x7fa8201069c0 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa8201972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa8201972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33054/0 (socket says 172.21.15.137:33054) 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 -- 172.21.15.137:0/52895317 learned_addr learned my addr 172.21.15.137:0/52895317 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa81ffff640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa820101350 0x7fa820197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa82579a640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201069c0 0x7fa82019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 -- 172.21.15.137:0/52895317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201069c0 msgr2=0x7fa82019bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201069c0 0x7fa82019bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 -- 172.21.15.137:0/52895317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa820101350 msgr2=0x7fa820197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa820101350 0x7fa820197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa824f99640 1 -- 172.21.15.137:0/52895317 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82019c2e0 con 0x7fa8201009b0 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.052+0000 7fa82579a640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201069c0 0x7fa82019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.053+0000 7fa824f99640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa8201972f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa81400eb00 tx=0x7fa81400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.053+0000 7fa81dffb640 1 -- 172.21.15.137:0/52895317 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81400bc30 con 0x7fa8201009b0 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.053+0000 7fa81dffb640 1 -- 172.21.15.137:0/52895317 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81400bdd0 con 0x7fa8201009b0 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.053+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82019c5d0 con 0x7fa8201009b0 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.053+0000 7fa81dffb640 1 -- 172.21.15.137:0/52895317 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814017410 con 0x7fa8201009b0 2024-09-12T12:25:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.053+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8200698a0 con 0x7fa8201009b0 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.055+0000 7fa81dffb640 1 -- 172.21.15.137:0/52895317 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa814005020 con 0x7fa8201009b0 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 Processor -- start 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 -- start start 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa81dffb640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8100a2da0 0x7fa8100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227accdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acd2f0 0x558227adbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 0x558227ade4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558227a312f0 con 0x558227acc9d0 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5582279d9270 con 0x558227acd2f0 2024-09-12T12:25:47.063 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82a081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558227a2d2b0 con 0x558227adc0c0 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227accdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa824f99640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8100a2da0 0x7fa8100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa81ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acd2f0 0x558227adbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227accdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33056/0 (socket says 172.21.15.137:33056) 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 -- 172.21.15.137:0/3710630432 learned_addr learned my addr 172.21.15.137:0/3710630432 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa81dffb640 1 -- 172.21.15.137:0/52895317 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fa8140d0c60 con 0x7fa8201009b0 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 -- 172.21.15.137:0/3710630432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 msgr2=0x558227ade4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 0x558227ade4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 -- 172.21.15.137:0/3710630432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acd2f0 msgr2=0x558227adbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acd2f0 0x558227adbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa82579a640 1 -- 172.21.15.137:0/3710630432 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582279d8340 con 0x558227acc9d0 2024-09-12T12:25:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.056+0000 7fa81ffff640 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acd2f0 0x558227adbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa824f99640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8100a2da0 0x7fa8100a5260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8140ce9d0 tx=0x7fa814012880 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa82579a640 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227accdb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa8180138d0 tx=0x7fa818013d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb4c3e26c814be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa80affd640 1 -- 172.21.15.137:0/3710630432 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818017070 con 0x558227acc9d0 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa80affd640 1 -- 172.21.15.137:0/3710630432 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818003970 con 0x558227acc9d0 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa80affd640 1 -- 172.21.15.137:0/3710630432 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180044c0 con 0x558227acc9d0 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa82a081740 1 -- 172.21.15.137:0/3710630432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 msgr2=0x558227accdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa82a081740 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227accdb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa8180138d0 tx=0x7fa818013d90 comp rx=0 tx=0).stop 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa82a081740 1 -- 172.21.15.137:0/3710630432 shutdown_connections 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.057+0000 7fa82a081740 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 0x558227ade4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acd2f0 0x558227adbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 --2- 172.21.15.137:0/3710630432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227accdb0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.065 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 -- 172.21.15.137:0/3710630432 >> 172.21.15.137:0/3710630432 conn(0x558227ac9060 msgr2=0x558227ac9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 -- 172.21.15.137:0/3710630432 shutdown_connections 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 -- 172.21.15.137:0/3710630432 wait complete. 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 -- start start 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 Processor -- start 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acc9d0 0x558227ad9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227ad9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227ad9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x558227adecf0 con 0x558227acd2f0 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5582279ba290 con 0x558227adc0c0 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558227adee70 con 0x558227acc9d0 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82a081740 10 client.0 _ll_register_callbacks cb 0x5582279eb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa81ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227ad9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa81ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227ad9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33066/0 (socket says 172.21.15.137:33066) 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa81ffff640 1 -- 172.21.15.137:0/3450672706 learned_addr learned my addr 172.21.15.137:0/3450672706 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.058+0000 7fa82579a640 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acc9d0 0x558227ad9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa82579a640 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227ad9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa81ffff640 1 -- 172.21.15.137:0/3450672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acc9d0 msgr2=0x558227ad9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa81ffff640 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acc9d0 0x558227ad9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa81ffff640 1 -- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 msgr2=0x558227ad9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa81ffff640 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227ad9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa81ffff640 1 -- 172.21.15.137:0/3450672706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558227744210 con 0x558227acd2f0 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa81ffff640 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227ad9580 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa80c02a940 tx=0x7fa80c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb0ae63885265b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa7fffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa7fffff640 1 -- 172.21.15.137:0/3450672706 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c035040 con 0x558227acd2f0 2024-09-12T12:25:47.067 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa7fffff640 1 -- 172.21.15.137:0/3450672706 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fa80c004240 con 0x558227acd2f0 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa818003970 con 0x558227acd2f0 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa7fffff640 1 -- 172.21.15.137:0/3450672706 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa80c004440 con 0x558227acd2f0 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa82a081740 10 client.16965 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558227810620 con 0x558227acd2f0 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.059+0000 7fa7fffff640 1 -- 172.21.15.137:0/3450672706 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818003970 con 0x558227acd2f0 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa7fffff640 1 -- 172.21.15.137:0/3450672706 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7fa80c04d020 con 0x558227acd2f0 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82a081740 10 client.16965 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82a081740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82a081740 20 client.16965 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82a081740 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227aec100 0x558227aee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82a081740 4 client.16965 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:25:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558227a3ea20 con 0x558227aec100 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82579a640 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227aec100 0x558227aee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa7fffff640 10 client.16965 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.060+0000 7fa82579a640 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227aec100 0x558227aee4f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294a07ea5173b54a server_cookie=be283d8e8f24c895 in_seq=0 out_seq=0 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa7fffff640 1 -- 172.21.15.137:0/3450672706 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558227a3ea20 con 0x558227aec100 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa7fffff640 10 client.16965 handle_command_reply: tid=0 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 client.16965 shutdown 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 msgr2=0x558227ad9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227ad9580 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa80c02a940 tx=0x7fa80c0317e0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 shutdown_connections 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227ad9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227aec100 0x558227aee4f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa7fffff640 0 client.16965 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227ad9580 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 --2- 172.21.15.137:0/3450672706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acc9d0 0x558227ad9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 >> 172.21.15.137:0/3450672706 conn(0x558227ac9060 msgr2=0x558227acedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 shutdown_connections 2024-09-12T12:25:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 1 -- 172.21.15.137:0/3450672706 wait complete. 2024-09-12T12:25:47.070 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.061+0000 7fa82a081740 20 client.16965 trim_cache size 0 max 16384 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 Processor -- start 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 -- start start 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acc9d0 0x558227ad98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227adafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 0x558227af3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201ade40 con 0x558227acd2f0 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201ab6e0 con 0x558227acc9d0 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82a081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201a9550 con 0x558227adc0c0 2024-09-12T12:25:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227adafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227adafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33076/0 (socket says 172.21.15.137:33076) 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 -- 172.21.15.137:0/1539857477 learned_addr learned my addr 172.21.15.137:0/1539857477 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82579a640 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acc9d0 0x558227ad98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82579a640 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 0x558227af3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 -- 172.21.15.137:0/1539857477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 msgr2=0x558227af3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 0x558227af3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 -- 172.21.15.137:0/1539857477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acc9d0 msgr2=0x558227ad98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acc9d0 0x558227ad98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 -- 172.21.15.137:0/1539857477 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558227ac8e90 con 0x558227acd2f0 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa82579a640 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acc9d0 0x558227ad98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa81ffff640 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227adafb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0467b0 tx=0x7fa80c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdad9d52ac18cd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa80a7fc640 1 -- 172.21.15.137:0/1539857477 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c035070 con 0x558227acd2f0 2024-09-12T12:25:47.083 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa80a7fc640 1 -- 172.21.15.137:0/1539857477 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80c005510 con 0x558227acd2f0 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.077+0000 7fa80a7fc640 1 -- 172.21.15.137:0/1539857477 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c004bd0 con 0x558227acd2f0 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 -- 172.21.15.137:0/1539857477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 msgr2=0x558227adafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227adafb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0467b0 tx=0x7fa80c02a420 comp rx=0 tx=0).stop 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 -- 172.21.15.137:0/1539857477 shutdown_connections 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227adc0c0 0x558227af3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acd2f0 0x558227adafb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 --2- 172.21.15.137:0/1539857477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227acc9d0 0x558227ad98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 -- 172.21.15.137:0/1539857477 >> 172.21.15.137:0/1539857477 conn(0x558227ac9060 msgr2=0x558227ada8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 -- 172.21.15.137:0/1539857477 shutdown_connections 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.078+0000 7fa82a081740 1 -- 172.21.15.137:0/1539857477 wait complete. 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 -- start start 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 Processor -- start 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227adf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acd2f0 0x558227adfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227b4ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201a9ba0 con 0x558227acc9d0 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201ab010 con 0x558227adc0c0 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8201aa270 con 0x558227acd2f0 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82a081740 10 client.0 _ll_register_callbacks cb 0x7fa80c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227adf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227adf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33092/0 (socket says 172.21.15.137:33092) 2024-09-12T12:25:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 -- 172.21.15.137:0/1045744534 learned_addr learned my addr 172.21.15.137:0/1045744534 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa81ffff640 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acd2f0 0x558227adfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227b4ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 -- 172.21.15.137:0/1045744534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acd2f0 msgr2=0x558227adfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acd2f0 0x558227adfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 -- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 msgr2=0x558227b4ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227b4ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.079+0000 7fa82579a640 1 -- 172.21.15.137:0/1045744534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558227b4f3e0 con 0x558227acc9d0 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82579a640 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227adf880 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa818004190 tx=0x7fa818013640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f9bd241bfbd8a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa7fffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa7fffff640 1 -- 172.21.15.137:0/1045744534 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818017040 con 0x558227acc9d0 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa7fffff640 1 -- 172.21.15.137:0/1045744534 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fa81800fdd0 con 0x558227acc9d0 2024-09-12T12:25:47.086 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80c005510 con 0x558227acc9d0 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa7fffff640 1 -- 172.21.15.137:0/1045744534 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa818003970 con 0x558227acc9d0 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 10 client.16971 fetch_fsmap learned FSMap version 39 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558227b56780 con 0x558227acc9d0 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa7fffff640 1 -- 172.21.15.137:0/1045744534 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80c005510 con 0x558227acc9d0 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa7fffff640 1 -- 172.21.15.137:0/1045744534 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 39) ==== 8350+0+0 (secure 0 0 0) 0x7fa818040070 con 0x558227acc9d0 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 10 client.16971 fetch_fsmap finished waiting for FSMap version 39 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 20 client.16971 populate_metadata read hostname 'smithi137' 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227b56d00 0x558227b59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 4 client.16971 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558227a38d70 con 0x558227b56d00 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.080+0000 7fa81ffff640 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227b56d00 0x558227b59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:25:47.087 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.081+0000 7fa7fffff640 10 client.16971 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.081+0000 7fa81ffff640 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227b56d00 0x558227b59120 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66f6777de26574a2 server_cookie=ad3f0215524a599d in_seq=0 out_seq=0 2024-09-12T12:25:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.180+0000 7fa7fffff640 1 -- 172.21.15.137:0/1045744534 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558227a38d70 con 0x558227b56d00 2024-09-12T12:25:47.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.180+0000 7fa7fffff640 10 client.16971 handle_command_reply: tid=0 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.180+0000 7fa82a081740 1 client.16971 shutdown 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 msgr2=0x558227adf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227adf880 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa818004190 tx=0x7fa818013640 comp rx=0 tx=0).stop 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 shutdown_connections 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558227adc0c0 0x558227b4ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558227acd2f0 0x558227adfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x558227b56d00 0x558227b59120 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa7fffff640 0 client.16971 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 --2- 172.21.15.137:0/1045744534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558227acc9d0 0x558227adf880 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 >> 172.21.15.137:0/1045744534 conn(0x558227ac9060 msgr2=0x558227ac81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 shutdown_connections 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 1 -- 172.21.15.137:0/1045744534 wait complete. 2024-09-12T12:25:47.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.181+0000 7fa82a081740 20 client.16971 trim_cache size 0 max 16384 2024-09-12T12:25:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:25:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:25:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:25:47.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8100a2da0 msgr2=0x7fa8100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8100a2da0 0x7fa8100a5260 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa8140ce9d0 tx=0x7fa814012880 comp rx=0 tx=0).stop 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 msgr2=0x7fa8201972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa8201972f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa81400eb00 tx=0x7fa81400ef00 comp rx=0 tx=0).stop 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 shutdown_connections 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa8100a2da0 0x7fa8100a5260 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8201069c0 0x7fa82019bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa820101350 0x7fa820197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 --2- 172.21.15.137:0/52895317 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa8201009b0 0x7fa8201972f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 >> 172.21.15.137:0/52895317 conn(0x7fa820077990 msgr2=0x7fa8201037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:25:47.185 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 shutdown_connections 2024-09-12T12:25:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:25:47.182+0000 7fa8271fd640 1 -- 172.21.15.137:0/52895317 wait complete. 2024-09-12T12:25:47.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:47 smithi171 ceph-mon[35098]: pgmap v395: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-12T12:25:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:47 smithi137 ceph-mon[30950]: pgmap v395: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-12T12:25:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:47 smithi189 ceph-mon[35475]: pgmap v395: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-12T12:25:47.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 980993 7.46 MB/sec execute 562 sec latency 51.605 ms 2024-09-12T12:25:48.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 982936 7.45 MB/sec execute 563 sec latency 19.367 ms 2024-09-12T12:25:49.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:49 smithi171 ceph-mon[35098]: pgmap v396: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-12T12:25:49.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:49 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:25:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:49 smithi137 ceph-mon[30950]: pgmap v396: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-12T12:25:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:49 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:25:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:49 smithi189 ceph-mon[35475]: pgmap v396: 97 pgs: 97 active+clean; 283 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-12T12:25:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:49 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:25:49.762 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984176 7.46 MB/sec execute 564 sec latency 582.734 ms 2024-09-12T12:25:50.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984176 7.45 MB/sec execute 565 sec latency 1582.843 ms 2024-09-12T12:25:51.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:51 smithi171 ceph-mon[35098]: pgmap v397: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.9 MiB/s wr, 292 op/s 2024-09-12T12:25:51.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:51 smithi137 ceph-mon[30950]: pgmap v397: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.9 MiB/s wr, 292 op/s 2024-09-12T12:25:51.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:51 smithi189 ceph-mon[35475]: pgmap v397: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.9 MiB/s wr, 292 op/s 2024-09-12T12:25:51.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984176 7.43 MB/sec execute 566 sec latency 2582.959 ms 2024-09-12T12:25:52.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984176 7.42 MB/sec execute 567 sec latency 3583.016 ms 2024-09-12T12:25:53.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:53 smithi171 ceph-mon[35098]: pgmap v398: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-09-12T12:25:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:53 smithi137 ceph-mon[30950]: pgmap v398: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-09-12T12:25:53.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:53 smithi189 ceph-mon[35475]: pgmap v398: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-09-12T12:25:53.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984244 7.41 MB/sec execute 568 sec latency 3715.564 ms 2024-09-12T12:25:54.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984244 7.39 MB/sec execute 569 sec latency 1764.348 ms 2024-09-12T12:25:55.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:55 smithi171 ceph-mon[35098]: pgmap v399: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-09-12T12:25:55.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:55 smithi137 ceph-mon[30950]: pgmap v399: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-09-12T12:25:55.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:55 smithi189 ceph-mon[35475]: pgmap v399: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-09-12T12:25:55.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984244 7.38 MB/sec execute 570 sec latency 2764.447 ms 2024-09-12T12:25:56.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984244 7.37 MB/sec execute 571 sec latency 3764.585 ms 2024-09-12T12:25:57.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:57 smithi171 ceph-mon[35098]: pgmap v400: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-12T12:25:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:57 smithi137 ceph-mon[30950]: pgmap v400: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-12T12:25:57.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:57 smithi189 ceph-mon[35475]: pgmap v400: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-09-12T12:25:57.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 984244 7.36 MB/sec execute 572 sec latency 4764.679 ms 2024-09-12T12:25:58.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 985212 7.35 MB/sec execute 573 sec latency 4898.658 ms 2024-09-12T12:25:59.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:25:59 smithi171 ceph-mon[35098]: pgmap v401: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 914 KiB/s wr, 130 op/s 2024-09-12T12:25:59.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:25:59 smithi137 ceph-mon[30950]: pgmap v401: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 914 KiB/s wr, 130 op/s 2024-09-12T12:25:59.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:25:59 smithi189 ceph-mon[35475]: pgmap v401: 97 pgs: 97 active+clean; 284 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 914 KiB/s wr, 130 op/s 2024-09-12T12:25:59.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987722 7.36 MB/sec execute 574 sec latency 59.944 ms 2024-09-12T12:26:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 -- 172.21.15.137:0/3858018255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 msgr2=0x7f32a4107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 --2- 172.21.15.137:0/3858018255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a4107f60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3294009930 tx=0x7f329402a420 comp rx=0 tx=0).stop 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 -- 172.21.15.137:0/3858018255 shutdown_connections 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 --2- 172.21.15.137:0/3858018255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32a4102480 0x7f32a410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 --2- 172.21.15.137:0/3858018255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32a4101ae0 0x7f32a4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 --2- 172.21.15.137:0/3858018255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a4107f60 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 -- 172.21.15.137:0/3858018255 >> 172.21.15.137:0/3858018255 conn(0x7f32a40fd7e0 msgr2=0x7f32a40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 -- 172.21.15.137:0/3858018255 shutdown_connections 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.272+0000 7f32a9db9640 1 -- 172.21.15.137:0/3858018255 wait complete. 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 Processor -- start 2024-09-12T12:26:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 -- start start 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32a4101ae0 0x7f32a4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32a4102480 0x7f32a41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4111f30 con 0x7f32a4107b80 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f32a4111db0 con 0x7f32a4102480 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a9db9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41120b0 con 0x7f32a4101ae0 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48068/0 (socket says 172.21.15.137:48068) 2024-09-12T12:26:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 -- 172.21.15.137:0/1103612690 learned_addr learned my addr 172.21.15.137:0/1103612690 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a37fe640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32a4101ae0 0x7f32a4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32a4102480 0x7f32a41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 -- 172.21.15.137:0/1103612690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32a4101ae0 msgr2=0x7f32a4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32a4101ae0 0x7f32a4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 -- 172.21.15.137:0/1103612690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32a4102480 msgr2=0x7f32a41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32a4102480 0x7f32a41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.273+0000 7f32a3fff640 1 -- 172.21.15.137:0/1103612690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a419d350 con 0x7f32a4107b80 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.274+0000 7f32a3fff640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a419cc20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f329c00eaa0 tx=0x7f329c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.274+0000 7f32a0ff9640 1 -- 172.21.15.137:0/1103612690 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c00cdb0 con 0x7f32a4107b80 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.274+0000 7f32a0ff9640 1 -- 172.21.15.137:0/1103612690 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c005020 con 0x7f32a4107b80 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.274+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a419d640 con 0x7f32a4107b80 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.274+0000 7f32a0ff9640 1 -- 172.21.15.137:0/1103612690 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0116b0 con 0x7f32a4107b80 2024-09-12T12:26:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.274+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a41a4ed0 con 0x7f32a4107b80 2024-09-12T12:26:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.276+0000 7f32a0ff9640 1 -- 172.21.15.137:0/1103612690 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f329c005560 con 0x7f32a4107b80 2024-09-12T12:26:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.276+0000 7f32acc28740 1 Processor -- start 2024-09-12T12:26:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.276+0000 7f32acc28740 1 -- start start 2024-09-12T12:26:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a0ff9640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f328c0a2da0 0x7f328c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32acc28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb5af0 0x55a7dabb5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32acc28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb6410 0x55a7dabc0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32acc28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabc30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32acc28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7dab1a2f0 con 0x55a7dabb6410 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32acc28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a7daac2270 con 0x55a7dabb5af0 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32acc28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7dab162b0 con 0x55a7dabc0cb0 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a0ff9640 1 -- 172.21.15.137:0/1103612690 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f329c0cf6a0 con 0x7f32a4107b80 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f328c0a2da0 0x7f328c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb6410 0x55a7dabc0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb6410 0x55a7dabc0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48084/0 (socket says 172.21.15.137:48084) 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 -- 172.21.15.137:0/1870789284 learned_addr learned my addr 172.21.15.137:0/1870789284 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a37fe640 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb5af0 0x55a7dabb5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a3fff640 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabc30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 -- 172.21.15.137:0/1870789284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 msgr2=0x55a7dabc30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabc30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 -- 172.21.15.137:0/1870789284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb5af0 msgr2=0x55a7dabb5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb5af0 0x55a7dabb5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 -- 172.21.15.137:0/1870789284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7daac1340 con 0x55a7dabb6410 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a37fe640 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb5af0 0x55a7dabb5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:00.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f328c0a2da0 0x7f328c0a5260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3298009fd0 tx=0x7f3298009b00 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb6410 0x55a7dabc0770 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3298005a70 tx=0x7f329801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a537762f262e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f3289ffb640 1 -- 172.21.15.137:0/1870789284 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298024070 con 0x55a7dabb6410 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.277+0000 7f3289ffb640 1 -- 172.21.15.137:0/1870789284 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298009860 con 0x55a7dabb6410 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f3289ffb640 1 -- 172.21.15.137:0/1870789284 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298020950 con 0x55a7dabb6410 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 -- 172.21.15.137:0/1870789284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb6410 msgr2=0x55a7dabc0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb6410 0x55a7dabc0770 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3298005a70 tx=0x7f329801c000 comp rx=0 tx=0).stop 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 -- 172.21.15.137:0/1870789284 shutdown_connections 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabc30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb6410 0x55a7dabc0770 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 --2- 172.21.15.137:0/1870789284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb5af0 0x55a7dabb5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 -- 172.21.15.137:0/1870789284 >> 172.21.15.137:0/1870789284 conn(0x55a7dabb2060 msgr2=0x55a7dabb2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 -- 172.21.15.137:0/1870789284 shutdown_connections 2024-09-12T12:26:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.278+0000 7f32acc28740 1 -- 172.21.15.137:0/1870789284 wait complete. 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 -- start start 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 Processor -- start 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabbdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dabbe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a7dabc3880 con 0x55a7dabb5af0 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a7daaa3290 con 0x55a7dabb6410 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a7dabc3a00 con 0x55a7dabc0cb0 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32acc28740 10 client.0 _ll_register_callbacks cb 0x55a7daad4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabbdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabbdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48086/0 (socket says 172.21.15.137:48086) 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 -- 172.21.15.137:0/873163128 learned_addr learned my addr 172.21.15.137:0/873163128 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a3fff640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dabbe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 -- 172.21.15.137:0/873163128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 msgr2=0x55a7dabbe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabbe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 -- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 msgr2=0x55a7dabbe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dabbe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a37fe640 1 -- 172.21.15.137:0/873163128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7da82d210 con 0x55a7dabb5af0 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.279+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dabbe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:00.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32a37fe640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabbdc00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f329402a940 tx=0x7f3294031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f1adf16fe24633 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f3282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f3282ffd640 1 -- 172.21.15.137:0/873163128 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294035040 con 0x55a7dabb5af0 2024-09-12T12:26:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f3282ffd640 1 -- 172.21.15.137:0/873163128 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f3294004240 con 0x55a7dabb5af0 2024-09-12T12:26:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298009860 con 0x55a7dabb5af0 2024-09-12T12:26:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f3282ffd640 1 -- 172.21.15.137:0/873163128 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3294004440 con 0x55a7dabb5af0 2024-09-12T12:26:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 10 client.16983 fetch_fsmap learned FSMap version 40 2024-09-12T12:26:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a7da8f9620 con 0x55a7dabb5af0 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f3282ffd640 1 -- 172.21.15.137:0/873163128 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298009860 con 0x55a7dabb5af0 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f3282ffd640 1 -- 172.21.15.137:0/873163128 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f3294045640 con 0x55a7dabb5af0 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 10 client.16983 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 20 client.16983 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dabd0cc0 0x55a7dabd30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 4 client.16983 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.280+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7dab27a20 con 0x55a7dabd0cc0 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.281+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dabd0cc0 0x55a7dabd30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.281+0000 7f3282ffd640 10 client.16983 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.281+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dabd0cc0 0x55a7dabd30b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f5fb57e0ba1250 server_cookie=42f44536da083087 in_seq=0 out_seq=0 2024-09-12T12:26:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f3282ffd640 1 -- 172.21.15.137:0/873163128 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7dab27a20 con 0x55a7dabd0cc0 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f3282ffd640 10 client.16983 handle_command_reply: tid=0 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 client.16983 shutdown 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 msgr2=0x55a7dabbdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabbdc00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f329402a940 tx=0x7f3294031860 comp rx=0 tx=0).stop 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 shutdown_connections 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabc0cb0 0x55a7dabbe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dabbe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dabd0cc0 0x55a7dabd30b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 --2- 172.21.15.137:0/873163128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabbdc00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f3282ffd640 0 client.16983 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 >> 172.21.15.137:0/873163128 conn(0x55a7dabb2060 msgr2=0x55a7dabb7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 shutdown_connections 2024-09-12T12:26:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 1 -- 172.21.15.137:0/873163128 wait complete. 2024-09-12T12:26:00.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.282+0000 7f32acc28740 20 client.16983 trim_cache size 0 max 16384 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 Processor -- start 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 -- start start 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabb8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb6410 0x55a7dabbe4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabc0cb0 0x55a7dabc90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41b00d0 con 0x55a7dabb5af0 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41ad970 con 0x55a7dabc0cb0 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32acc28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41ab7e0 con 0x55a7dabb6410 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabb8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabb8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48100/0 (socket says 172.21.15.137:48100) 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32a37fe640 1 -- 172.21.15.137:0/1827276905 learned_addr learned my addr 172.21.15.137:0/1827276905 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.296+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb6410 0x55a7dabbe4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32a37fe640 1 -- 172.21.15.137:0/1827276905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb6410 msgr2=0x55a7dabbe4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32a37fe640 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb6410 0x55a7dabbe4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32a37fe640 1 -- 172.21.15.137:0/1827276905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabc0cb0 msgr2=0x55a7dabc90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32a37fe640 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabc0cb0 0x55a7dabc90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32a37fe640 1 -- 172.21.15.137:0/1827276905 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7dabb2cf0 con 0x55a7dabb5af0 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb6410 0x55a7dabbe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32a37fe640 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabb8560 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f329402a460 tx=0x7f3294005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454a64e01921166c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32897fa640 1 -- 172.21.15.137:0/1827276905 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3294035030 con 0x55a7dabb5af0 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32897fa640 1 -- 172.21.15.137:0/1827276905 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294045de0 con 0x55a7dabb5af0 2024-09-12T12:26:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32897fa640 1 -- 172.21.15.137:0/1827276905 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32940053a0 con 0x55a7dabb5af0 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32acc28740 1 -- 172.21.15.137:0/1827276905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 msgr2=0x55a7dabb8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32acc28740 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabb8560 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f329402a460 tx=0x7f3294005e80 comp rx=0 tx=0).stop 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32acc28740 1 -- 172.21.15.137:0/1827276905 shutdown_connections 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32acc28740 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabc0cb0 0x55a7dabc90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32acc28740 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb6410 0x55a7dabbe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32acc28740 1 --2- 172.21.15.137:0/1827276905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabb5af0 0x55a7dabb8560 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.297+0000 7f32acc28740 1 -- 172.21.15.137:0/1827276905 >> 172.21.15.137:0/1827276905 conn(0x55a7dabb2060 msgr2=0x55a7dabbfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 -- 172.21.15.137:0/1827276905 shutdown_connections 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 -- 172.21.15.137:0/1827276905 wait complete. 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 -- start start 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 Processor -- start 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb5af0 0x55a7dac33950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dac33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabc0cb0 0x55a7dac343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41abe30 con 0x55a7dabc0cb0 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41ad2a0 con 0x55a7dabb6410 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32a41ac500 con 0x55a7dabb5af0 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32acc28740 10 client.0 _ll_register_callbacks cb 0x7f3294009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabc0cb0 0x55a7dac343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32a3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabc0cb0 0x55a7dac343d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48108/0 (socket says 172.21.15.137:48108) 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32a37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb5af0 0x55a7dac33950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32a3fff640 1 -- 172.21.15.137:0/2738583151 learned_addr learned my addr 172.21.15.137:0/2738583151 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.298+0000 7f32a2ffd640 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dac33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32a3fff640 1 -- 172.21.15.137:0/2738583151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb5af0 msgr2=0x55a7dac33950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32a3fff640 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb5af0 0x55a7dac33950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32a3fff640 1 -- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 msgr2=0x55a7dac33e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32a3fff640 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dac33e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32a3fff640 1 -- 172.21.15.137:0/2738583151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7dabad2d0 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32a3fff640 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabc0cb0 0x55a7dac343d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f329c0cce10 tx=0x7f329c09c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2375bb5b7f5b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f3282ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f3282ffd640 1 -- 172.21.15.137:0/2738583151 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c01f040 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f3282ffd640 1 -- 172.21.15.137:0/2738583151 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f329c0cf0d0 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3294045de0 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f3282ffd640 1 -- 172.21.15.137:0/2738583151 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f329c0cf2d0 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 10 client.16989 fetch_fsmap learned FSMap version 40 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a7dac3fca0 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f3282ffd640 1 -- 172.21.15.137:0/2738583151 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3294045de0 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f3282ffd640 1 -- 172.21.15.137:0/2738583151 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 40) ==== 8354+0+0 (secure 0 0 0) 0x7f329c00f040 con 0x55a7dabc0cb0 2024-09-12T12:26:00.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 10 client.16989 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 20 client.16989 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dac403f0 0x55a7dac42810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 4 client.16989 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.299+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7dab21d70 con 0x55a7dac403f0 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.300+0000 7f32a37fe640 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dac403f0 0x55a7dac42810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.300+0000 7f3282ffd640 10 client.16989 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:00.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.300+0000 7f32a37fe640 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dac403f0 0x55a7dac42810 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a07f408802d0f122 server_cookie=7d8b6a7021c61f0b in_seq=0 out_seq=0 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f3282ffd640 1 -- 172.21.15.137:0/2738583151 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7dab21d70 con 0x55a7dac403f0 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f3282ffd640 10 client.16989 handle_command_reply: tid=0 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 client.16989 shutdown 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabc0cb0 msgr2=0x55a7dac343d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabc0cb0 0x55a7dac343d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f329c0cce10 tx=0x7f329c09c3b0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 shutdown_connections 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a7dac403f0 0x55a7dac42810 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a7dabc0cb0 0x55a7dac343d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f3282ffd640 0 client.16989 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:00.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a7dabb6410 0x55a7dac33e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 --2- 172.21.15.137:0/2738583151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a7dabb5af0 0x55a7dac33950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 >> 172.21.15.137:0/2738583151 conn(0x55a7dabb2060 msgr2=0x55a7dabbf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 shutdown_connections 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 1 -- 172.21.15.137:0/2738583151 wait complete. 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.355+0000 7f32acc28740 20 client.16989 trim_cache size 0 max 16384 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:26:00.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:26:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f328c0a2da0 msgr2=0x7f328c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f328c0a2da0 0x7f328c0a5260 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3298009fd0 tx=0x7f3298009b00 comp rx=0 tx=0).stop 2024-09-12T12:26:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 msgr2=0x7f32a419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:00.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a419cc20 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f329c00eaa0 tx=0x7f329c00ef60 comp rx=0 tx=0).stop 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 shutdown_connections 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f328c0a2da0 0x7f328c0a5260 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32a4107b80 0x7f32a419cc20 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32a4102480 0x7f32a41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 --2- 172.21.15.137:0/1103612690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32a4101ae0 0x7f32a4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 >> 172.21.15.137:0/1103612690 conn(0x7f32a40fd7e0 msgr2=0x7f32a40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 shutdown_connections 2024-09-12T12:26:00.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:00.356+0000 7f32a9db9640 1 -- 172.21.15.137:0/1103612690 wait complete. 2024-09-12T12:26:00.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987722 7.35 MB/sec execute 575 sec latency 1060.023 ms 2024-09-12T12:26:01.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:01 smithi171 ceph-mon[35098]: pgmap v402: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-12T12:26:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:01 smithi137 ceph-mon[30950]: pgmap v402: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-12T12:26:01.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:01 smithi189 ceph-mon[35475]: pgmap v402: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-12T12:26:01.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987722 7.34 MB/sec execute 576 sec latency 2060.106 ms 2024-09-12T12:26:02.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987722 7.33 MB/sec execute 577 sec latency 3060.163 ms 2024-09-12T12:26:03.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:03 smithi171 ceph-mon[35098]: pgmap v403: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 295 KiB/s wr, 38 op/s 2024-09-12T12:26:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:03 smithi137 ceph-mon[30950]: pgmap v403: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 295 KiB/s wr, 38 op/s 2024-09-12T12:26:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:03 smithi189 ceph-mon[35475]: pgmap v403: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 295 KiB/s wr, 38 op/s 2024-09-12T12:26:03.763 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987789 7.31 MB/sec execute 578 sec latency 3204.504 ms 2024-09-12T12:26:04.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987789 7.30 MB/sec execute 579 sec latency 1762.257 ms 2024-09-12T12:26:05.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:05 smithi171 ceph-mon[35098]: pgmap v404: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 295 KiB/s wr, 38 op/s 2024-09-12T12:26:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:05 smithi137 ceph-mon[30950]: pgmap v404: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 295 KiB/s wr, 38 op/s 2024-09-12T12:26:05.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:05 smithi189 ceph-mon[35475]: pgmap v404: 97 pgs: 97 active+clean; 287 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 295 KiB/s wr, 38 op/s 2024-09-12T12:26:05.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987789 7.29 MB/sec execute 580 sec latency 2762.372 ms 2024-09-12T12:26:06.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987789 7.27 MB/sec execute 581 sec latency 3762.480 ms 2024-09-12T12:26:07.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:07 smithi171 ceph-mon[35098]: pgmap v405: 97 pgs: 97 active+clean; 283 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-09-12T12:26:07.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:07 smithi137 ceph-mon[30950]: pgmap v405: 97 pgs: 97 active+clean; 283 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-09-12T12:26:07.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:07 smithi189 ceph-mon[35475]: pgmap v405: 97 pgs: 97 active+clean; 283 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-09-12T12:26:07.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 987789 7.26 MB/sec execute 582 sec latency 4762.532 ms 2024-09-12T12:26:08.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:08 smithi137 ceph-mon[30950]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T12:26:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:08 smithi189 ceph-mon[35475]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T12:26:08.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:08 smithi171 ceph-mon[35098]: osdmap e102: 12 total, 12 up, 12 in 2024-09-12T12:26:08.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 989036 7.26 MB/sec execute 583 sec latency 4903.406 ms 2024-09-12T12:26:09.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:09 smithi137 ceph-mon[30950]: pgmap v407: 97 pgs: 97 active+clean; 283 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.9 MiB/s wr, 180 op/s 2024-09-12T12:26:09.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:09 smithi189 ceph-mon[35475]: pgmap v407: 97 pgs: 97 active+clean; 283 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.9 MiB/s wr, 180 op/s 2024-09-12T12:26:09.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:09 smithi171 ceph-mon[35098]: pgmap v407: 97 pgs: 97 active+clean; 283 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 2.9 MiB/s wr, 180 op/s 2024-09-12T12:26:09.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991300 7.27 MB/sec execute 584 sec latency 176.979 ms 2024-09-12T12:26:10.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991300 7.26 MB/sec execute 585 sec latency 1177.057 ms 2024-09-12T12:26:11.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:11 smithi137 ceph-mon[30950]: pgmap v408: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-12T12:26:11.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:11 smithi189 ceph-mon[35475]: pgmap v408: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-12T12:26:11.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:11 smithi171 ceph-mon[35098]: pgmap v408: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-12T12:26:11.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991300 7.25 MB/sec execute 586 sec latency 2177.153 ms 2024-09-12T12:26:12.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991300 7.23 MB/sec execute 587 sec latency 3177.199 ms 2024-09-12T12:26:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 -- 172.21.15.137:0/2920686577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 msgr2=0x7f4dfc1107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 --2- 172.21.15.137:0/2920686577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc1107c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009900 tx=0x7f4df402a410 comp rx=0 tx=0).stop 2024-09-12T12:26:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 -- 172.21.15.137:0/2920686577 shutdown_connections 2024-09-12T12:26:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 --2- 172.21.15.137:0/2920686577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc1107c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 --2- 172.21.15.137:0/2920686577 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dfc1011d0 0x7f4dfc10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 --2- 172.21.15.137:0/2920686577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dfc1008b0 0x7f4dfc100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.453 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 -- 172.21.15.137:0/2920686577 >> 172.21.15.137:0/2920686577 conn(0x7f4dfc0fc6b0 msgr2=0x7f4dfc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 -- 172.21.15.137:0/2920686577 shutdown_connections 2024-09-12T12:26:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 -- 172.21.15.137:0/2920686577 wait complete. 2024-09-12T12:26:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.452+0000 7f4e01653640 1 Processor -- start 2024-09-12T12:26:13.454 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4e01653640 1 -- start start 2024-09-12T12:26:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4e01653640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dfc1008b0 0x7f4dfc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4e01653640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dfc1011d0 0x7f4dfc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4e01653640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4e01653640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1140b0 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4e01653640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc113f30 con 0x7f4dfc1008b0 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4e01653640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc114230 con 0x7f4dfc1011d0 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44398/0 (socket says 172.21.15.137:44398) 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 -- 172.21.15.137:0/3245195246 learned_addr learned my addr 172.21.15.137:0/3245195246 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dfc1011d0 0x7f4dfc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dfc1008b0 0x7f4dfc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 -- 172.21.15.137:0/3245195246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dfc1011d0 msgr2=0x7f4dfc19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dfc1011d0 0x7f4dfc19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 -- 172.21.15.137:0/3245195246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dfc1008b0 msgr2=0x7f4dfc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dfc1008b0 0x7f4dfc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 -- 172.21.15.137:0/3245195246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc19f550 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.453+0000 7f4dfb7fe640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc19ee20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4df4005ab0 tx=0x7f4df40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.454+0000 7f4de3fff640 1 -- 172.21.15.137:0/3245195246 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df403f030 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.454+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc19f7e0 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.454+0000 7f4de3fff640 1 -- 172.21.15.137:0/3245195246 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df4003ce0 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.454+0000 7f4de3fff640 1 -- 172.21.15.137:0/3245195246 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4035070 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.454+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1a70d0 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.456+0000 7f4de3fff640 1 -- 172.21.15.137:0/3245195246 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4df4042030 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.456+0000 7f4e043f2740 1 Processor -- start 2024-09-12T12:26:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4e043f2740 1 -- start start 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd10b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd10bcb0 0x55babd10e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4e043f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55babd0652f0 con 0x55babd101410 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4e043f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55babd00d270 con 0x55babd100af0 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4e043f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55babd0612b0 con 0x55babd10bcb0 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4de3fff640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4de40a2da0 0x7f4de40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd10b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd10b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44410/0 (socket says 172.21.15.137:44410) 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2745740353 learned_addr learned my addr 172.21.15.137:0/2745740353 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:13.464 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfb7fe640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4de40a2da0 0x7f4de40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd10bcb0 0x55babd10e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4de3fff640 1 -- 172.21.15.137:0/3245195246 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f4df40ef810 con 0x7f4dfc10e3d0 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2745740353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd10bcb0 msgr2=0x55babd10e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd10bcb0 0x55babd10e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2745740353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 msgr2=0x55babd100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2745740353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55babd00c340 con 0x55babd101410 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.457+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.458+0000 7f4dfb7fe640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4de40a2da0 0x7f4de40a5260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4df40ed030 tx=0x7f4df408bc60 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.458+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd10b770 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b760 tx=0x7f4dec00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56935d5c6f8d922 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.458+0000 7f4de17fa640 1 -- 172.21.15.137:0/2745740353 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec003dd0 con 0x55babd101410 2024-09-12T12:26:13.465 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.458+0000 7f4de17fa640 1 -- 172.21.15.137:0/2745740353 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dec015070 con 0x55babd101410 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.458+0000 7f4de17fa640 1 -- 172.21.15.137:0/2745740353 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec011440 con 0x55babd101410 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.458+0000 7f4e043f2740 1 -- 172.21.15.137:0/2745740353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 msgr2=0x55babd10b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.458+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd10b770 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b760 tx=0x7f4dec00bc20 comp rx=0 tx=0).stop 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- 172.21.15.137:0/2745740353 shutdown_connections 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd10bcb0 0x55babd10e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd10b770 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2745740353 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- 172.21.15.137:0/2745740353 >> 172.21.15.137:0/2745740353 conn(0x55babd0fd060 msgr2=0x55babd0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- 172.21.15.137:0/2745740353 shutdown_connections 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- 172.21.15.137:0/2745740353 wait complete. 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- start start 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 Processor -- start 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd109060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd1095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55babd10e880 con 0x55babd101410 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55babcfee290 con 0x55babd10bcb0 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55babd10ea00 con 0x55babd100af0 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4e043f2740 10 client.0 _ll_register_callbacks cb 0x55babd01f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd109060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4dfa7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4dfa7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:35024/0 (socket says 172.21.15.137:35024) 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4dfa7fc640 1 -- 172.21.15.137:0/1628628921 learned_addr learned my addr 172.21.15.137:0/1628628921 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd109060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44424/0 (socket says 172.21.15.137:44424) 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.459+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd1095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4dfaffd640 1 -- 172.21.15.137:0/1628628921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 msgr2=0x55babd108b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd108b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4dfaffd640 1 -- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 msgr2=0x55babd1095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd1095a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4dfaffd640 1 -- 172.21.15.137:0/1628628921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55babcd78210 con 0x55babd101410 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd109060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b760 tx=0x7f4dec003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e27112695491b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4ddeff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4ddeff5640 1 -- 172.21.15.137:0/1628628921 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec028060 con 0x55babd101410 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4ddeff5640 1 -- 172.21.15.137:0/1628628921 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f4dec011850 con 0x55babd101410 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dec015070 con 0x55babd101410 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4ddeff5640 1 -- 172.21.15.137:0/1628628921 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4dec00be70 con 0x55babd101410 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4e043f2740 10 client.17001 fetch_fsmap learned FSMap version 41 2024-09-12T12:26:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55babce44620 con 0x55babd101410 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.460+0000 7f4ddeff5640 1 -- 172.21.15.137:0/1628628921 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dec015070 con 0x55babd101410 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4ddeff5640 1 -- 172.21.15.137:0/1628628921 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f4dec022030 con 0x55babd101410 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4e043f2740 10 client.17001 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4e043f2740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4e043f2740 20 client.17001 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4e043f2740 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd11bbb0 0x55babd11dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4e043f2740 4 client.17001 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55babd072a20 con 0x55babd11bbb0 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd11bbb0 0x55babd11dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd11bbb0 0x55babd11dfa0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c6d07b2829ebca server_cookie=2dc4362b22705bac in_seq=0 out_seq=0 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.461+0000 7f4ddeff5640 10 client.17001 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4ddeff5640 1 -- 172.21.15.137:0/1628628921 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55babd072a20 con 0x55babd11bbb0 2024-09-12T12:26:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4ddeff5640 10 client.17001 handle_command_reply: tid=0 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 client.17001 shutdown 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 msgr2=0x55babd109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd109060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00b760 tx=0x7f4dec003e00 comp rx=0 tx=0).stop 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 shutdown_connections 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd1095a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd11bbb0 0x55babd11dfa0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd109060 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 --2- 172.21.15.137:0/1628628921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 >> 172.21.15.137:0/1628628921 conn(0x55babd0fd060 msgr2=0x55babd102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4ddeff5640 0 client.17001 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 shutdown_connections 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 1 -- 172.21.15.137:0/1628628921 wait complete. 2024-09-12T12:26:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.462+0000 7f4e043f2740 20 client.17001 trim_cache size 0 max 16384 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.476+0000 7f4e043f2740 1 Processor -- start 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4e043f2740 1 -- start start 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd1093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd1235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd123b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4e043f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1b2280 con 0x55babd101410 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4e043f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1afaf0 con 0x55babd10bcb0 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4e043f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1ad960 con 0x55babd100af0 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd1235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd1235d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44434/0 (socket says 172.21.15.137:44434) 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2741683304 learned_addr learned my addr 172.21.15.137:0/2741683304 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd1093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd123b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2741683304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 msgr2=0x55babd1093d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd1093d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2741683304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 msgr2=0x55babd123b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd123b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 -- 172.21.15.137:0/2741683304 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55babd0fce90 con 0x55babd101410 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd1235d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4dec004a50 tx=0x7f4dec004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c604c925b066eca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4de0ff9640 1 -- 172.21.15.137:0/2741683304 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec004790 con 0x55babd101410 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4de0ff9640 1 -- 172.21.15.137:0/2741683304 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dec015070 con 0x55babd101410 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.477+0000 7f4de0ff9640 1 -- 172.21.15.137:0/2741683304 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec010580 con 0x55babd101410 2024-09-12T12:26:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 -- 172.21.15.137:0/2741683304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 msgr2=0x55babd1235d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd1235d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4dec004a50 tx=0x7f4dec004120 comp rx=0 tx=0).stop 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 -- 172.21.15.137:0/2741683304 shutdown_connections 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd10bcb0 0x55babd123b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd101410 0x55babd1235d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 --2- 172.21.15.137:0/2741683304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd100af0 0x55babd1093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 -- 172.21.15.137:0/2741683304 >> 172.21.15.137:0/2741683304 conn(0x55babd0fd060 msgr2=0x55babd10aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 -- 172.21.15.137:0/2741683304 shutdown_connections 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 -- 172.21.15.137:0/2741683304 wait complete. 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 -- start start 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.478+0000 7f4e043f2740 1 Processor -- start 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd1145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd101410 0x55babd1059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4e043f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd10bcb0 0x55babd106130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4e043f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1adfb0 con 0x55babd10bcb0 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4e043f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1af420 con 0x55babd100af0 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4e043f2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1ae680 con 0x55babd101410 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4e043f2740 10 client.0 _ll_register_callbacks cb 0x7f4df0009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd10bcb0 0x55babd106130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd101410 0x55babd1059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd10bcb0 0x55babd106130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:44440/0 (socket says 172.21.15.137:44440) 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 -- 172.21.15.137:0/4217667950 learned_addr learned my addr 172.21.15.137:0/4217667950 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd1145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 -- 172.21.15.137:0/4217667950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd101410 msgr2=0x55babd1059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd101410 0x55babd1059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 -- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 msgr2=0x55babd1145c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd1145c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 -- 172.21.15.137:0/4217667950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55babd1120a0 con 0x55babd10bcb0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd101410 0x55babd1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4dfa7fc640 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd10bcb0 0x55babd106130 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x55babcfee350 tx=0x7f4dec004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66796af5c0d63cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4ddeff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.479+0000 7f4ddeff5640 1 -- 172.21.15.137:0/4217667950 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0034060 con 0x55babd10bcb0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4ddeff5640 1 -- 172.21.15.137:0/4217667950 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f4df0004ab0 con 0x55babd10bcb0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dec015070 con 0x55babd10bcb0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4ddeff5640 1 -- 172.21.15.137:0/4217667950 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4df0004cb0 con 0x55babd10bcb0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 10 client.17007 fetch_fsmap learned FSMap version 41 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55babd1867e0 con 0x55babd10bcb0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4ddeff5640 1 -- 172.21.15.137:0/4217667950 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dec015070 con 0x55babd10bcb0 2024-09-12T12:26:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4ddeff5640 1 -- 172.21.15.137:0/4217667950 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f4df000f040 con 0x55babd10bcb0 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 10 client.17007 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 20 client.17007 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd186f40 0x55babd189360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 4 client.17007 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55babd06cd70 con 0x55babd186f40 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.480+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd186f40 0x55babd189360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.481+0000 7f4ddeff5640 10 client.17007 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.481+0000 7f4dfaffd640 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd186f40 0x55babd189360 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a670cf17d8de7f server_cookie=7bc0503c2ea7ee2c in_seq=0 out_seq=0 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.518+0000 7f4ddeff5640 1 -- 172.21.15.137:0/4217667950 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55babd06cd70 con 0x55babd186f40 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.518+0000 7f4ddeff5640 10 client.17007 handle_command_reply: tid=0 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.518+0000 7f4e043f2740 1 client.17007 shutdown 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.518+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd10bcb0 msgr2=0x55babd106130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.518+0000 7f4e043f2740 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd10bcb0 0x55babd106130 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x55babcfee350 tx=0x7f4dec004a80 comp rx=0 tx=0).stop 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.518+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 shutdown_connections 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.518+0000 7f4e043f2740 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55babd186f40 0x55babd189360 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e043f2740 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55babd10bcb0 0x55babd106130 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e043f2740 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55babd101410 0x55babd1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e043f2740 1 --2- 172.21.15.137:0/4217667950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55babd100af0 0x55babd1145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4ddeff5640 0 client.17007 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 >> 172.21.15.137:0/4217667950 conn(0x55babd0fd060 msgr2=0x55babd0fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 shutdown_connections 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e043f2740 1 -- 172.21.15.137:0/4217667950 wait complete. 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e043f2740 20 client.17007 trim_cache size 0 max 16384 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:26:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:26:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4de40a2da0 msgr2=0x7f4de40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4de40a2da0 0x7f4de40a5260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4df40ed030 tx=0x7f4df408bc60 comp rx=0 tx=0).stop 2024-09-12T12:26:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 msgr2=0x7f4dfc19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc19ee20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4df4005ab0 tx=0x7f4df40042f0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 shutdown_connections 2024-09-12T12:26:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4de40a2da0 0x7f4de40a5260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4dfc10e3d0 0x7f4dfc19ee20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4dfc1011d0 0x7f4dfc19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 --2- 172.21.15.137:0/3245195246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dfc1008b0 0x7f4dfc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 >> 172.21.15.137:0/3245195246 conn(0x7f4dfc0fc6b0 msgr2=0x7f4dfc0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 shutdown_connections 2024-09-12T12:26:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:13.519+0000 7f4e01653640 1 -- 172.21.15.137:0/3245195246 wait complete. 2024-09-12T12:26:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:13 smithi137 ceph-mon[30950]: pgmap v409: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-12T12:26:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:13 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:26:13.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:13 smithi137 ceph-mon[30950]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T12:26:13.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:13 smithi189 ceph-mon[35475]: pgmap v409: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-12T12:26:13.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:13 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:26:13.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:13 smithi189 ceph-mon[35475]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T12:26:13.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:13 smithi171 ceph-mon[35098]: pgmap v409: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.0 MiB/s wr, 184 op/s 2024-09-12T12:26:13.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:13 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:26:13.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:13 smithi171 ceph-mon[35098]: osdmap e103: 12 total, 12 up, 12 in 2024-09-12T12:26:13.764 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991367 7.22 MB/sec execute 588 sec latency 3312.385 ms 2024-09-12T12:26:14.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991367 7.21 MB/sec execute 589 sec latency 1771.523 ms 2024-09-12T12:26:15.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:15 smithi137 ceph-mon[30950]: pgmap v411: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 510 KiB/s wr, 49 op/s 2024-09-12T12:26:15.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:15 smithi189 ceph-mon[35475]: pgmap v411: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 510 KiB/s wr, 49 op/s 2024-09-12T12:26:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:15 smithi171 ceph-mon[35098]: pgmap v411: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 816 KiB/s rd, 510 KiB/s wr, 49 op/s 2024-09-12T12:26:15.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991367 7.20 MB/sec execute 590 sec latency 2771.644 ms 2024-09-12T12:26:16.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991367 7.18 MB/sec execute 591 sec latency 3771.761 ms 2024-09-12T12:26:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:17 smithi137 ceph-mon[30950]: pgmap v412: 97 pgs: 97 active+clean; 281 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 602 KiB/s wr, 83 op/s 2024-09-12T12:26:17.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:17 smithi189 ceph-mon[35475]: pgmap v412: 97 pgs: 97 active+clean; 281 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 602 KiB/s wr, 83 op/s 2024-09-12T12:26:17.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:17 smithi171 ceph-mon[35098]: pgmap v412: 97 pgs: 97 active+clean; 281 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1007 KiB/s rd, 602 KiB/s wr, 83 op/s 2024-09-12T12:26:17.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 991367 7.17 MB/sec execute 592 sec latency 4771.879 ms 2024-09-12T12:26:18.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 992490 7.17 MB/sec execute 593 sec latency 4903.824 ms 2024-09-12T12:26:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:19 smithi137 ceph-mon[30950]: pgmap v413: 97 pgs: 97 active+clean; 281 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 503 KiB/s wr, 69 op/s 2024-09-12T12:26:19.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:19 smithi189 ceph-mon[35475]: pgmap v413: 97 pgs: 97 active+clean; 281 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 503 KiB/s wr, 69 op/s 2024-09-12T12:26:19.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:19 smithi171 ceph-mon[35098]: pgmap v413: 97 pgs: 97 active+clean; 281 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 503 KiB/s wr, 69 op/s 2024-09-12T12:26:19.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 994847 7.18 MB/sec execute 594 sec latency 122.464 ms 2024-09-12T12:26:20.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 994847 7.17 MB/sec execute 595 sec latency 1122.564 ms 2024-09-12T12:26:21.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:21 smithi137 ceph-mon[30950]: pgmap v414: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 441 KiB/s wr, 67 op/s 2024-09-12T12:26:21.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:21 smithi189 ceph-mon[35475]: pgmap v414: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 441 KiB/s wr, 67 op/s 2024-09-12T12:26:21.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:21 smithi171 ceph-mon[35098]: pgmap v414: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 441 KiB/s wr, 67 op/s 2024-09-12T12:26:21.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 994847 7.16 MB/sec execute 596 sec latency 2122.630 ms 2024-09-12T12:26:22.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 994847 7.14 MB/sec execute 597 sec latency 3122.751 ms 2024-09-12T12:26:23.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:23 smithi137 ceph-mon[30950]: pgmap v415: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 441 KiB/s wr, 67 op/s 2024-09-12T12:26:23.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:23 smithi189 ceph-mon[35475]: pgmap v415: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 441 KiB/s wr, 67 op/s 2024-09-12T12:26:23.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:23 smithi171 ceph-mon[35098]: pgmap v415: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 441 KiB/s wr, 67 op/s 2024-09-12T12:26:23.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 994914 7.13 MB/sec execute 598 sec latency 3252.741 ms 2024-09-12T12:26:24.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 994914 7.12 MB/sec execute 599 sec latency 1764.743 ms 2024-09-12T12:26:25.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:25 smithi137 ceph-mon[30950]: pgmap v416: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 405 KiB/s wr, 61 op/s 2024-09-12T12:26:25.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:25 smithi189 ceph-mon[35475]: pgmap v416: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 405 KiB/s wr, 61 op/s 2024-09-12T12:26:25.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:25 smithi171 ceph-mon[35098]: pgmap v416: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 405 KiB/s wr, 61 op/s 2024-09-12T12:26:25.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 cleanup 600 sec 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/3217456809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 msgr2=0x7f4fb810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/3217456809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb810e9b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009cd0 tx=0x7f4fb002a8e0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/3217456809 shutdown_connections 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/3217456809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb810e9b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/3217456809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb8101ae0 0x7f4fb8101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/3217456809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fb8107b80 0x7f4fb8107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/3217456809 >> 172.21.15.137:0/3217456809 conn(0x7f4fb80fd7e0 msgr2=0x7f4fb80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/3217456809 shutdown_connections 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/3217456809 wait complete. 2024-09-12T12:26:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 Processor -- start 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.614+0000 7f4fbe4f1640 1 -- start start 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fbe4f1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb8101ae0 0x7f4fb80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fbe4f1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fbe4f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fb8107b80 0x7f4fb8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fbe4f1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111ef0 con 0x7f4fb8102480 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fbe4f1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8111d70 con 0x7f4fb8101ae0 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fbe4f1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8112070 con 0x7f4fb8107b80 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34228/0 (socket says 172.21.15.137:34228) 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 -- 172.21.15.137:0/1076726709 learned_addr learned my addr 172.21.15.137:0/1076726709 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb8101ae0 0x7f4fb80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fb8107b80 0x7f4fb8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 -- 172.21.15.137:0/1076726709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fb8107b80 msgr2=0x7f4fb8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fb8107b80 0x7f4fb8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 -- 172.21.15.137:0/1076726709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb8101ae0 msgr2=0x7f4fb80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb8101ae0 0x7f4fb80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.615+0000 7f4fb77fe640 1 -- 172.21.15.137:0/1076726709 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb81a6800 con 0x7f4fb8102480 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.616+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb8074a30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9f0 tx=0x7f4fac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.616+0000 7f4fb57fa640 1 -- 172.21.15.137:0/1076726709 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00cd80 con 0x7f4fb8102480 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.616+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb81a6af0 con 0x7f4fb8102480 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.616+0000 7f4fb57fa640 1 -- 172.21.15.137:0/1076726709 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fac010910 con 0x7f4fb8102480 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.616+0000 7f4fb57fa640 1 -- 172.21.15.137:0/1076726709 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0188f0 con 0x7f4fb8102480 2024-09-12T12:26:26.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.616+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb81a7040 con 0x7f4fb8102480 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.618+0000 7f4fb57fa640 1 -- 172.21.15.137:0/1076726709 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4fac010ac0 con 0x7f4fb8102480 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.618+0000 7f4fc128f740 1 Processor -- start 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.618+0000 7f4fc128f740 1 -- start start 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c5410 0x55e3221cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221cfcb0 0x55e3221d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fc128f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3221292f0 con 0x55e3221c4af0 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fc128f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e3220d1270 con 0x55e3221c5410 2024-09-12T12:26:26.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fc128f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e3221252b0 con 0x55e3221cfcb0 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34232/0 (socket says 172.21.15.137:34232) 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 -- 172.21.15.137:0/3430969879 learned_addr learned my addr 172.21.15.137:0/3430969879 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221cfcb0 0x55e3221d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fb57fa640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4fa00a2da0 0x7f4fa00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c5410 0x55e3221cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4fa00a2da0 0x7f4fa00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 -- 172.21.15.137:0/3430969879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221cfcb0 msgr2=0x55e3221d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221cfcb0 0x55e3221d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 -- 172.21.15.137:0/3430969879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c5410 msgr2=0x55e3221cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c5410 0x55e3221cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 -- 172.21.15.137:0/3430969879 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3220d0340 con 0x55e3221c4af0 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fb57fa640 1 -- 172.21.15.137:0/1076726709 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f4fac0cf890 con 0x7f4fb8102480 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.619+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221c4ed0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0004af0 tx=0x7f4fb00318c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758ec9d88c91e2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4f9e7fc640 1 -- 172.21.15.137:0/3430969879 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0031430 con 0x55e3221c4af0 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4f9e7fc640 1 -- 172.21.15.137:0/3430969879 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0036040 con 0x55e3221c4af0 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4f9e7fc640 1 -- 172.21.15.137:0/3430969879 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00305a0 con 0x55e3221c4af0 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fc128f740 1 -- 172.21.15.137:0/3430969879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 msgr2=0x55e3221c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4fa00a2da0 0x7f4fa00a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4fb004d000 tx=0x7f4fb004b000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fc128f740 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221c4ed0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0004af0 tx=0x7f4fb00318c0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fc128f740 1 -- 172.21.15.137:0/3430969879 shutdown_connections 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fc128f740 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221cfcb0 0x55e3221d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fc128f740 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c5410 0x55e3221cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fc128f740 1 --2- 172.21.15.137:0/3430969879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221c4ed0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.620+0000 7f4fc128f740 1 -- 172.21.15.137:0/3430969879 >> 172.21.15.137:0/3430969879 conn(0x55e3221c1060 msgr2=0x55e3221c1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 -- 172.21.15.137:0/3430969879 shutdown_connections 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 -- 172.21.15.137:0/3430969879 wait complete. 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 -- start start 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 Processor -- start 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e3221d2880 con 0x55e3221c4af0 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e3220b2290 con 0x55e3221cfcb0 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e3221d2a00 con 0x55e3221c5410 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fc128f740 10 client.0 _ll_register_callbacks cb 0x55e3220e3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:26.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.621+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34246/0 (socket says 172.21.15.137:34246) 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 -- 172.21.15.137:0/1805664129 learned_addr learned my addr 172.21.15.137:0/1805664129 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 -- 172.21.15.137:0/1805664129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 msgr2=0x55e3221cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 -- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 msgr2=0x55e3221cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 -- 172.21.15.137:0/1805664129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e321e3c210 con 0x55e3221c4af0 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221ccb30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800b9f0 tx=0x7f4fa800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debcaf8ef064614c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:26.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4f977fe640 1 -- 172.21.15.137:0/1805664129 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8011440 con 0x55e3221c4af0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4f977fe640 1 -- 172.21.15.137:0/1805664129 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f4fa80115e0 con 0x55e3221c4af0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb0036040 con 0x55e3221c4af0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4f977fe640 1 -- 172.21.15.137:0/1805664129 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4fa80117e0 con 0x55e3221c4af0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fc128f740 10 client.17019 fetch_fsmap learned FSMap version 41 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.622+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e321f08620 con 0x55e3221c4af0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4f977fe640 1 -- 172.21.15.137:0/1805664129 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb0036040 con 0x55e3221c4af0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4f977fe640 1 -- 172.21.15.137:0/1805664129 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f4fa8022070 con 0x55e3221c4af0 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4fc128f740 10 client.17019 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4fc128f740 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4fc128f740 20 client.17019 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e3221dfc30 0x55e3221e2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4fc128f740 4 client.17019 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e322136a20 con 0x55e3221dfc30 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.623+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e3221dfc30 0x55e3221e2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.624+0000 7f4f977fe640 10 client.17019 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.624+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e3221dfc30 0x55e3221e2060 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5e1cf620067cfac server_cookie=f679299e3f55f246 in_seq=0 out_seq=0 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.624+0000 7f4f977fe640 1 -- 172.21.15.137:0/1805664129 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e322136a20 con 0x55e3221dfc30 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.624+0000 7f4f977fe640 10 client.17019 handle_command_reply: tid=0 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 client.17019 shutdown 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 msgr2=0x55e3221ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221ccb30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800b9f0 tx=0x7f4fa800beb0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 shutdown_connections 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.632 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e3221dfc30 0x55e3221e2060 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1805664129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c4af0 0x55e3221ccb30 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 >> 172.21.15.137:0/1805664129 conn(0x55e3221c1060 msgr2=0x55e3221c01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4f977fe640 0 client.17019 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 shutdown_connections 2024-09-12T12:26:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 1 -- 172.21.15.137:0/1805664129 wait complete. 2024-09-12T12:26:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.625+0000 7f4fc128f740 20 client.17019 trim_cache size 0 max 16384 2024-09-12T12:26:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 Processor -- start 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 -- start start 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c4af0 0x55e3221c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c5410 0x55e3221cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81b2270 con 0x55e3221c5410 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81afae0 con 0x55e3221cfcb0 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.639+0000 7f4fc128f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81ad950 con 0x55e3221c4af0 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c5410 0x55e3221cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c5410 0x55e3221cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34260/0 (socket says 172.21.15.137:34260) 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 -- 172.21.15.137:0/313940325 learned_addr learned my addr 172.21.15.137:0/313940325 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c4af0 0x55e3221c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 -- 172.21.15.137:0/313940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c4af0 msgr2=0x55e3221c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c4af0 0x55e3221c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 -- 172.21.15.137:0/313940325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 msgr2=0x55e3221e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 -- 172.21.15.137:0/313940325 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3221c0e90 con 0x55e3221c5410 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c5410 0x55e3221cd3b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cd060 tx=0x7f4fac0d1d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd2b2bb1c60ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4f9dffb640 1 -- 172.21.15.137:0/313940325 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0a0180 con 0x55e3221c5410 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4f9dffb640 1 -- 172.21.15.137:0/313940325 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fac0a4a40 con 0x55e3221c5410 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.640+0000 7f4f9dffb640 1 -- 172.21.15.137:0/313940325 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac09b480 con 0x55e3221c5410 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 -- 172.21.15.137:0/313940325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c5410 msgr2=0x55e3221cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.646 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c5410 0x55e3221cd3b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cd060 tx=0x7f4fac0d1d70 comp rx=0 tx=0).stop 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 -- 172.21.15.137:0/313940325 shutdown_connections 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221cfcb0 0x55e3221e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221c5410 0x55e3221cd3b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 --2- 172.21.15.137:0/313940325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c4af0 0x55e3221c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 -- 172.21.15.137:0/313940325 >> 172.21.15.137:0/313940325 conn(0x55e3221c1060 msgr2=0x55e3221cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 -- 172.21.15.137:0/313940325 shutdown_connections 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 -- 172.21.15.137:0/313940325 wait complete. 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 -- start start 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.641+0000 7f4fc128f740 1 Processor -- start 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c4af0 0x55e3221c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221cfcb0 0x55e3221ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81adfa0 con 0x55e3221cfcb0 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81af410 con 0x55e3221c4af0 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb81ae670 con 0x55e3221c5410 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 10 client.0 _ll_register_callbacks cb 0x7f4fa8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221cfcb0 0x55e3221ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221cfcb0 0x55e3221ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34270/0 (socket says 172.21.15.137:34270) 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 -- 172.21.15.137:0/1333996548 learned_addr learned my addr 172.21.15.137:0/1333996548 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fb77fe640 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c4af0 0x55e3221c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 -- 172.21.15.137:0/1333996548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 msgr2=0x55e3221c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 -- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c4af0 msgr2=0x55e3221c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c4af0 0x55e3221c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 -- 172.21.15.137:0/1333996548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3221bc2d0 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fbca8e640 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221cfcb0 0x55e3221ca510 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009a50 tx=0x7f4fb0009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7851b8dd193a4a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4f977fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4f977fe640 1 -- 172.21.15.137:0/1333996548 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0062060 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4f977fe640 1 -- 172.21.15.137:0/1333996548 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f4fb0004b40 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fac0a4a40 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.642+0000 7f4f977fe640 1 -- 172.21.15.137:0/1333996548 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4fb0031d90 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 10 client.17025 fetch_fsmap learned FSMap version 41 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e3221cf6c0 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4f977fe640 1 -- 172.21.15.137:0/1333996548 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fac0a4a40 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4f977fe640 1 -- 172.21.15.137:0/1333996548 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 41) ==== 8358+0+0 (secure 0 0 0) 0x7f4fb0030070 con 0x55e3221cfcb0 2024-09-12T12:26:26.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 10 client.17025 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 20 client.17025 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e32224ab90 0x55e32224cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 4 client.17025 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e322130d70 con 0x55e32224ab90 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.643+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e32224ab90 0x55e32224cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.644+0000 7f4f977fe640 10 client.17025 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:26.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.644+0000 7f4fb7fff640 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e32224ab90 0x55e32224cf80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bad0d88ae38b1 server_cookie=d3270948030befe4 in_seq=0 out_seq=0 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4f977fe640 1 -- 172.21.15.137:0/1333996548 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e322130d70 con 0x55e32224ab90 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4f977fe640 10 client.17025 handle_command_reply: tid=0 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 client.17025 shutdown 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221cfcb0 msgr2=0x55e3221ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221cfcb0 0x55e3221ca510 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009a50 tx=0x7f4fb0009540 comp rx=0 tx=0).stop 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 shutdown_connections 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e32224ab90 0x55e32224cf80 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3221cfcb0 0x55e3221ca510 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4f977fe640 0 client.17025 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e3221c5410 0x55e3221c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 --2- 172.21.15.137:0/1333996548 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e3221c4af0 0x55e3221c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 >> 172.21.15.137:0/1333996548 conn(0x55e3221c1060 msgr2=0x55e3221ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 shutdown_connections 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 1 -- 172.21.15.137:0/1333996548 wait complete. 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fc128f740 20 client.17025 trim_cache size 0 max 16384 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:26:26.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:26:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:26:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4fa00a2da0 msgr2=0x7f4fa00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4fa00a2da0 0x7f4fa00a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4fb004d000 tx=0x7f4fb004b000 comp rx=0 tx=0).stop 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 msgr2=0x7f4fb8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.693+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb8074a30 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00e9f0 tx=0x7f4fac00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 shutdown_connections 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4fa00a2da0 0x7f4fa00a5260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fb8107b80 0x7f4fb8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4fb8102480 0x7f4fb8074a30 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 --2- 172.21.15.137:0/1076726709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4fb8101ae0 0x7f4fb80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 >> 172.21.15.137:0/1076726709 conn(0x7f4fb80fd7e0 msgr2=0x7f4fb80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 shutdown_connections 2024-09-12T12:26:26.697 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:26.694+0000 7f4fbe4f1640 1 -- 172.21.15.137:0/1076726709 wait complete. 2024-09-12T12:26:26.765 INFO:tasks.workunit.client.0.smithi137.stdout: 1 cleanup 601 sec 2024-09-12T12:26:27.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:27 smithi137 ceph-mon[30950]: pgmap v417: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 427 KiB/s wr, 67 op/s 2024-09-12T12:26:27.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:27 smithi189 ceph-mon[35475]: pgmap v417: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 427 KiB/s wr, 67 op/s 2024-09-12T12:26:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:27 smithi171 ceph-mon[35098]: pgmap v417: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 427 KiB/s wr, 67 op/s 2024-09-12T12:26:27.766 INFO:tasks.workunit.client.0.smithi137.stdout: 1 cleanup 602 sec 2024-09-12T12:26:28.108 INFO:tasks.workunit.client.0.smithi137.stdout: 0 cleanup 602 sec 2024-09-12T12:26:28.108 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-09-12T12:26:28.108 INFO:tasks.workunit.client.0.smithi137.stdout: Operation Count AvgLat MaxLat 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: ---------------------------------------- 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: NTCreateX 135553 1.842 28954.084 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Close 99619 0.002 0.084 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Rename 5742 17.023 20041.713 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Unlink 27336 3.783 22734.281 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Deltree 4 83.112 149.326 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Mkdir 2 0.191 0.377 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Qpathinfo 122879 0.873 69.135 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Qfileinfo 21568 0.001 0.028 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Qfsinfo 22527 0.129 5.098 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Sfileinfo 11038 0.017 3.523 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: Find 47522 0.168 46.487 2024-09-12T12:26:28.109 INFO:tasks.workunit.client.0.smithi137.stdout: WriteX 67694 0.047 57.910 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stdout: ReadX 212674 0.011 21.565 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stdout: LockX 442 0.638 3.980 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stdout: UnlockX 442 0.592 7.923 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stdout: Flush 9491 2.621 48.190 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stdout:Throughput 7.12023 MB/sec 1 clients 1 procs max_latency=28954.092 ms 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stderr:+ dbench 10 2024-09-12T12:26:28.110 INFO:tasks.workunit.client.0.smithi137.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-12T12:26:28.111 INFO:tasks.workunit.client.0.smithi137.stdout: 2024-09-12T12:26:28.111 INFO:tasks.workunit.client.0.smithi137.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-12T12:26:28.111 INFO:tasks.workunit.client.0.smithi137.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-12T12:26:28.210 INFO:tasks.workunit.client.0.smithi137.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-12T12:26:28.210 INFO:tasks.workunit.client.0.smithi137.stdout:releasing clients 2024-09-12T12:26:28.410 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T12:26:28.410 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T12:26:28.411 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:26:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:26:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:26:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:26:28.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:26:28.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:26:28.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:26:28.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:26:28.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:26:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:26:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:26:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:26:28.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:26:28.746 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:26:29.210 INFO:tasks.workunit.client.0.smithi137.stdout: 10 339 122.39 MB/sec warmup 1 sec latency 647.318 ms 2024-09-12T12:26:29.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 -- 172.21.15.137:0/1158311480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841082a0 msgr2=0x7f498410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:29.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 --2- 172.21.15.137:0/1158311480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841082a0 0x7f498410ec40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f497c009cd0 tx=0x7f497c02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 -- 172.21.15.137:0/1158311480 shutdown_connections 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 --2- 172.21.15.137:0/1158311480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841082a0 0x7f498410ec40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 --2- 172.21.15.137:0/1158311480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49841036e0 0x7f4984107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 --2- 172.21.15.137:0/1158311480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4984102d30 0x7f4984103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 -- 172.21.15.137:0/1158311480 >> 172.21.15.137:0/1158311480 conn(0x7f49840fc7d0 msgr2=0x7f49840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 -- 172.21.15.137:0/1158311480 shutdown_connections 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 -- 172.21.15.137:0/1158311480 wait complete. 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 Processor -- start 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.312+0000 7f498b354640 1 -- start start 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f498b354640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4984102d30 0x7f4984198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f498b354640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841036e0 0x7f4984198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:29.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f498b354640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49841082a0 0x7f498419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f498b354640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49841120a0 con 0x7f49841036e0 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f498b354640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4984111f20 con 0x7f4984102d30 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f498b354640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4984112220 con 0x7f49841082a0 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49888ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841036e0 0x7f4984198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4984102d30 0x7f4984198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49898f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49841082a0 0x7f498419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4984102d30 0x7f4984198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:53156/0 (socket says 172.21.15.137:53156) 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49888ef640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841036e0 0x7f4984198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39964/0 (socket says 172.21.15.137:39964) 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 -- 172.21.15.137:0/3251234107 learned_addr learned my addr 172.21.15.137:0/3251234107 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 -- 172.21.15.137:0/3251234107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49841082a0 msgr2=0x7f498419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49841082a0 0x7f498419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 -- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841036e0 msgr2=0x7f4984198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841036e0 0x7f4984198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 -- 172.21.15.137:0/3251234107 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498419d520 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49888ef640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841036e0 0x7f4984198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f49890f0640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4984102d30 0x7f4984198570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f498000e9f0 tx=0x7f498000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f497a7fc640 1 -- 172.21.15.137:0/3251234107 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498000cd80 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.313+0000 7f497a7fc640 1 -- 172.21.15.137:0/3251234107 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4980005500 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.314+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498419d810 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.314+0000 7f497a7fc640 1 -- 172.21.15.137:0/3251234107 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980010690 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.314+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49841a50f0 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.314+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4984103110 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.316+0000 7f497a7fc640 1 -- 172.21.15.137:0/3251234107 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f49800046c0 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.316+0000 7f497a7fc640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49700a2d70 0x7f49700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.317+0000 7f49888ef640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49700a2d70 0x7f49700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.317+0000 7f497a7fc640 1 -- 172.21.15.137:0/3251234107 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f49800cf990 con 0x7f4984102d30 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.317+0000 7f49888ef640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49700a2d70 0x7f49700a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f49840fede0 tx=0x7f4974005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:29.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.320+0000 7f497a7fc640 1 -- 172.21.15.137:0/3251234107 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f49800541b0 con 0x7f4984102d30 2024-09-12T12:26:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:29 smithi137 ceph-mon[30950]: pgmap v418: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 348 KiB/s wr, 42 op/s 2024-09-12T12:26:29.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:29 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:26:29.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.555+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49841998d0 con 0x7f4984102d30 2024-09-12T12:26:29.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.555+0000 7f497a7fc640 1 -- 172.21.15.137:0/3251234107 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11938 (secure 0 0 0) 0x7f498009c5d0 con 0x7f4984102d30 2024-09-12T12:26:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:26:29.555 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":42,"btime":"2024-09-12T12:26:29:107116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:26:28.165083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:26:29.556 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 42 2024-09-12T12:26:29.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.557+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49700a2d70 msgr2=0x7f49700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:29.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.557+0000 7f498b354640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49700a2d70 0x7f49700a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f49840fede0 tx=0x7f4974005f90 comp rx=0 tx=0).stop 2024-09-12T12:26:29.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4984102d30 msgr2=0x7f4984198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:29.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4984102d30 0x7f4984198570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f498000e9f0 tx=0x7f498000edf0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 shutdown_connections 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f49700a2d70 0x7f49700a5230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f49841082a0 0x7f498419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49841036e0 0x7f4984198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 --2- 172.21.15.137:0/3251234107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4984102d30 0x7f4984198570 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 >> 172.21.15.137:0/3251234107 conn(0x7f49840fc7d0 msgr2=0x7f4984105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 shutdown_connections 2024-09-12T12:26:29.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:29.558+0000 7f498b354640 1 -- 172.21.15.137:0/3251234107 wait complete. 2024-09-12T12:26:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:29 smithi189 ceph-mon[35475]: pgmap v418: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 348 KiB/s wr, 42 op/s 2024-09-12T12:26:29.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:29 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:26:29.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:29 smithi171 ceph-mon[35098]: pgmap v418: 97 pgs: 97 active+clean; 284 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 348 KiB/s wr, 42 op/s 2024-09-12T12:26:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:29 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:26:30.210 INFO:tasks.workunit.client.0.smithi137.stdout: 10 748 99.87 MB/sec warmup 2 sec latency 1647.651 ms 2024-09-12T12:26:30.408 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T12:26:30.409 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T12:26:30.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:30 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3251234107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:26:30.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:30 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3251234107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:26:30.684 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:26:30.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:30 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3251234107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:26:31.210 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1284 80.22 MB/sec warmup 3 sec latency 2647.678 ms 2024-09-12T12:26:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.190+0000 7ff6f00a8640 1 -- 172.21.15.137:0/1886209690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8106f00 msgr2=0x7ff6e810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.190+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/1886209690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8106f00 0x7ff6e810ea80 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009900 tx=0x7ff6e002a410 comp rx=0 tx=0).stop 2024-09-12T12:26:31.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 -- 172.21.15.137:0/1886209690 shutdown_connections 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/1886209690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8106f00 0x7ff6e810ea80 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/1886209690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6e8102780 0x7ff6e81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/1886209690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e8101dd0 0x7ff6e81021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 -- 172.21.15.137:0/1886209690 >> 172.21.15.137:0/1886209690 conn(0x7ff6e80fd8b0 msgr2=0x7ff6e80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 -- 172.21.15.137:0/1886209690 shutdown_connections 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 -- 172.21.15.137:0/1886209690 wait complete. 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 Processor -- start 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 -- start start 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8101dd0 0x7ff6e81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.191+0000 7ff6f00a8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6e8102780 0x7ff6e8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6f00a8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e8106f00 0x7ff6e819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6f00a8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e8111f60 con 0x7ff6e8101dd0 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6f00a8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e8111de0 con 0x7ff6e8106f00 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6f00a8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81120e0 con 0x7ff6e8102780 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8101dd0 0x7ff6e81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8101dd0 0x7ff6e81983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39972/0 (socket says 172.21.15.137:39972) 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ed643640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6e8102780 0x7ff6e8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 -- 172.21.15.137:0/2064524719 learned_addr learned my addr 172.21.15.137:0/2064524719 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ee645640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e8106f00 0x7ff6e819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 -- 172.21.15.137:0/2064524719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6e8102780 msgr2=0x7ff6e8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6e8102780 0x7ff6e8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 -- 172.21.15.137:0/2064524719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e8106f00 msgr2=0x7ff6e819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e8106f00 0x7ff6e819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 -- 172.21.15.137:0/2064524719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e819d370 con 0x7ff6e8101dd0 2024-09-12T12:26:31.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ed643640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6e8102780 0x7ff6e8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ee645640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e8106f00 0x7ff6e819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6ede44640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8101dd0 0x7ff6e81983c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400ebc0 tx=0x7ff6e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.192+0000 7ff6deffd640 1 -- 172.21.15.137:0/2064524719 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4010630 con 0x7ff6e8101dd0 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.193+0000 7ff6deffd640 1 -- 172.21.15.137:0/2064524719 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e4010cb0 con 0x7ff6e8101dd0 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.193+0000 7ff6deffd640 1 -- 172.21.15.137:0/2064524719 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4018bd0 con 0x7ff6e8101dd0 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.193+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e819d660 con 0x7ff6e8101dd0 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.193+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e81a4f40 con 0x7ff6e8101dd0 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.195+0000 7ff6deffd640 1 -- 172.21.15.137:0/2064524719 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff6e40107d0 con 0x7ff6e8101dd0 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 Processor -- start 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 -- start start 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fd760 0x556c767fdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c76808400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c7680ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556c767632a0 con 0x556c767fe080 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c7670ab10 con 0x556c767fd760 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6f2481740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556c7675f260 con 0x556c76808940 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c76808400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6deffd640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff6d40a2d70 0x7ff6d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6ee645640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fd760 0x556c767fdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c76808400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39986/0 (socket says 172.21.15.137:39986) 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6ee645640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fd760 0x556c767fdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:53184/0 (socket says 172.21.15.137:53184) 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6ede44640 1 -- 172.21.15.137:0/2287166732 learned_addr learned my addr 172.21.15.137:0/2287166732 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.196+0000 7ff6ed643640 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c7680ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ee645640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff6d40a2d70 0x7ff6d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ed643640 1 -- 172.21.15.137:0/2287166732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fd760 msgr2=0x556c767fdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ed643640 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fd760 0x556c767fdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6deffd640 1 -- 172.21.15.137:0/2064524719 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7ff6e40cf990 con 0x7ff6e8101dd0 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ed643640 1 -- 172.21.15.137:0/2287166732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 msgr2=0x556c76808400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ed643640 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c76808400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ed643640 1 -- 172.21.15.137:0/2287166732 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c764d4b90 con 0x556c76808940 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ed643640 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c7680ad30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8005fd0 tx=0x7ff6d8010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4248014a01a38fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6ee645640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff6d40a2d70 0x7ff6d40a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0004770 tx=0x7ff6e002a410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6d3fff640 1 -- 172.21.15.137:0/2287166732 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8011700 con 0x556c76808940 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6d3fff640 1 -- 172.21.15.137:0/2287166732 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d8011d80 con 0x556c76808940 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.197+0000 7ff6d3fff640 1 -- 172.21.15.137:0/2287166732 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6d801a380 con 0x556c76808940 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 -- 172.21.15.137:0/2287166732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 msgr2=0x556c7680ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c7680ad30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8005fd0 tx=0x7ff6d8010860 comp rx=0 tx=0).stop 2024-09-12T12:26:31.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 -- 172.21.15.137:0/2287166732 shutdown_connections 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c7680ad30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c76808400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2287166732 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fd760 0x556c767fdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 -- 172.21.15.137:0/2287166732 >> 172.21.15.137:0/2287166732 conn(0x556c767f9cf0 msgr2=0x556c767fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 -- 172.21.15.137:0/2287166732 shutdown_connections 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 -- 172.21.15.137:0/2287166732 wait complete. 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.198+0000 7ff6f2481740 1 -- start start 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 1 Processor -- start 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c76805820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76805d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c76808940 0x556c768062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556c7680b510 con 0x556c76808940 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c766eb6e0 con 0x556c767fe080 2024-09-12T12:26:31.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556c7680b690 con 0x556c767fd760 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6f2481740 10 client.0 _ll_register_callbacks cb 0x556c7670fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c76808940 0x556c768062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c76805820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c76808940 0x556c768062a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40000/0 (socket says 172.21.15.137:40000) 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ed643640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76805d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c76805820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:50966/0 (socket says 172.21.15.137:50966) 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 -- 172.21.15.137:0/289830129 learned_addr learned my addr 172.21.15.137:0/289830129 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 -- 172.21.15.137:0/289830129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 msgr2=0x556c76805820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c76805820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 -- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 msgr2=0x556c76805d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76805d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ee645640 1 -- 172.21.15.137:0/289830129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c76814a00 con 0x556c76808940 2024-09-12T12:26:31.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.199+0000 7ff6ed643640 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76805d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6ee645640 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c76808940 0x556c768062a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0033000 tx=0x7ff6e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef167bdc9de56a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6d17fa640 1 -- 172.21.15.137:0/289830129 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e004f020 con 0x556c76808940 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6d17fa640 1 -- 172.21.15.137:0/289830129 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7ff6e0003a00 con 0x556c76808940 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6d8011d80 con 0x556c76808940 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6d17fa640 1 -- 172.21.15.137:0/289830129 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff6e005a050 con 0x556c76808940 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 10 client.17043 fetch_fsmap learned FSMap version 42 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556c76817390 con 0x556c76808940 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6d17fa640 1 -- 172.21.15.137:0/289830129 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6d8011d80 con 0x556c76808940 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6d17fa640 1 -- 172.21.15.137:0/289830129 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7ff6e004bbc0 con 0x556c76808940 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 10 client.17043 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 10 client.17043 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:26:31.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 20 client.17043 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c768188f0 0x556c7681ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 4 client.17043 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.200+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c767709d0 con 0x556c768188f0 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.201+0000 7ff6ede44640 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c768188f0 0x556c7681ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.201+0000 7ff6d17fa640 10 client.17043 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.201+0000 7ff6ede44640 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c768188f0 0x556c7681ad70 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e831218643a67d server_cookie=5bede42ece89677b in_seq=0 out_seq=0 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.205+0000 7ff6d17fa640 1 -- 172.21.15.137:0/289830129 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c767709d0 con 0x556c768188f0 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.205+0000 7ff6d17fa640 10 client.17043 handle_command_reply: tid=0 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.205+0000 7ff6f2481740 1 client.17043 shutdown 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c76808940 msgr2=0x556c768062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c76808940 0x556c768062a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0033000 tx=0x7ff6e0009260 comp rx=0 tx=0).stop 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 shutdown_connections 2024-09-12T12:26:31.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c768188f0 0x556c7681ad70 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c76808940 0x556c768062a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76805d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6d17fa640 0 client.17043 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 --2- 172.21.15.137:0/289830129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c76805820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 >> 172.21.15.137:0/289830129 conn(0x556c767f9cf0 msgr2=0x556c767ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 shutdown_connections 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 1 -- 172.21.15.137:0/289830129 wait complete. 2024-09-12T12:26:31.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.206+0000 7ff6f2481740 20 client.17043 trim_cache size 0 max 16384 2024-09-12T12:26:31.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.223+0000 7ff6f2481740 1 Processor -- start 2024-09-12T12:26:31.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6f2481740 1 -- start start 2024-09-12T12:26:31.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fd760 0x556c767ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76820640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c76820b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6f2481740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81b01f0 con 0x556c767fd760 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6f2481740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81ada30 con 0x556c767fe080 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6f2481740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81ab8a0 con 0x556c76808940 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fd760 0x556c767ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fd760 0x556c767ff910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40002/0 (socket says 172.21.15.137:40002) 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 -- 172.21.15.137:0/2042207396 learned_addr learned my addr 172.21.15.137:0/2042207396 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ed643640 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76820640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ee645640 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c76820b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 -- 172.21.15.137:0/2042207396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 msgr2=0x556c76820b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c76820b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 -- 172.21.15.137:0/2042207396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 msgr2=0x556c76820640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76820640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 -- 172.21.15.137:0/2042207396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c767f9b20 con 0x556c767fd760 2024-09-12T12:26:31.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.224+0000 7ff6ede44640 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fd760 0x556c767ff910 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009a80 tx=0x7ff6e40d1ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575198977ead0275 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:31.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.225+0000 7ff6d37fe640 1 -- 172.21.15.137:0/2042207396 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4010e60 con 0x556c767fd760 2024-09-12T12:26:31.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.225+0000 7ff6d37fe640 1 -- 172.21.15.137:0/2042207396 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e4010ac0 con 0x556c767fd760 2024-09-12T12:26:31.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.225+0000 7ff6d37fe640 1 -- 172.21.15.137:0/2042207396 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e409b3d0 con 0x556c767fd760 2024-09-12T12:26:31.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.225+0000 7ff6f2481740 1 -- 172.21.15.137:0/2042207396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fd760 msgr2=0x556c767ff910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:31.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fd760 0x556c767ff910 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009a80 tx=0x7ff6e40d1ea0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 -- 172.21.15.137:0/2042207396 shutdown_connections 2024-09-12T12:26:31.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c76808940 0x556c76820b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c767fe080 0x556c76820640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 --2- 172.21.15.137:0/2042207396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fd760 0x556c767ff910 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 -- 172.21.15.137:0/2042207396 >> 172.21.15.137:0/2042207396 conn(0x556c767f9cf0 msgr2=0x556c767ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 -- 172.21.15.137:0/2042207396 shutdown_connections 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 -- 172.21.15.137:0/2042207396 wait complete. 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 -- start start 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.226+0000 7ff6f2481740 1 Processor -- start 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c7680cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c7680d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6f2481740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c76808940 0x556c76886a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c7680cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ed643640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c7680d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ede44640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c7680cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:50978/0 (socket says 172.21.15.137:50978) 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ed643640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c7680d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40014/0 (socket says 172.21.15.137:40014) 2024-09-12T12:26:31.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ede44640 1 -- 172.21.15.137:0/4149902996 learned_addr learned my addr 172.21.15.137:0/4149902996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81abef0 con 0x556c767fe080 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81ad360 con 0x556c76808940 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e81ac5c0 con 0x556c767fd760 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ee645640 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c76808940 0x556c76886a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6f2481740 10 client.0 _ll_register_callbacks cb 0x7ff6e0031ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ede44640 1 -- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c76808940 msgr2=0x556c76886a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ede44640 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c76808940 0x556c76886a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.227+0000 7ff6ede44640 1 -- 172.21.15.137:0/4149902996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 msgr2=0x556c7680d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6ede44640 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c7680d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6ede44640 1 -- 172.21.15.137:0/4149902996 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c767f5260 con 0x556c767fd760 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6ed643640 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c7680d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6ede44640 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c7680cd00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40a4610 tx=0x7ff6e409bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6469755cfd8c3320 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:31.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6d17fa640 1 -- 172.21.15.137:0/4149902996 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4010e60 con 0x556c767fd760 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6d17fa640 1 -- 172.21.15.137:0/4149902996 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7ff6e40cd380 con 0x556c767fd760 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e4010ac0 con 0x556c767fd760 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6d17fa640 1 -- 172.21.15.137:0/4149902996 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff6e40985f0 con 0x556c767fd760 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6f2481740 10 client.26428 fetch_fsmap learned FSMap version 42 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556c76887f90 con 0x556c767fd760 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.228+0000 7ff6d17fa640 1 -- 172.21.15.137:0/4149902996 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e4010ac0 con 0x556c767fd760 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6d17fa640 1 -- 172.21.15.137:0/4149902996 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7ff6e40ba070 con 0x556c767fd760 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6f2481740 10 client.26428 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6f2481740 10 client.26428 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6f2481740 20 client.26428 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6f2481740 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c76888750 0x556c7688ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:31.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6f2481740 4 client.26428 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T12:26:31.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x556c7676ad20 con 0x556c76888750 2024-09-12T12:26:31.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6ed643640 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c76888750 0x556c7688ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:31.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6d17fa640 10 client.26428 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:31.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:31.229+0000 7ff6ed643640 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c76888750 0x556c7688ab90 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b58ea9467fc391 server_cookie=4674aeeb933e6171 in_seq=0 out_seq=0 2024-09-12T12:26:31.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:31 smithi137 ceph-mon[30950]: pgmap v419: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 6.9 MiB/s wr, 101 op/s 2024-09-12T12:26:31.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:31 smithi189 ceph-mon[35475]: pgmap v419: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 6.9 MiB/s wr, 101 op/s 2024-09-12T12:26:31.600 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:31 smithi171 ceph-mon[35098]: pgmap v419: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 6.9 MiB/s wr, 101 op/s 2024-09-12T12:26:32.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6d17fa640 1 -- 172.21.15.137:0/4149902996 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556c7676ad20 con 0x556c76888750 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6d17fa640 10 client.26428 handle_command_reply: tid=0 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 client.26428 shutdown 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 msgr2=0x556c7680cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c7680cd00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40a4610 tx=0x7ff6e409bf80 comp rx=0 tx=0).stop 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 shutdown_connections 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c76808940 0x556c76886a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556c76888750 0x556c7688ab90 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c767fe080 0x556c7680d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 --2- 172.21.15.137:0/4149902996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c767fd760 0x556c7680cd00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6d17fa640 0 client.26428 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.054+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 >> 172.21.15.137:0/4149902996 conn(0x556c767f9cf0 msgr2=0x556c767ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:32.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 shutdown_connections 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f2481740 1 -- 172.21.15.137:0/4149902996 wait complete. 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f2481740 20 client.26428 trim_cache size 0 max 16384 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 42, 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 50, 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "5f89434e": { 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 50, 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:26:32.056 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 150.0, 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:26:32.057 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff6d40a2d70 msgr2=0x7ff6d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff6d40a2d70 0x7ff6d40a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0004770 tx=0x7ff6e002a410 comp rx=0 tx=0).stop 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8101dd0 msgr2=0x7ff6e81983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8101dd0 0x7ff6e81983c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400ebc0 tx=0x7ff6e400c6a0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 shutdown_connections 2024-09-12T12:26:32.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff6d40a2d70 0x7ff6d40a5230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e8106f00 0x7ff6e819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6e8102780 0x7ff6e8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 --2- 172.21.15.137:0/2064524719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6e8101dd0 0x7ff6e81983c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:32.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 >> 172.21.15.137:0/2064524719 conn(0x7ff6e80fd8b0 msgr2=0x7ff6e80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:32.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.055+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 shutdown_connections 2024-09-12T12:26:32.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:32.056+0000 7ff6f00a8640 1 -- 172.21.15.137:0/2064524719 wait complete. 2024-09-12T12:26:32.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 60.19 MB/sec warmup 4 sec latency 3647.706 ms 2024-09-12T12:26:32.768 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5f89434e', quiesce took 4.4 seconds. Will release after: 25.6 2024-09-12T12:26:33.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 48.15 MB/sec warmup 5 sec latency 4647.773 ms 2024-09-12T12:26:33.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:33 smithi137 ceph-mon[30950]: pgmap v420: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 6.7 MiB/s wr, 70 op/s 2024-09-12T12:26:33.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:33 smithi189 ceph-mon[35475]: pgmap v420: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 6.7 MiB/s wr, 70 op/s 2024-09-12T12:26:33.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:33 smithi171 ceph-mon[35098]: pgmap v420: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 6.7 MiB/s wr, 70 op/s 2024-09-12T12:26:34.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 40.13 MB/sec warmup 6 sec latency 5647.889 ms 2024-09-12T12:26:35.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 34.40 MB/sec warmup 7 sec latency 6647.972 ms 2024-09-12T12:26:35.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:35 smithi137 ceph-mon[30950]: pgmap v421: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 6.7 MiB/s wr, 70 op/s 2024-09-12T12:26:35.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:35 smithi189 ceph-mon[35475]: pgmap v421: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 6.7 MiB/s wr, 70 op/s 2024-09-12T12:26:35.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:35 smithi171 ceph-mon[35098]: pgmap v421: 97 pgs: 97 active+clean; 353 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 6.7 MiB/s wr, 70 op/s 2024-09-12T12:26:36.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 30.10 MB/sec warmup 8 sec latency 7648.063 ms 2024-09-12T12:26:37.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 26.75 MB/sec warmup 9 sec latency 8648.166 ms 2024-09-12T12:26:37.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:37 smithi137 ceph-mon[30950]: pgmap v422: 97 pgs: 97 active+clean; 360 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 7.4 MiB/s wr, 167 op/s 2024-09-12T12:26:37.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:37 smithi189 ceph-mon[35475]: pgmap v422: 97 pgs: 97 active+clean; 360 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 7.4 MiB/s wr, 167 op/s 2024-09-12T12:26:37.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:37 smithi171 ceph-mon[35098]: pgmap v422: 97 pgs: 97 active+clean; 360 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 7.4 MiB/s wr, 167 op/s 2024-09-12T12:26:38.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 24.08 MB/sec warmup 10 sec latency 9648.307 ms 2024-09-12T12:26:39.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 21.89 MB/sec warmup 11 sec latency 10648.392 ms 2024-09-12T12:26:39.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:39 smithi137 ceph-mon[30950]: pgmap v423: 97 pgs: 97 active+clean; 360 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 7.4 MiB/s wr, 155 op/s 2024-09-12T12:26:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:39 smithi189 ceph-mon[35475]: pgmap v423: 97 pgs: 97 active+clean; 360 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 7.4 MiB/s wr, 155 op/s 2024-09-12T12:26:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:39 smithi171 ceph-mon[35098]: pgmap v423: 97 pgs: 97 active+clean; 360 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 7.4 MiB/s wr, 155 op/s 2024-09-12T12:26:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 -- 172.21.15.137:0/4179844493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1069c0 msgr2=0x7f5e3c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 --2- 172.21.15.137:0/4179844493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1069c0 0x7f5e3c106da0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009cd0 tx=0x7f5e2c02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 -- 172.21.15.137:0/4179844493 shutdown_connections 2024-09-12T12:26:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 --2- 172.21.15.137:0/4179844493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e3c101350 0x7f5e3c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 --2- 172.21.15.137:0/4179844493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3c1009b0 0x7f5e3c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 --2- 172.21.15.137:0/4179844493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1069c0 0x7f5e3c106da0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 -- 172.21.15.137:0/4179844493 >> 172.21.15.137:0/4179844493 conn(0x7f5e3c077990 msgr2=0x7f5e3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 -- 172.21.15.137:0/4179844493 shutdown_connections 2024-09-12T12:26:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.791+0000 7f5e418d0640 1 -- 172.21.15.137:0/4179844493 wait complete. 2024-09-12T12:26:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 Processor -- start 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 -- start start 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1009b0 0x7f5e3c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e3c101350 0x7f5e3c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3c1069c0 0x7f5e3c1a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c110e00 con 0x7f5e3c1009b0 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c110c80 con 0x7f5e3c101350 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e418d0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c110f80 con 0x7f5e3c1069c0 2024-09-12T12:26:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e3affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1009b0 0x7f5e3c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e3affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1009b0 0x7f5e3c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39376/0 (socket says 172.21.15.137:39376) 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e3affd640 1 -- 172.21.15.137:0/1328666238 learned_addr learned my addr 172.21.15.137:0/1328666238 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3c1069c0 0x7f5e3c1a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.792+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e3c101350 0x7f5e3c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e3affd640 1 -- 172.21.15.137:0/1328666238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3c1069c0 msgr2=0x7f5e3c1a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e3affd640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3c1069c0 0x7f5e3c1a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e3affd640 1 -- 172.21.15.137:0/1328666238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e3c101350 msgr2=0x7f5e3c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e3affd640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e3c101350 0x7f5e3c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e3affd640 1 -- 172.21.15.137:0/1328666238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3c1a47b0 con 0x7f5e3c1009b0 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e3c101350 0x7f5e3c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e3affd640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1009b0 0x7f5e3c1991f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c005e30 tx=0x7f5e2c032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e408ce640 1 -- 172.21.15.137:0/1328666238 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c004040 con 0x7f5e3c1009b0 2024-09-12T12:26:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e408ce640 1 -- 172.21.15.137:0/1328666238 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e2c0046c0 con 0x7f5e3c1009b0 2024-09-12T12:26:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3c1a4980 con 0x7f5e3c1009b0 2024-09-12T12:26:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e408ce640 1 -- 172.21.15.137:0/1328666238 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c046480 con 0x7f5e3c1009b0 2024-09-12T12:26:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.793+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3c1a4e40 con 0x7f5e3c1009b0 2024-09-12T12:26:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.795+0000 7f5e408ce640 1 -- 172.21.15.137:0/1328666238 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5e2c0305a0 con 0x7f5e3c1009b0 2024-09-12T12:26:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 Processor -- start 2024-09-12T12:26:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 -- start start 2024-09-12T12:26:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb682812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb6828bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb6828e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eb681e12f0 con 0x55eb68281820 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eb68189270 con 0x55eb68280f00 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e4466a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb681dd2b0 con 0x55eb6828c0c0 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb6828bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb6828bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39378/0 (socket says 172.21.15.137:39378) 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 -- 172.21.15.137:0/1197003803 learned_addr learned my addr 172.21.15.137:0/1197003803 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e408ce640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5e280a2da0 0x7f5e280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb682812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb6828e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5e280a2da0 0x7f5e280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e408ce640 1 -- 172.21.15.137:0/1328666238 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f5e2c02e030 con 0x7f5e3c1009b0 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 -- 172.21.15.137:0/1197003803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 msgr2=0x55eb6828e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb6828e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 -- 172.21.15.137:0/1197003803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 msgr2=0x55eb682812e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb682812e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.796+0000 7f5e3affd640 1 -- 172.21.15.137:0/1197003803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb68188340 con 0x55eb68281820 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.797+0000 7f5e3affd640 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb6828bb80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0bcad0 tx=0x7f5e2c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f5ee923c2fbd4e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.797+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5e280a2da0 0x7f5e280a5260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5e34005f60 tx=0x7f5e34005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.797+0000 7f5e31ffb640 1 -- 172.21.15.137:0/1197003803 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0bc910 con 0x55eb68281820 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.797+0000 7f5e31ffb640 1 -- 172.21.15.137:0/1197003803 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e2c036070 con 0x55eb68281820 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.797+0000 7f5e31ffb640 1 -- 172.21.15.137:0/1197003803 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0bc730 con 0x55eb68281820 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 -- 172.21.15.137:0/1197003803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 msgr2=0x55eb6828bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb6828bb80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0bcad0 tx=0x7f5e2c02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 -- 172.21.15.137:0/1197003803 shutdown_connections 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb6828e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb6828bb80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 --2- 172.21.15.137:0/1197003803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb682812e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 -- 172.21.15.137:0/1197003803 >> 172.21.15.137:0/1197003803 conn(0x55eb6827d470 msgr2=0x55eb6827d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 -- 172.21.15.137:0/1197003803 shutdown_connections 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 -- 172.21.15.137:0/1197003803 wait complete. 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 -- start start 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.798+0000 7f5e4466a740 1 Processor -- start 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68288f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb68289490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb6828c0c0 0x55eb682899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55eb6828ea50 con 0x55eb68281820 2024-09-12T12:26:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eb6816a290 con 0x55eb6828c0c0 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55eb6828ebd0 con 0x55eb68280f00 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 10 client.0 _ll_register_callbacks cb 0x55eb6819b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb68289490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb68289490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39392/0 (socket says 172.21.15.137:39392) 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2029925728 learned_addr learned my addr 172.21.15.137:0/2029925728 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3affd640 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68288f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb6828c0c0 0x55eb682899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2029925728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 msgr2=0x55eb68288f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68288f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb6828c0c0 msgr2=0x55eb682899d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb6828c0c0 0x55eb682899d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2029925728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb67ef4210 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb68289490 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5e24010780 tx=0x7f5e24010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68aed8acbb6c9afc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e1affd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e1affd640 1 -- 172.21.15.137:0/2029925728 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2401d060 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e1affd640 1 -- 172.21.15.137:0/2029925728 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f5e2400eda0 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.799+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e2c036070 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e1affd640 1 -- 172.21.15.137:0/2029925728 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5e24018430 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 10 client.17061 fetch_fsmap learned FSMap version 42 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55eb67fc0620 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e1affd640 1 -- 172.21.15.137:0/2029925728 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e2c036070 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e1affd640 1 -- 172.21.15.137:0/2029925728 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f5e2400d530 con 0x55eb68281820 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 10 client.17061 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 20 client.17061 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb6829be60 0x55eb6829e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 4 client.17061 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb681eea20 con 0x55eb6829be60 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.800+0000 7f5e3affd640 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb6829be60 0x55eb6829e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.801+0000 7f5e1affd640 10 client.17061 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.801+0000 7f5e3affd640 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb6829be60 0x55eb6829e270 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af2dc94f1144ef4 server_cookie=af72f6750b5bddfa in_seq=0 out_seq=0 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.801+0000 7f5e1affd640 1 -- 172.21.15.137:0/2029925728 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb681eea20 con 0x55eb6829be60 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.801+0000 7f5e1affd640 10 client.17061 handle_command_reply: tid=0 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.801+0000 7f5e4466a740 1 client.17061 shutdown 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 msgr2=0x55eb68289490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb68289490 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5e24010780 tx=0x7f5e24010c40 comp rx=0 tx=0).stop 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 shutdown_connections 2024-09-12T12:26:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb6828c0c0 0x55eb682899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb6829be60 0x55eb6829e270 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb68289490 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2029925728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68288f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e1affd640 0 client.17061 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 >> 172.21.15.137:0/2029925728 conn(0x55eb6827d470 msgr2=0x55eb682831d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 shutdown_connections 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 1 -- 172.21.15.137:0/2029925728 wait complete. 2024-09-12T12:26:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.802+0000 7f5e4466a740 20 client.17061 trim_cache size 0 max 16384 2024-09-12T12:26:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 Processor -- start 2024-09-12T12:26:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 -- start start 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb68289800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb682a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb682a4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1addc0 con 0x55eb68281820 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1ab600 con 0x55eb68280f00 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e4466a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1a9470 con 0x55eb6828c0c0 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e3a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb682a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e3a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb682a3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39408/0 (socket says 172.21.15.137:39408) 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2585131043 learned_addr learned my addr 172.21.15.137:0/2585131043 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e3affd640 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb68289800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.816+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb682a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2585131043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 msgr2=0x55eb682a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb682a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2585131043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 msgr2=0x55eb68289800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb68289800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e3a7fc640 1 -- 172.21.15.137:0/2585131043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb6827d2a0 con 0x55eb68281820 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb682a3b40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5e24010470 tx=0x7f5e24004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c24293a638ea1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e317fa640 1 -- 172.21.15.137:0/2585131043 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400ddc0 con 0x55eb68281820 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e317fa640 1 -- 172.21.15.137:0/2585131043 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e24010020 con 0x55eb68281820 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e317fa640 1 -- 172.21.15.137:0/2585131043 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24005330 con 0x55eb68281820 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e4466a740 1 -- 172.21.15.137:0/2585131043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 msgr2=0x55eb682a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.817+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb682a3b40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5e24010470 tx=0x7f5e24004660 comp rx=0 tx=0).stop 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- 172.21.15.137:0/2585131043 shutdown_connections 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb6828c0c0 0x55eb682a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb68281820 0x55eb682a3b40 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2585131043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68280f00 0x55eb68289800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- 172.21.15.137:0/2585131043 >> 172.21.15.137:0/2585131043 conn(0x55eb6827d470 msgr2=0x55eb6828afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- 172.21.15.137:0/2585131043 shutdown_connections 2024-09-12T12:26:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- 172.21.15.137:0/2585131043 wait complete. 2024-09-12T12:26:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- start start 2024-09-12T12:26:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 Processor -- start 2024-09-12T12:26:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68285f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68281820 0x55eb68286480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb6828c0c0 0x55eb682869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1a9ac0 con 0x55eb6828c0c0 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1aaf30 con 0x55eb68281820 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.818+0000 7f5e4466a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3c1aa190 con 0x55eb68280f00 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e4466a740 10 client.0 _ll_register_callbacks cb 0x7f5e2400b630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb6828c0c0 0x55eb682869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb6828c0c0 0x55eb682869c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39412/0 (socket says 172.21.15.137:39412) 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 -- 172.21.15.137:0/2996988086 learned_addr learned my addr 172.21.15.137:0/2996988086 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68281820 0x55eb68286480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3affd640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68285f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 -- 172.21.15.137:0/2996988086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 msgr2=0x55eb68285f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68285f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 -- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68281820 msgr2=0x55eb68286480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68281820 0x55eb68286480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 -- 172.21.15.137:0/2996988086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb68305b10 con 0x55eb6828c0c0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3a7fc640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68281820 0x55eb68286480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e3b7fe640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb6828c0c0 0x55eb682869c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5e340041f0 tx=0x7f5e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ca28acbf41a79b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e1affd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e1affd640 1 -- 172.21.15.137:0/2996988086 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400fe60 con 0x55eb6828c0c0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e1affd640 1 -- 172.21.15.137:0/2996988086 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f5e34005020 con 0x55eb6828c0c0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e24010020 con 0x55eb6828c0c0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e1affd640 1 -- 172.21.15.137:0/2996988086 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5e34005220 con 0x55eb6828c0c0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e4466a740 10 client.17067 fetch_fsmap learned FSMap version 42 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.819+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55eb68306e00 con 0x55eb6828c0c0 2024-09-12T12:26:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e1affd640 1 -- 172.21.15.137:0/2996988086 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e24010020 con 0x55eb6828c0c0 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e1affd640 1 -- 172.21.15.137:0/2996988086 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f5e3401d040 con 0x55eb6828c0c0 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e4466a740 10 client.17067 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e4466a740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e4466a740 20 client.17067 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb68307530 0x55eb68309950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e4466a740 4 client.17067 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb681e8d70 con 0x55eb68307530 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.820+0000 7f5e3affd640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb68307530 0x55eb68309950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.821+0000 7f5e1affd640 10 client.17067 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:39.821+0000 7f5e3affd640 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb68307530 0x55eb68309950 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b7a3fa4801b3aa server_cookie=cb1d34aea869e6c6 in_seq=0 out_seq=0 2024-09-12T12:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e1affd640 1 -- 172.21.15.137:0/2996988086 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb681e8d70 con 0x55eb68307530 2024-09-12T12:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e1affd640 10 client.17067 handle_command_reply: tid=0 2024-09-12T12:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 client.17067 shutdown 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb6828c0c0 msgr2=0x55eb682869c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb6828c0c0 0x55eb682869c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5e340041f0 tx=0x7f5e340046c0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 shutdown_connections 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55eb68307530 0x55eb68309950 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb6828c0c0 0x55eb682869c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eb68281820 0x55eb68286480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 --2- 172.21.15.137:0/2996988086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb68280f00 0x55eb68285f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e1affd640 0 client.17067 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 >> 172.21.15.137:0/2996988086 conn(0x55eb6827d470 msgr2=0x55eb6828a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 shutdown_connections 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 1 -- 172.21.15.137:0/2996988086 wait complete. 2024-09-12T12:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.093+0000 7f5e4466a740 20 client.17067 trim_cache size 0 max 16384 2024-09-12T12:26:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:26:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:26:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:26:40.095 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5e280a2da0 msgr2=0x7f5e280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5e280a2da0 0x7f5e280a5260 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5e34005f60 tx=0x7f5e34005f90 comp rx=0 tx=0).stop 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1009b0 msgr2=0x7f5e3c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1009b0 0x7f5e3c1991f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c005e30 tx=0x7f5e2c032fb0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 shutdown_connections 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5e280a2da0 0x7f5e280a5260 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3c1069c0 0x7f5e3c1a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e3c101350 0x7f5e3c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 --2- 172.21.15.137:0/1328666238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e3c1009b0 0x7f5e3c1991f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:40.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 >> 172.21.15.137:0/1328666238 conn(0x7f5e3c077990 msgr2=0x7f5e3c103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 shutdown_connections 2024-09-12T12:26:40.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:40.094+0000 7f5e418d0640 1 -- 172.21.15.137:0/1328666238 wait complete. 2024-09-12T12:26:40.211 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 20.06 MB/sec warmup 12 sec latency 11648.519 ms 2024-09-12T12:26:41.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 18.52 MB/sec warmup 13 sec latency 12648.621 ms 2024-09-12T12:26:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:41 smithi137 ceph-mon[30950]: pgmap v424: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 7.4 MiB/s wr, 161 op/s 2024-09-12T12:26:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:41 smithi189 ceph-mon[35475]: pgmap v424: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 7.4 MiB/s wr, 161 op/s 2024-09-12T12:26:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:41 smithi171 ceph-mon[35098]: pgmap v424: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 7.4 MiB/s wr, 161 op/s 2024-09-12T12:26:42.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 17.20 MB/sec warmup 14 sec latency 13648.710 ms 2024-09-12T12:26:43.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 16.05 MB/sec warmup 15 sec latency 14648.754 ms 2024-09-12T12:26:43.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:43 smithi137 ceph-mon[30950]: pgmap v425: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 818 KiB/s wr, 102 op/s 2024-09-12T12:26:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:43 smithi189 ceph-mon[35475]: pgmap v425: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 818 KiB/s wr, 102 op/s 2024-09-12T12:26:43.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:43 smithi171 ceph-mon[35098]: pgmap v425: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 818 KiB/s wr, 102 op/s 2024-09-12T12:26:44.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 15.05 MB/sec warmup 16 sec latency 15649.083 ms 2024-09-12T12:26:45.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 14.16 MB/sec warmup 17 sec latency 16649.188 ms 2024-09-12T12:26:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:45 smithi137 ceph-mon[30950]: pgmap v426: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 818 KiB/s wr, 102 op/s 2024-09-12T12:26:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:45 smithi189 ceph-mon[35475]: pgmap v426: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 818 KiB/s wr, 102 op/s 2024-09-12T12:26:45.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:45 smithi171 ceph-mon[35098]: pgmap v426: 97 pgs: 97 active+clean; 360 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 818 KiB/s wr, 102 op/s 2024-09-12T12:26:46.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 13.38 MB/sec warmup 18 sec latency 17649.310 ms 2024-09-12T12:26:47.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 12.67 MB/sec warmup 19 sec latency 18649.407 ms 2024-09-12T12:26:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:47 smithi137 ceph-mon[30950]: pgmap v427: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 885 KiB/s wr, 258 op/s 2024-09-12T12:26:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:47 smithi189 ceph-mon[35475]: pgmap v427: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 885 KiB/s wr, 258 op/s 2024-09-12T12:26:47.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:47 smithi171 ceph-mon[35098]: pgmap v427: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 885 KiB/s wr, 258 op/s 2024-09-12T12:26:48.212 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 12.04 MB/sec warmup 20 sec latency 19649.497 ms 2024-09-12T12:26:49.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 11.47 MB/sec warmup 21 sec latency 20649.594 ms 2024-09-12T12:26:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:49 smithi137 ceph-mon[30950]: pgmap v428: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 84 KiB/s wr, 160 op/s 2024-09-12T12:26:49.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:49 smithi189 ceph-mon[35475]: pgmap v428: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 84 KiB/s wr, 160 op/s 2024-09-12T12:26:49.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:49 smithi171 ceph-mon[35098]: pgmap v428: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 84 KiB/s wr, 160 op/s 2024-09-12T12:26:50.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 10.94 MB/sec warmup 22 sec latency 21649.697 ms 2024-09-12T12:26:51.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 10.47 MB/sec warmup 23 sec latency 22649.804 ms 2024-09-12T12:26:51.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:51 smithi137 ceph-mon[30950]: pgmap v429: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 88 KiB/s wr, 163 op/s 2024-09-12T12:26:51.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:51 smithi189 ceph-mon[35475]: pgmap v429: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 88 KiB/s wr, 163 op/s 2024-09-12T12:26:51.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:51 smithi171 ceph-mon[35098]: pgmap v429: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 88 KiB/s wr, 163 op/s 2024-09-12T12:26:52.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 10.03 MB/sec warmup 24 sec latency 23649.912 ms 2024-09-12T12:26:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 -- 172.21.15.137:0/329739568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 msgr2=0x7fde70106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 --2- 172.21.15.137:0/329739568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde70106da0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fde64009930 tx=0x7fde6402a420 comp rx=0 tx=0).stop 2024-09-12T12:26:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 -- 172.21.15.137:0/329739568 shutdown_connections 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 --2- 172.21.15.137:0/329739568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde70101350 0x7fde7010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 --2- 172.21.15.137:0/329739568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde701009b0 0x7fde70100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 --2- 172.21.15.137:0/329739568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde70106da0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 -- 172.21.15.137:0/329739568 >> 172.21.15.137:0/329739568 conn(0x7fde70077990 msgr2=0x7fde700fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 -- 172.21.15.137:0/329739568 shutdown_connections 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.185+0000 7fde77e30640 1 -- 172.21.15.137:0/329739568 wait complete. 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 Processor -- start 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 -- start start 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde701009b0 0x7fde701983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde70101350 0x7fde701988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde7019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde70111ef0 con 0x7fde701069c0 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde70111d70 con 0x7fde701009b0 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde77e30640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde70112070 con 0x7fde70101350 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde763cd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde7019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde763cd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde7019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48186/0 (socket says 172.21.15.137:48186) 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde763cd640 1 -- 172.21.15.137:0/2019958678 learned_addr learned my addr 172.21.15.137:0/2019958678 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde753cb640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde70101350 0x7fde701988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.186+0000 7fde75bcc640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde701009b0 0x7fde701983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde763cd640 1 -- 172.21.15.137:0/2019958678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde70101350 msgr2=0x7fde701988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde763cd640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde70101350 0x7fde701988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde763cd640 1 -- 172.21.15.137:0/2019958678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde701009b0 msgr2=0x7fde701983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde763cd640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde701009b0 0x7fde701983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde763cd640 1 -- 172.21.15.137:0/2019958678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7019d390 con 0x7fde701069c0 2024-09-12T12:26:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde763cd640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde7019cc60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fde6800eaa0 tx=0x7fde6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde62ffd640 1 -- 172.21.15.137:0/2019958678 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6800cdb0 con 0x7fde701069c0 2024-09-12T12:26:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde62ffd640 1 -- 172.21.15.137:0/2019958678 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde68005020 con 0x7fde701069c0 2024-09-12T12:26:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7019d680 con 0x7fde701069c0 2024-09-12T12:26:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde62ffd640 1 -- 172.21.15.137:0/2019958678 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde680116b0 con 0x7fde701069c0 2024-09-12T12:26:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.187+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde701a4f10 con 0x7fde701069c0 2024-09-12T12:26:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.189+0000 7fde62ffd640 1 -- 172.21.15.137:0/2019958678 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fde68005560 con 0x7fde701069c0 2024-09-12T12:26:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 Processor -- start 2024-09-12T12:26:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 -- start start 2024-09-12T12:26:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe47db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5616dfdac2f0 con 0x5616dfe570c0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616dfd54270 con 0x5616dfe482f0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde7abc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5616dfda82b0 con 0x5616dfe479d0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde62ffd640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fde5c0a2da0 0x7fde5c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48198/0 (socket says 172.21.15.137:48198) 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde753cb640 1 -- 172.21.15.137:0/3947144608 learned_addr learned my addr 172.21.15.137:0/3947144608 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde75bcc640 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe56b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde753cb640 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe47db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde763cd640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fde5c0a2da0 0x7fde5c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.190+0000 7fde62ffd640 1 -- 172.21.15.137:0/2019958678 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fde680cf640 con 0x7fde701069c0 2024-09-12T12:26:53.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde753cb640 1 -- 172.21.15.137:0/3947144608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 msgr2=0x5616dfe47db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde753cb640 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe47db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde753cb640 1 -- 172.21.15.137:0/3947144608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 msgr2=0x5616dfe56b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde753cb640 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe56b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde753cb640 1 -- 172.21.15.137:0/3947144608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616dfd53340 con 0x5616dfe570c0 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde75bcc640 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde753cb640 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe594b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011840 tx=0x7fde6c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858be7edb221e7ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde763cd640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fde5c0a2da0 0x7fde5c0a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde680a42c0 tx=0x7fde68091010 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde5bfff640 1 -- 172.21.15.137:0/3947144608 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c004030 con 0x5616dfe570c0 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde5bfff640 1 -- 172.21.15.137:0/3947144608 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c00fca0 con 0x5616dfe570c0 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.191+0000 7fde5bfff640 1 -- 172.21.15.137:0/3947144608 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0038e0 con 0x5616dfe570c0 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 -- 172.21.15.137:0/3947144608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 msgr2=0x5616dfe594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.200 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe594b0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011840 tx=0x7fde6c0097d0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 -- 172.21.15.137:0/3947144608 shutdown_connections 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe594b0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe56b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 --2- 172.21.15.137:0/3947144608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe47db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 -- 172.21.15.137:0/3947144608 >> 172.21.15.137:0/3947144608 conn(0x5616dfe44060 msgr2=0x5616dfe44440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 -- 172.21.15.137:0/3947144608 shutdown_connections 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 -- 172.21.15.137:0/3947144608 wait complete. 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 -- start start 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.192+0000 7fde7abc8740 1 Processor -- start 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe54080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.201 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe54b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5616dfe59cf0 con 0x5616dfe570c0 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616dfd35290 con 0x5616dfe482f0 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5616dfe59e70 con 0x5616dfe479d0 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 10 client.0 _ll_register_callbacks cb 0x5616dfd66140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe54b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe54b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48214/0 (socket says 172.21.15.137:48214) 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 -- 172.21.15.137:0/501191657 learned_addr learned my addr 172.21.15.137:0/501191657 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe54080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde75bcc640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 -- 172.21.15.137:0/501191657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 msgr2=0x5616dfe54080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe54080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 -- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 msgr2=0x5616dfe545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 -- 172.21.15.137:0/501191657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616dfabf210 con 0x5616dfe570c0 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde75bcc640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde753cb640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe54b00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011810 tx=0x7fde6c004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b42fd5f290ba15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde597fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde597fa640 1 -- 172.21.15.137:0/501191657 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c02c060 con 0x5616dfe570c0 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde597fa640 1 -- 172.21.15.137:0/501191657 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fde6c01f600 con 0x5616dfe570c0 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.193+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c00fca0 con 0x5616dfe570c0 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde597fa640 1 -- 172.21.15.137:0/501191657 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde6c00fe50 con 0x5616dfe570c0 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 10 client.17079 fetch_fsmap learned FSMap version 42 2024-09-12T12:26:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5616dfb8b620 con 0x5616dfe570c0 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde597fa640 1 -- 172.21.15.137:0/501191657 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c00fca0 con 0x5616dfe570c0 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde597fa640 1 -- 172.21.15.137:0/501191657 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fde6c01ea30 con 0x5616dfe570c0 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 10 client.17079 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 20 client.17079 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfe66fc0 0x5616dfe693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 4 client.17079 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616dfdb9a20 con 0x5616dfe66fc0 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.194+0000 7fde75bcc640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfe66fc0 0x5616dfe693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.195+0000 7fde597fa640 10 client.17079 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.195+0000 7fde75bcc640 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfe66fc0 0x5616dfe693b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e340ee83ead5748 server_cookie=59efb8b8aad37354 in_seq=0 out_seq=0 2024-09-12T12:26:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde597fa640 1 -- 172.21.15.137:0/501191657 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616dfdb9a20 con 0x5616dfe66fc0 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde597fa640 10 client.17079 handle_command_reply: tid=0 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 client.17079 shutdown 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 msgr2=0x5616dfe54b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe54b00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fde6c011810 tx=0x7fde6c004560 comp rx=0 tx=0).stop 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 shutdown_connections 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfe66fc0 0x5616dfe693b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe54b00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde597fa640 0 client.17079 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 --2- 172.21.15.137:0/501191657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe479d0 0x5616dfe54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 >> 172.21.15.137:0/501191657 conn(0x5616dfe44060 msgr2=0x5616dfe49dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:53.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 shutdown_connections 2024-09-12T12:26:53.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 1 -- 172.21.15.137:0/501191657 wait complete. 2024-09-12T12:26:53.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.196+0000 7fde7abc8740 20 client.17079 trim_cache size 0 max 16384 2024-09-12T12:26:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 Processor -- start 2024-09-12T12:26:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 -- start start 2024-09-12T12:26:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe479d0 0x5616dfe6eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe482f0 0x5616dfe6f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde701adf30 con 0x5616dfe570c0 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde701ab770 con 0x5616dfe479d0 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.210+0000 7fde7abc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde701a95e0 con 0x5616dfe482f0 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48224/0 (socket says 172.21.15.137:48224) 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 -- 172.21.15.137:0/2828910414 learned_addr learned my addr 172.21.15.137:0/2828910414 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe479d0 0x5616dfe6eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde75bcc640 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe482f0 0x5616dfe6f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 -- 172.21.15.137:0/2828910414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe482f0 msgr2=0x5616dfe6f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe482f0 0x5616dfe6f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 -- 172.21.15.137:0/2828910414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe479d0 msgr2=0x5616dfe6eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe479d0 0x5616dfe6eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 -- 172.21.15.137:0/2828910414 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616dfe597f0 con 0x5616dfe570c0 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde75bcc640 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe482f0 0x5616dfe6f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde753cb640 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe512c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0056c0 tx=0x7fde6c032040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa70642be7f9eae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde5b7fe640 1 -- 172.21.15.137:0/2828910414 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c02c040 con 0x5616dfe570c0 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde5b7fe640 1 -- 172.21.15.137:0/2828910414 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c00fca0 con 0x5616dfe570c0 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.211+0000 7fde5b7fe640 1 -- 172.21.15.137:0/2828910414 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00e980 con 0x5616dfe570c0 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- 172.21.15.137:0/2828910414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 msgr2=0x5616dfe512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe512c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0056c0 tx=0x7fde6c032040 comp rx=0 tx=0).stop 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- 172.21.15.137:0/2828910414 shutdown_connections 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe570c0 0x5616dfe512c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe482f0 0x5616dfe6f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 --2- 172.21.15.137:0/2828910414 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe479d0 0x5616dfe6eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- 172.21.15.137:0/2828910414 >> 172.21.15.137:0/2828910414 conn(0x5616dfe44060 msgr2=0x5616dfe54860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- 172.21.15.137:0/2828910414 shutdown_connections 2024-09-12T12:26:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- 172.21.15.137:0/2828910414 wait complete. 2024-09-12T12:26:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- start start 2024-09-12T12:26:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 Processor -- start 2024-09-12T12:26:53.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 9.63 MB/sec warmup 25 sec latency 24649.958 ms 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe479d0 0x5616dfe5f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe5f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe570c0 0x5616dfe5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fde701a9c30 con 0x5616dfe479d0 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde701ab0a0 con 0x5616dfe482f0 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.212+0000 7fde7abc8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde701aa300 con 0x5616dfe570c0 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde7abc8740 10 client.0 _ll_register_callbacks cb 0x7fde6c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe479d0 0x5616dfe5f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe479d0 0x5616dfe5f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48226/0 (socket says 172.21.15.137:48226) 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 -- 172.21.15.137:0/543056376 learned_addr learned my addr 172.21.15.137:0/543056376 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde75bcc640 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe5f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 -- 172.21.15.137:0/543056376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe570c0 msgr2=0x5616dfe5a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe570c0 0x5616dfe5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 -- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 msgr2=0x5616dfe5f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe5f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 -- 172.21.15.137:0/543056376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616dfe5b080 con 0x5616dfe479d0 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde75bcc640 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe5f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde753cb640 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe479d0 0x5616dfe5f3e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009cd0 tx=0x7fde6c005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efff970c2a3095c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde597fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:26:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde597fa640 1 -- 172.21.15.137:0/543056376 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c02c040 con 0x5616dfe479d0 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde597fa640 1 -- 172.21.15.137:0/543056376 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fde6c005300 con 0x5616dfe479d0 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c00fca0 con 0x5616dfe479d0 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde597fa640 1 -- 172.21.15.137:0/543056376 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde6c011020 con 0x5616dfe479d0 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde7abc8740 10 client.17085 fetch_fsmap learned FSMap version 42 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5616dfed16c0 con 0x5616dfe479d0 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.213+0000 7fde597fa640 1 -- 172.21.15.137:0/543056376 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c00fca0 con 0x5616dfe479d0 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde597fa640 1 -- 172.21.15.137:0/543056376 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fde6c040030 con 0x5616dfe479d0 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde7abc8740 10 client.17085 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde7abc8740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde7abc8740 20 client.17085 populate_metadata read hostname 'smithi137' 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde7abc8740 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfed1d10 0x5616dfed4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde7abc8740 4 client.17085 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:26:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616dfdb3d70 con 0x5616dfed1d10 2024-09-12T12:26:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde75bcc640 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfed1d10 0x5616dfed4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde597fa640 10 client.17085 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.214+0000 7fde75bcc640 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfed1d10 0x5616dfed4130 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e713c5d6e9f7f450 server_cookie=bdf31336cb1b82ee in_seq=0 out_seq=0 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde597fa640 1 -- 172.21.15.137:0/543056376 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616dfdb3d70 con 0x5616dfed1d10 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde597fa640 10 client.17085 handle_command_reply: tid=0 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 client.17085 shutdown 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe479d0 msgr2=0x5616dfe5f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe479d0 0x5616dfe5f3e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009cd0 tx=0x7fde6c005a70 comp rx=0 tx=0).stop 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 shutdown_connections 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5616dfe570c0 0x5616dfe5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616dfe482f0 0x5616dfe5f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5616dfed1d10 0x5616dfed4130 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 --2- 172.21.15.137:0/543056376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5616dfe479d0 0x5616dfe5f3e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 >> 172.21.15.137:0/543056376 conn(0x5616dfe44060 msgr2=0x5616dfe49580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde597fa640 0 client.17085 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 shutdown_connections 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 1 -- 172.21.15.137:0/543056376 wait complete. 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.238+0000 7fde7abc8740 20 client.17085 trim_cache size 0 max 16384 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:26:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:26:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:26:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:26:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fde5c0a2da0 msgr2=0x7fde5c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fde5c0a2da0 0x7fde5c0a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde680a42c0 tx=0x7fde68091010 comp rx=0 tx=0).stop 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 msgr2=0x7fde7019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde7019cc60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fde6800eaa0 tx=0x7fde6800ef60 comp rx=0 tx=0).stop 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 shutdown_connections 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fde5c0a2da0 0x7fde5c0a5260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde701069c0 0x7fde7019cc60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde70101350 0x7fde701988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 --2- 172.21.15.137:0/2019958678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde701009b0 0x7fde701983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 >> 172.21.15.137:0/2019958678 conn(0x7fde70077990 msgr2=0x7fde70103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 shutdown_connections 2024-09-12T12:26:53.242 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:26:53.239+0000 7fde77e30640 1 -- 172.21.15.137:0/2019958678 wait complete. 2024-09-12T12:26:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:53 smithi137 ceph-mon[30950]: pgmap v430: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 72 KiB/s wr, 158 op/s 2024-09-12T12:26:53.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:53 smithi189 ceph-mon[35475]: pgmap v430: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 72 KiB/s wr, 158 op/s 2024-09-12T12:26:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:53 smithi171 ceph-mon[35098]: pgmap v430: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 72 KiB/s wr, 158 op/s 2024-09-12T12:26:54.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 9.26 MB/sec warmup 26 sec latency 25650.003 ms 2024-09-12T12:26:55.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 8.92 MB/sec warmup 27 sec latency 26650.102 ms 2024-09-12T12:26:55.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:55 smithi137 ceph-mon[30950]: pgmap v431: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 72 KiB/s wr, 158 op/s 2024-09-12T12:26:55.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:55 smithi189 ceph-mon[35475]: pgmap v431: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 72 KiB/s wr, 158 op/s 2024-09-12T12:26:55.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:55 smithi171 ceph-mon[35098]: pgmap v431: 97 pgs: 97 active+clean; 356 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 72 KiB/s wr, 158 op/s 2024-09-12T12:26:56.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 8.60 MB/sec warmup 28 sec latency 27650.240 ms 2024-09-12T12:26:57.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 8.30 MB/sec warmup 29 sec latency 28650.349 ms 2024-09-12T12:26:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:57 smithi137 ceph-mon[30950]: pgmap v432: 97 pgs: 97 active+clean; 473 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.8 MiB/s wr, 209 op/s 2024-09-12T12:26:57.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:57 smithi189 ceph-mon[35475]: pgmap v432: 97 pgs: 97 active+clean; 473 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.8 MiB/s wr, 209 op/s 2024-09-12T12:26:57.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:57 smithi171 ceph-mon[35098]: pgmap v432: 97 pgs: 97 active+clean; 473 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.8 MiB/s wr, 209 op/s 2024-09-12T12:26:58.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 8.03 MB/sec warmup 30 sec latency 29650.456 ms 2024-09-12T12:26:58.370 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:26:58.634 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:26:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 -- 172.21.15.137:0/3855178026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c10c170 msgr2=0x7f4f1c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 --2- 172.21.15.137:0/3855178026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c10c170 0x7f4f1c10e610 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009ca0 tx=0x7f4f1402a950 comp rx=0 tx=0).stop 2024-09-12T12:26:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 -- 172.21.15.137:0/3855178026 shutdown_connections 2024-09-12T12:26:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 --2- 172.21.15.137:0/3855178026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c10c170 0x7f4f1c10e610 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 --2- 172.21.15.137:0/3855178026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f1c103660 0x7f4f1c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 --2- 172.21.15.137:0/3855178026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f1c102d40 0x7f4f1c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 -- 172.21.15.137:0/3855178026 >> 172.21.15.137:0/3855178026 conn(0x7f4f1c0fc7a0 msgr2=0x7f4f1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 -- 172.21.15.137:0/3855178026 shutdown_connections 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 -- 172.21.15.137:0/3855178026 wait complete. 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 Processor -- start 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.167+0000 7f4f22478640 1 -- start start 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c102d40 0x7f4f1c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f1c103660 0x7f4f1c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f1c10c170 0x7f4f1c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c111f30 con 0x7f4f1c102d40 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c111db0 con 0x7f4f1c10c170 2024-09-12T12:26:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c1120b0 con 0x7f4f1c103660 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c102d40 0x7f4f1c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c102d40 0x7f4f1c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56860/0 (socket says 172.21.15.137:56860) 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 -- 172.21.15.137:0/2980016498 learned_addr learned my addr 172.21.15.137:0/2980016498 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1b7fe640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f1c103660 0x7f4f1c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f20a15640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f1c10c170 0x7f4f1c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 -- 172.21.15.137:0/2980016498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f1c103660 msgr2=0x7f4f1c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f1c103660 0x7f4f1c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 -- 172.21.15.137:0/2980016498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f1c10c170 msgr2=0x7f4f1c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f1c10c170 0x7f4f1c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 -- 172.21.15.137:0/2980016498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c19d300 con 0x7f4f1c102d40 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f1bfff640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c102d40 0x7f4f1c198350 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c920 tx=0x7f4f0c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f197fa640 1 -- 172.21.15.137:0/2980016498 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c007c70 con 0x7f4f1c102d40 2024-09-12T12:26:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f197fa640 1 -- 172.21.15.137:0/2980016498 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0c007e10 con 0x7f4f1c102d40 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f197fa640 1 -- 172.21.15.137:0/2980016498 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c018890 con 0x7f4f1c102d40 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c19d5f0 con 0x7f4f1c102d40 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.168+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a4ed0 con 0x7f4f1c102d40 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.169+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c103120 con 0x7f4f1c102d40 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.170+0000 7f4f197fa640 1 -- 172.21.15.137:0/2980016498 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4f0c018a70 con 0x7f4f1c102d40 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.171+0000 7f4f197fa640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4f040a2cf0 0x7f4f040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.171+0000 7f4f1b7fe640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4f040a2cf0 0x7f4f040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.172+0000 7f4f197fa640 1 -- 172.21.15.137:0/2980016498 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f4f0c0cf580 con 0x7f4f1c102d40 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.172+0000 7f4f1b7fe640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4f040a2cf0 0x7f4f040a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0fecc0 tx=0x7f4f1000b2e0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:26:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.176+0000 7f4f197fa640 1 -- 172.21.15.137:0/2980016498 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f0c014030 con 0x7f4f1c102d40 2024-09-12T12:26:59.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 7.77 MB/sec warmup 31 sec latency 30650.504 ms 2024-09-12T12:26:59.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.378+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1c1996b0 con 0x7f4f1c102d40 2024-09-12T12:26:59.379 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:26:59 smithi137 ceph-mon[30950]: pgmap v433: 97 pgs: 97 active+clean; 473 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.8 MiB/s wr, 54 op/s 2024-09-12T12:26:59.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.379+0000 7f4f197fa640 1 -- 172.21.15.137:0/2980016498 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11938 (secure 0 0 0) 0x7f4f0c09c1c0 con 0x7f4f1c102d40 2024-09-12T12:26:59.380 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:26:59.380 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":42,"btime":"2024-09-12T12:26:29:107116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:26:28.165083+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:26:59.380 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 42 2024-09-12T12:26:59.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4f040a2cf0 msgr2=0x7f4f040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4f040a2cf0 0x7f4f040a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0fecc0 tx=0x7f4f1000b2e0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c102d40 msgr2=0x7f4f1c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c102d40 0x7f4f1c198350 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00c920 tx=0x7f4f0c00cde0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 shutdown_connections 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4f040a2cf0 0x7f4f040a51b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f1c10c170 0x7f4f1c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f1c103660 0x7f4f1c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 --2- 172.21.15.137:0/2980016498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f1c102d40 0x7f4f1c198350 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 >> 172.21.15.137:0/2980016498 conn(0x7f4f1c0fc7a0 msgr2=0x7f4f1c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 shutdown_connections 2024-09-12T12:26:59.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:26:59.381+0000 7f4f22478640 1 -- 172.21.15.137:0/2980016498 wait complete. 2024-09-12T12:26:59.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:26:59 smithi189 ceph-mon[35475]: pgmap v433: 97 pgs: 97 active+clean; 473 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.8 MiB/s wr, 54 op/s 2024-09-12T12:26:59.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:26:59 smithi171 ceph-mon[35098]: pgmap v433: 97 pgs: 97 active+clean; 473 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.8 MiB/s wr, 54 op/s 2024-09-12T12:27:00.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1324 7.52 MB/sec warmup 32 sec latency 31650.601 ms 2024-09-12T12:27:00.336 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 5f89434e --release --await' 2024-09-12T12:27:00.336 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 5f89434e --release --await 2024-09-12T12:27:00.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:00 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2980016498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:00.604 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:00 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2980016498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:00.604 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:27:00.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:00 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2980016498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 -- 172.21.15.137:0/3623473728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 msgr2=0x7fc6681021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 --2- 172.21.15.137:0/3623473728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 0x7fc6681021b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009900 tx=0x7fc65c02a3f0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 -- 172.21.15.137:0/3623473728 shutdown_connections 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 --2- 172.21.15.137:0/3623473728 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc668106f00 0x7fc66810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 --2- 172.21.15.137:0/3623473728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc668102780 0x7fc6681067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 --2- 172.21.15.137:0/3623473728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 0x7fc6681021b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 -- 172.21.15.137:0/3623473728 >> 172.21.15.137:0/3623473728 conn(0x7fc6680fd8b0 msgr2=0x7fc6680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 -- 172.21.15.137:0/3623473728 shutdown_connections 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.113+0000 7fc66ee64640 1 -- 172.21.15.137:0/3623473728 wait complete. 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 Processor -- start 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 -- start start 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 0x7fc668198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc668102780 0x7fc668198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc668106f00 0x7fc66819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc668111f60 con 0x7fc668106f00 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc668111de0 con 0x7fc668102780 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.114+0000 7fc66ee64640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6681120e0 con 0x7fc668101dd0 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 0x7fc668198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66d401640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc668106f00 0x7fc66819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 0x7fc668198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:37498/0 (socket says 172.21.15.137:37498) 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66cc00640 1 -- 172.21.15.137:0/2956222473 learned_addr learned my addr 172.21.15.137:0/2956222473 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc667fff640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc668102780 0x7fc668198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66d401640 1 -- 172.21.15.137:0/2956222473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 msgr2=0x7fc668198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66d401640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 0x7fc668198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66d401640 1 -- 172.21.15.137:0/2956222473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc668102780 msgr2=0x7fc668198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66d401640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc668102780 0x7fc668198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66d401640 1 -- 172.21.15.137:0/2956222473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66819d3d0 con 0x7fc668106f00 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66d401640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc668106f00 0x7fc66819ccf0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc660009fd0 tx=0x7fc66000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc665ffb640 1 -- 172.21.15.137:0/2956222473 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66000ce20 con 0x7fc668106f00 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66819d6c0 con 0x7fc668106f00 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc665ffb640 1 -- 172.21.15.137:0/2956222473 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc660005020 con 0x7fc668106f00 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc665ffb640 1 -- 172.21.15.137:0/2956222473 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc660010720 con 0x7fc668106f00 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.115+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6681a4fa0 con 0x7fc668106f00 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.117+0000 7fc665ffb640 1 -- 172.21.15.137:0/2956222473 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc660005540 con 0x7fc668106f00 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 Processor -- start 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 -- start start 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc665ffb640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6580a2cf0 0x7fc6580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592e9420620 con 0x5592e94bb060 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5592e93c7e50 con 0x5592e94c6240 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc67123d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5592e941c5e0 con 0x5592e94bb980 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc665ffb640 1 -- 172.21.15.137:0/2956222473 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fc6600cfbb0 con 0x7fc668106f00 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94bb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56892/0 (socket says 172.21.15.137:56892) 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc66cc00640 1 -- 172.21.15.137:0/1680502033 learned_addr learned my addr 172.21.15.137:0/1680502033 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc667fff640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6580a2cf0 0x7fc6580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.118+0000 7fc66d401640 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc66cc00640 1 -- 172.21.15.137:0/1680502033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 msgr2=0x5592e94c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc66cc00640 1 -- 172.21.15.137:0/1680502033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 msgr2=0x5592e94c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc66cc00640 1 -- 172.21.15.137:0/1680502033 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e93b8d40 con 0x5592e94bb060 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc66d401640 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94bb440 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009a20 tx=0x7fc65c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75467ae0f7426894 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc667fff640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6580a2cf0 0x7fc6580a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5592e94b8450 tx=0x7fc654009f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc652ffd640 1 -- 172.21.15.137:0/1680502033 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c040070 con 0x5592e94bb060 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc652ffd640 1 -- 172.21.15.137:0/1680502033 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc65c004030 con 0x5592e94bb060 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.119+0000 7fc652ffd640 1 -- 172.21.15.137:0/1680502033 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c035040 con 0x5592e94bb060 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 -- 172.21.15.137:0/1680502033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 msgr2=0x5592e94bb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94bb440 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009a20 tx=0x7fc65c005dd0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 -- 172.21.15.137:0/1680502033 shutdown_connections 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 --2- 172.21.15.137:0/1680502033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94bb440 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 -- 172.21.15.137:0/1680502033 >> 172.21.15.137:0/1680502033 conn(0x5592e94b75f0 msgr2=0x5592e94b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 -- 172.21.15.137:0/1680502033 shutdown_connections 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 -- 172.21.15.137:0/1680502033 wait complete. 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 -- start start 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.120+0000 7fc67123d740 1 Processor -- start 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94c3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5592e94c8e10 con 0x5592e94bb060 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5592e93a8f10 con 0x5592e94c6240 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5592e94c8f90 con 0x5592e94bb980 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 10 client.0 _ll_register_callbacks cb 0x5592e93ccdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:01.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56900/0 (socket says 172.21.15.137:56900) 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 -- 172.21.15.137:0/1107884347 learned_addr learned my addr 172.21.15.137:0/1107884347 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc667fff640 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 -- 172.21.15.137:0/1107884347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 msgr2=0x5592e94c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 -- 172.21.15.137:0/1107884347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 msgr2=0x5592e94c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 -- 172.21.15.137:0/1107884347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e912f580 con 0x5592e94bb060 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94c3160 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02a430 tx=0x7fc65c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b86f5a964dcf3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc647fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc647fff640 1 -- 172.21.15.137:0/1107884347 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c040070 con 0x5592e94bb060 2024-09-12T12:27:01.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc647fff640 1 -- 172.21.15.137:0/1107884347 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fc65c0315b0 con 0x5592e94bb060 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.121+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc65c004030 con 0x5592e94bb060 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc647fff640 1 -- 172.21.15.137:0/1107884347 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc65c035040 con 0x5592e94bb060 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 10 client.17103 fetch_fsmap learned FSMap version 42 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5592e91f9960 con 0x5592e94bb060 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc647fff640 1 -- 172.21.15.137:0/1107884347 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc65c004030 con 0x5592e94bb060 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc647fff640 1 -- 172.21.15.137:0/1107884347 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fc65c055020 con 0x5592e94bb060 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 10 client.17103 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 10 client.17103 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 20 client.17103 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e94d5be0 0x5592e94d7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 4 client.17103 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592e942db90 con 0x5592e94d5be0 2024-09-12T12:27:01.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.122+0000 7fc667fff640 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e94d5be0 0x5592e94d7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.123+0000 7fc667fff640 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e94d5be0 0x5592e94d7fd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb07b3f3e153a3c0 server_cookie=2aad1762e90327f in_seq=0 out_seq=0 2024-09-12T12:27:01.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.123+0000 7fc647fff640 10 client.17103 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.123+0000 7fc647fff640 1 -- 172.21.15.137:0/1107884347 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592e942db90 con 0x5592e94d5be0 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.123+0000 7fc647fff640 10 client.17103 handle_command_reply: tid=0 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.123+0000 7fc67123d740 1 client.17103 shutdown 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.123+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 msgr2=0x5592e94c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.123+0000 7fc67123d740 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94c3160 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc65c02a430 tx=0x7fc65c009680 comp rx=0 tx=0).stop 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 shutdown_connections 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e94d5be0 0x5592e94d7fd0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 --2- 172.21.15.137:0/1107884347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94c3160 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 >> 172.21.15.137:0/1107884347 conn(0x5592e94b75f0 msgr2=0x5592e94bd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:01.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc647fff640 0 client.17103 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:01.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 shutdown_connections 2024-09-12T12:27:01.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 1 -- 172.21.15.137:0/1107884347 wait complete. 2024-09-12T12:27:01.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.124+0000 7fc67123d740 20 client.17103 trim_cache size 0 max 16384 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 Processor -- start 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 -- start start 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94bb060 0x5592e94bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb980 0x5592e94b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94c6240 0x5592e94dd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc6681b00b0 con 0x5592e94bb980 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc6681ad8f0 con 0x5592e94bb060 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.140+0000 7fc67123d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6681ab760 con 0x5592e94c6240 2024-09-12T12:27:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb980 0x5592e94b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb980 0x5592e94b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56910/0 (socket says 172.21.15.137:56910) 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 -- 172.21.15.137:0/1509580132 learned_addr learned my addr 172.21.15.137:0/1509580132 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94bb060 0x5592e94bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc66d401640 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94c6240 0x5592e94dd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 -- 172.21.15.137:0/1509580132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94c6240 msgr2=0x5592e94dd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94c6240 0x5592e94dd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 -- 172.21.15.137:0/1509580132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94bb060 msgr2=0x5592e94bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94bb060 0x5592e94bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 -- 172.21.15.137:0/1509580132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592e94c5ad0 con 0x5592e94bb980 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc667fff640 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb980 0x5592e94b6640 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5592e94c29f0 tx=0x7fc654014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6296575cee2b439b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:01.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc6527fc640 1 -- 172.21.15.137:0/1509580132 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65402d070 con 0x5592e94bb980 2024-09-12T12:27:01.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc6527fc640 1 -- 172.21.15.137:0/1509580132 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc654012d30 con 0x5592e94bb980 2024-09-12T12:27:01.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.141+0000 7fc6527fc640 1 -- 172.21.15.137:0/1509580132 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6540275f0 con 0x5592e94bb980 2024-09-12T12:27:01.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 -- 172.21.15.137:0/1509580132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb980 msgr2=0x5592e94b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb980 0x5592e94b6640 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x5592e94c29f0 tx=0x7fc654014000 comp rx=0 tx=0).stop 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 -- 172.21.15.137:0/1509580132 shutdown_connections 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94c6240 0x5592e94dd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb980 0x5592e94b6640 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 --2- 172.21.15.137:0/1509580132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94bb060 0x5592e94bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 -- 172.21.15.137:0/1509580132 >> 172.21.15.137:0/1509580132 conn(0x5592e94b75f0 msgr2=0x5592e94b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 -- 172.21.15.137:0/1509580132 shutdown_connections 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 -- 172.21.15.137:0/1509580132 wait complete. 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 -- start start 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.142+0000 7fc67123d740 1 Processor -- start 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc67123d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc67123d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc67123d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc67123d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc6681abdb0 con 0x5592e94bb060 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc67123d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc6681ad220 con 0x5592e94c6240 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc67123d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc6681ac480 con 0x5592e94bb980 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc67123d740 10 client.0 _ll_register_callbacks cb 0x7fc65c009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56924/0 (socket says 172.21.15.137:56924) 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 -- 172.21.15.137:0/1848691535 learned_addr learned my addr 172.21.15.137:0/1848691535 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc667fff640 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66d401640 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 -- 172.21.15.137:0/1848691535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 msgr2=0x5592e94ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 -- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 msgr2=0x5592e94ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 -- 172.21.15.137:0/1848691535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592e94caf10 con 0x5592e94bb060 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc66cc00640 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94ce2a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009a20 tx=0x7fc65c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11343940b29079b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc647fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.143+0000 7fc647fff640 1 -- 172.21.15.137:0/1848691535 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65c040070 con 0x5592e94bb060 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc647fff640 1 -- 172.21.15.137:0/1848691535 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fc65c0319c0 con 0x5592e94bb060 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc654012d30 con 0x5592e94bb060 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc647fff640 1 -- 172.21.15.137:0/1848691535 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc65c005d50 con 0x5592e94bb060 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 10 client.17109 fetch_fsmap learned FSMap version 42 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5592e95410e0 con 0x5592e94bb060 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc647fff640 1 -- 172.21.15.137:0/1848691535 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc654012d30 con 0x5592e94bb060 2024-09-12T12:27:01.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc647fff640 1 -- 172.21.15.137:0/1848691535 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fc65c055050 con 0x5592e94bb060 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 10 client.17109 fetch_fsmap finished waiting for FSMap version 42 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 10 client.17109 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 20 client.17109 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e95417b0 0x5592e9543bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 4 client.17109 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5f89434e", "release": true, "await": true}] 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5f89434e", "release": true, "await": true}) -- 0x5592e94280a0 con 0x5592e95417b0 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.144+0000 7fc667fff640 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e95417b0 0x5592e9543bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.145+0000 7fc647fff640 10 client.17109 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:01.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.145+0000 7fc667fff640 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e95417b0 0x5592e9543bd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ab52dc46f030cc server_cookie=89945c26b00caf4b in_seq=0 out_seq=0 2024-09-12T12:27:01.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc647fff640 1 -- 172.21.15.137:0/1848691535 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5592e94280a0 con 0x5592e95417b0 2024-09-12T12:27:01.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc647fff640 10 client.17109 handle_command_reply: tid=0 2024-09-12T12:27:01.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 client.17109 shutdown 2024-09-12T12:27:01.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 msgr2=0x5592e94ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94ce2a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009a20 tx=0x7fc65c005760 comp rx=0 tx=0).stop 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 shutdown_connections 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5592e94c6240 0x5592e94ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5592e94bb980 0x5592e94ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5592e95417b0 0x5592e9543bd0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 --2- 172.21.15.137:0/1848691535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5592e94bb060 0x5592e94ce2a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc647fff640 0 client.17109 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 >> 172.21.15.137:0/1848691535 conn(0x5592e94b75f0 msgr2=0x5592e94c3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 shutdown_connections 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 1 -- 172.21.15.137:0/1848691535 wait complete. 2024-09-12T12:27:01.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.201+0000 7fc67123d740 20 client.17109 trim_cache size 0 max 16384 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 42, 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 52, 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout: "5f89434e": { 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 52, 2024-09-12T12:27:01.204 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 150.0, 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:01.205 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:01.206 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:01.206 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:01.206 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:01.206 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:27:01.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6580a2cf0 msgr2=0x7fc6580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6580a2cf0 0x7fc6580a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5592e94b8450 tx=0x7fc654009f90 comp rx=0 tx=0).stop 2024-09-12T12:27:01.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc668106f00 msgr2=0x7fc66819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc668106f00 0x7fc66819ccf0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc660009fd0 tx=0x7fc66000ef10 comp rx=0 tx=0).stop 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 shutdown_connections 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc6580a2cf0 0x7fc6580a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc668106f00 0x7fc66819ccf0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc668102780 0x7fc668198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 --2- 172.21.15.137:0/2956222473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc668101dd0 0x7fc668198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 >> 172.21.15.137:0/2956222473 conn(0x7fc6680fd8b0 msgr2=0x7fc6680fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 shutdown_connections 2024-09-12T12:27:01.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:01.202+0000 7fc66ee64640 1 -- 172.21.15.137:0/2956222473 wait complete. 2024-09-12T12:27:01.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1325 7.30 MB/sec warmup 33 sec latency 32650.646 ms 2024-09-12T12:27:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:01 smithi137 ceph-mon[30950]: pgmap v434: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.9 MiB/s wr, 63 op/s 2024-09-12T12:27:01.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:01 smithi189 ceph-mon[35475]: pgmap v434: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.9 MiB/s wr, 63 op/s 2024-09-12T12:27:01.600 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:01 smithi171 ceph-mon[35098]: pgmap v434: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.9 MiB/s wr, 63 op/s 2024-09-12T12:27:02.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 1999 7.38 MB/sec warmup 34 sec latency 32671.976 ms 2024-09-12T12:27:02.240 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5f89434e', total seconds elapsed: 33.8 2024-09-12T12:27:02.240 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-12T12:27:02.240 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:27:02.639 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:27:03.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 2261 7.67 MB/sec warmup 35 sec latency 146.870 ms 2024-09-12T12:27:03.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.232+0000 7fa877fff640 1 -- 172.21.15.137:0/2412145646 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87002f300 con 0x7fa880102570 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 -- 172.21.15.137:0/2412145646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 msgr2=0x7fa8801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 --2- 172.21.15.137:0/2412145646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 0x7fa8801029b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa870009860 tx=0x7fa87002a3d0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 -- 172.21.15.137:0/2412145646 shutdown_connections 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 --2- 172.21.15.137:0/2412145646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa880102ef0 0x7fa88010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 --2- 172.21.15.137:0/2412145646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 0x7fa8801029b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 --2- 172.21.15.137:0/2412145646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 0x7fa880101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 -- 172.21.15.137:0/2412145646 >> 172.21.15.137:0/2412145646 conn(0x7fa8800fc7d0 msgr2=0x7fa8800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 -- 172.21.15.137:0/2412145646 shutdown_connections 2024-09-12T12:27:03.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.233+0000 7fa8875c0640 1 -- 172.21.15.137:0/2412145646 wait complete. 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.234+0000 7fa8875c0640 1 Processor -- start 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.234+0000 7fa8875c0640 1 -- start start 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa8875c0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 0x7fa880197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa8875c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 0x7fa880197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa88535c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 0x7fa880197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa88535c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 0x7fa880197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:37558/0 (socket says 172.21.15.137:37558) 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa884b5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 0x7fa880197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa884b5b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 0x7fa880197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56950/0 (socket says 172.21.15.137:56950) 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa88535c640 1 -- 172.21.15.137:0/3780345594 learned_addr learned my addr 172.21.15.137:0/3780345594 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa8875c0640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa880102ef0 0x7fa88019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa880110fa0 con 0x7fa880102570 2024-09-12T12:27:03.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa880110e20 con 0x7fa880102ef0 2024-09-12T12:27:03.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa880111120 con 0x7fa880100e20 2024-09-12T12:27:03.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa885b5d640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa880102ef0 0x7fa88019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa88535c640 1 -- 172.21.15.137:0/3780345594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa880102ef0 msgr2=0x7fa88019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa88535c640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa880102ef0 0x7fa88019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa88535c640 1 -- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 msgr2=0x7fa880197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.235+0000 7fa88535c640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 0x7fa880197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.236+0000 7fa88535c640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88019c3b0 con 0x7fa880100e20 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.236+0000 7fa88535c640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 0x7fa880197400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e9c0 tx=0x7fa87c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.236+0000 7fa8767fc640 1 -- 172.21.15.137:0/3780345594 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00ccc0 con 0x7fa880100e20 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.236+0000 7fa8767fc640 1 -- 172.21.15.137:0/3780345594 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87c005500 con 0x7fa880100e20 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.236+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88019c6a0 con 0x7fa880100e20 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.236+0000 7fa8767fc640 1 -- 172.21.15.137:0/3780345594 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c010430 con 0x7fa880100e20 2024-09-12T12:27:03.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.236+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa880069f70 con 0x7fa880100e20 2024-09-12T12:27:03.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.238+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8801021b0 con 0x7fa880100e20 2024-09-12T12:27:03.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.239+0000 7fa8767fc640 1 -- 172.21.15.137:0/3780345594 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa87c0046c0 con 0x7fa880100e20 2024-09-12T12:27:03.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.240+0000 7fa8767fc640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa86c0a2d70 0x7fa86c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:03.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.240+0000 7fa884b5b640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa86c0a2d70 0x7fa86c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:03.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.240+0000 7fa8767fc640 1 -- 172.21.15.137:0/3780345594 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8528+0+0 (secure 0 0 0) 0x7fa87c0cf760 con 0x7fa880100e20 2024-09-12T12:27:03.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.241+0000 7fa884b5b640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa86c0a2d70 0x7fa86c0a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa8800ffda0 tx=0x7fa870005c90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:03.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.247+0000 7fa8767fc640 1 -- 172.21.15.137:0/3780345594 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa87c09c380 con 0x7fa880100e20 2024-09-12T12:27:03.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.466+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa88006a990 con 0x7fa880100e20 2024-09-12T12:27:03.468 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:03 smithi137 ceph-mon[30950]: pgmap v435: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.9 MiB/s wr, 60 op/s 2024-09-12T12:27:03.468 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:03 smithi137 ceph-mon[30950]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T12:27:03.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.467+0000 7fa8767fc640 1 -- 172.21.15.137:0/3780345594 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11942 (secure 0 0 0) 0x7fa87c09bc80 con 0x7fa880100e20 2024-09-12T12:27:03.468 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:27:03.469 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":43,"btime":"2024-09-12T12:27:03:300053+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:27:03.165607+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25922":{"gid":25922,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/630156999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":630156999},{"type":"v1","addr":"172.21.15.189:6835","nonce":630156999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:27:03.469 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 43 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa86c0a2d70 msgr2=0x7fa86c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa86c0a2d70 0x7fa86c0a5230 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa8800ffda0 tx=0x7fa870005c90 comp rx=0 tx=0).stop 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 msgr2=0x7fa880197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 0x7fa880197400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00e9c0 tx=0x7fa87c00ee80 comp rx=0 tx=0).stop 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 shutdown_connections 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa86c0a2d70 0x7fa86c0a5230 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa880102ef0 0x7fa88019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa880102570 0x7fa880197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 --2- 172.21.15.137:0/3780345594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa880100e20 0x7fa880197400 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 >> 172.21.15.137:0/3780345594 conn(0x7fa8800fc7d0 msgr2=0x7fa8800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 shutdown_connections 2024-09-12T12:27:03.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:03.470+0000 7fa8875c0640 1 -- 172.21.15.137:0/3780345594 wait complete. 2024-09-12T12:27:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:03 smithi189 ceph-mon[35475]: pgmap v435: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.9 MiB/s wr, 60 op/s 2024-09-12T12:27:03.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:03 smithi189 ceph-mon[35475]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T12:27:03.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:03 smithi171 ceph-mon[35098]: pgmap v435: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 9.9 MiB/s wr, 60 op/s 2024-09-12T12:27:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:03 smithi171 ceph-mon[35098]: osdmap e104: 12 total, 12 up, 12 in 2024-09-12T12:27:04.209 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-12T12:27:04.210 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-12T12:27:04.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 2791 8.88 MB/sec warmup 36 sec latency 112.868 ms 2024-09-12T12:27:04.478 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:27:04.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:04 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:27:04.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:04 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3780345594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:04 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:27:04.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:04 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3780345594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:04 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:27:04.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:04 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3780345594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:05.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 -- 172.21.15.137:0/3863199439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 msgr2=0x7f541c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:05.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 --2- 172.21.15.137:0/3863199439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 0x7f541c10ea90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f5414009ca0 tx=0x7f541402a8c0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 -- 172.21.15.137:0/3863199439 shutdown_connections 2024-09-12T12:27:05.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 --2- 172.21.15.137:0/3863199439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 0x7f541c10ea90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 --2- 172.21.15.137:0/3863199439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c102790 0x7f541c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 --2- 172.21.15.137:0/3863199439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f541c101de0 0x7f541c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 -- 172.21.15.137:0/3863199439 >> 172.21.15.137:0/3863199439 conn(0x7f541c0fd8c0 msgr2=0x7f541c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 -- 172.21.15.137:0/3863199439 shutdown_connections 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.066+0000 7f5420fe6640 1 -- 172.21.15.137:0/3863199439 wait complete. 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.067+0000 7f5420fe6640 1 Processor -- start 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.067+0000 7f5420fe6640 1 -- start start 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.067+0000 7f5420fe6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c101de0 0x7f541c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5420fe6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f541c102790 0x7f541c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5420fe6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 0x7f541c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5420fe6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f541c111f70 con 0x7f541c106f10 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5420fe6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f541c111df0 con 0x7f541c102790 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5420fe6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1120f0 con 0x7f541c101de0 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541a59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c101de0 0x7f541c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541ad9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 0x7f541c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541a59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c101de0 0x7f541c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:37572/0 (socket says 172.21.15.137:37572) 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541a59c640 1 -- 172.21.15.137:0/1593438773 learned_addr learned my addr 172.21.15.137:0/1593438773 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5411d9b640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f541c102790 0x7f541c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541ad9d640 1 -- 172.21.15.137:0/1593438773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c101de0 msgr2=0x7f541c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541ad9d640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c101de0 0x7f541c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541ad9d640 1 -- 172.21.15.137:0/1593438773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f541c102790 msgr2=0x7f541c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541ad9d640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f541c102790 0x7f541c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541ad9d640 1 -- 172.21.15.137:0/1593438773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c19d420 con 0x7f541c106f10 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541a59c640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c101de0 0x7f541c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f541ad9d640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 0x7f541c19cd40 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f541402ade0 tx=0x7f54140319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5413fff640 1 -- 172.21.15.137:0/1593438773 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414003b70 con 0x7f541c106f10 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5413fff640 1 -- 172.21.15.137:0/1593438773 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5414003d10 con 0x7f541c106f10 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c19d6b0 con 0x7f541c106f10 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.068+0000 7f5413fff640 1 -- 172.21.15.137:0/1593438773 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541402fc40 con 0x7f541c106f10 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.069+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1a4ff0 con 0x7f541c106f10 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.071+0000 7f5413fff640 1 -- 172.21.15.137:0/1593438773 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5414004050 con 0x7f541c106f10 2024-09-12T12:27:05.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.071+0000 7f54233bf740 1 Processor -- start 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.071+0000 7f54233bf740 1 -- start start 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f54233bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff75b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f54233bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff76080 0x55977ff80400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f54233bf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff80940 0x55977ff82d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f54233bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55977fedb2a0 con 0x55977ff75760 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f54233bf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55977fe82b10 con 0x55977ff80940 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f54233bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55977fed7260 con 0x55977ff76080 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5413fff640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54080a2cf0 0x7f54080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff75b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff75b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56974/0 (socket says 172.21.15.137:56974) 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 -- 172.21.15.137:0/4138343951 learned_addr learned my addr 172.21.15.137:0/4138343951 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff80940 0x55977ff82d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5413fff640 1 -- 172.21.15.137:0/1593438773 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8528+0+0 (secure 0 0 0) 0x7f54140f0290 con 0x7f541c106f10 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f541a59c640 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff76080 0x55977ff80400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f541ad9d640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54080a2cf0 0x7f54080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 -- 172.21.15.137:0/4138343951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff76080 msgr2=0x55977ff80400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff76080 0x55977ff80400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 -- 172.21.15.137:0/4138343951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff80940 msgr2=0x55977ff82d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff80940 0x55977ff82d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 -- 172.21.15.137:0/4138343951 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977fc4cb90 con 0x55977ff75760 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.072+0000 7f5411d9b640 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff75b40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55977ff73070 tx=0x7f5404005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1153476c9f9f7e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.073+0000 7f5410d99640 1 -- 172.21.15.137:0/4138343951 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54040131b0 con 0x55977ff75760 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.073+0000 7f5410d99640 1 -- 172.21.15.137:0/4138343951 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5404013ce0 con 0x55977ff75760 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.073+0000 7f5410d99640 1 -- 172.21.15.137:0/4138343951 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400fea0 con 0x55977ff75760 2024-09-12T12:27:05.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.073+0000 7f54233bf740 1 -- 172.21.15.137:0/4138343951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 msgr2=0x55977ff75b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.073+0000 7f541ad9d640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54080a2cf0 0x7f54080a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5414009bf0 tx=0x7f541402a8c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.073+0000 7f54233bf740 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff75b40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55977ff73070 tx=0x7f5404005850 comp rx=0 tx=0).stop 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- 172.21.15.137:0/4138343951 shutdown_connections 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff80940 0x55977ff82d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff76080 0x55977ff80400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 --2- 172.21.15.137:0/4138343951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff75b40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- 172.21.15.137:0/4138343951 >> 172.21.15.137:0/4138343951 conn(0x55977ff71cf0 msgr2=0x55977ff720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- 172.21.15.137:0/4138343951 shutdown_connections 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- 172.21.15.137:0/4138343951 wait complete. 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- start start 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 Processor -- start 2024-09-12T12:27:05.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff7d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff80940 0x55977ff7e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55977ff83510 con 0x55977ff75760 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55977fe636e0 con 0x55977ff76080 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55977ff83690 con 0x55977ff80940 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f54233bf740 10 client.0 _ll_register_callbacks cb 0x55977fe87a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff7d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff7d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56988/0 (socket says 172.21.15.137:56988) 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f5411d9b640 1 -- 172.21.15.137:0/224546140 learned_addr learned my addr 172.21.15.137:0/224546140 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.074+0000 7f541a59c640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5411d9b640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff80940 0x55977ff7e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5411d9b640 1 -- 172.21.15.137:0/224546140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff80940 msgr2=0x55977ff7e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5411d9b640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff80940 0x55977ff7e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5411d9b640 1 -- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 msgr2=0x55977ff7dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5411d9b640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5411d9b640 1 -- 172.21.15.137:0/224546140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977ff8ca30 con 0x55977ff75760 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f541a59c640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5411d9b640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff7d850 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f54040058c0 tx=0x7f540400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcf7884fce42d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5401ffb640 1 -- 172.21.15.137:0/224546140 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404004260 con 0x55977ff75760 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5401ffb640 1 -- 172.21.15.137:0/224546140 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8528+0+0 (secure 0 0 0) 0x7f5404004400 con 0x55977ff75760 2024-09-12T12:27:05.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5404013ce0 con 0x55977ff75760 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5401ffb640 1 -- 172.21.15.137:0/224546140 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f540402f050 con 0x55977ff75760 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f54233bf740 10 client.17127 fetch_fsmap learned FSMap version 43 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55977ff8f3c0 con 0x55977ff75760 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.075+0000 7f5401ffb640 1 -- 172.21.15.137:0/224546140 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5404013ce0 con 0x55977ff75760 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f5401ffb640 1 -- 172.21.15.137:0/224546140 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f5404004cc0 con 0x55977ff75760 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f54233bf740 10 client.17127 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f54233bf740 10 client.17127 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f54233bf740 20 client.17127 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f54233bf740 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977ff90710 0x55977ff92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f54233bf740 4 client.17127 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55977fee89d0 con 0x55977ff90710 2024-09-12T12:27:05.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f541a59c640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977ff90710 0x55977ff92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f541a59c640 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977ff90710 0x55977ff92b00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bcb18fdf680982d server_cookie=96a9089a0c45b099 in_seq=0 out_seq=0 2024-09-12T12:27:05.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.076+0000 7f5401ffb640 10 client.17127 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f5401ffb640 1 -- 172.21.15.137:0/224546140 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55977fee89d0 con 0x55977ff90710 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f5401ffb640 10 client.17127 handle_command_reply: tid=0 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 client.17127 shutdown 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 msgr2=0x55977ff7d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff7d850 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f54040058c0 tx=0x7f540400fb60 comp rx=0 tx=0).stop 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 shutdown_connections 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff80940 0x55977ff7e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977ff90710 0x55977ff92b00 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 --2- 172.21.15.137:0/224546140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff75760 0x55977ff7d850 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f5401ffb640 0 client.17127 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:05.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 >> 172.21.15.137:0/224546140 conn(0x55977ff71cf0 msgr2=0x55977ff77a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:05.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 shutdown_connections 2024-09-12T12:27:05.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 1 -- 172.21.15.137:0/224546140 wait complete. 2024-09-12T12:27:05.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.078+0000 7f54233bf740 20 client.17127 trim_cache size 0 max 16384 2024-09-12T12:27:05.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 Processor -- start 2024-09-12T12:27:05.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 -- start start 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff7df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1b01f0 con 0x55977ff80940 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ada00 con 0x55977ff76080 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.096+0000 7f54233bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ab850 con 0x55977ff75760 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff98310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f541a59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff98310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56990/0 (socket says 172.21.15.137:56990) 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 -- 172.21.15.137:0/1898110404 learned_addr learned my addr 172.21.15.137:0/1898110404 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:05.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f541a59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:41030/0 (socket says 172.21.15.137:41030) 2024-09-12T12:27:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff7df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 -- 172.21.15.137:0/1898110404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 msgr2=0x55977ff7df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff7df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 -- 172.21.15.137:0/1898110404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 msgr2=0x55977ff7e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 -- 172.21.15.137:0/1898110404 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977ff75590 con 0x55977ff80940 2024-09-12T12:27:05.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5411d9b640 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff98310 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5404003af0 tx=0x7f540401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a995fde0c647ff89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5403fff640 1 -- 172.21.15.137:0/1898110404 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540400f790 con 0x55977ff80940 2024-09-12T12:27:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5403fff640 1 -- 172.21.15.137:0/1898110404 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5404024070 con 0x55977ff80940 2024-09-12T12:27:05.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.097+0000 7f5403fff640 1 -- 172.21.15.137:0/1898110404 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404005720 con 0x55977ff80940 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 -- 172.21.15.137:0/1898110404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 msgr2=0x55977ff98310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff98310 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f5404003af0 tx=0x7f540401c040 comp rx=0 tx=0).stop 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 -- 172.21.15.137:0/1898110404 shutdown_connections 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff98310 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff7e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 --2- 172.21.15.137:0/1898110404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff7df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 -- 172.21.15.137:0/1898110404 >> 172.21.15.137:0/1898110404 conn(0x55977ff71cf0 msgr2=0x55977ff7d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 -- 172.21.15.137:0/1898110404 shutdown_connections 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 -- 172.21.15.137:0/1898110404 wait complete. 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.098+0000 7f54233bf740 1 -- start start 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 1 Processor -- start 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff84c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1abec0 con 0x55977ff80940 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ad330 con 0x55977ff76080 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f541c1ac590 con 0x55977ff75760 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f54233bf740 10 client.0 _ll_register_callbacks cb 0x7f5404013610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff844e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:41036/0 (socket says 172.21.15.137:41036) 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f5411d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff891b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:37618/0 (socket says 172.21.15.137:37618) 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 -- 172.21.15.137:0/90376534 learned_addr learned my addr 172.21.15.137:0/90376534 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f5411d9b640 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff84c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 -- 172.21.15.137:0/90376534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 msgr2=0x55977ff891b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff891b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 -- 172.21.15.137:0/90376534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 msgr2=0x55977ff84c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff84c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 -- 172.21.15.137:0/90376534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55977ff6d260 con 0x55977ff76080 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f5411d9b640 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff891b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f541a59c640 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff844e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f540c004700 tx=0x7f540c01f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4919654020285614 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.099+0000 7f5401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f5401ffb640 1 -- 172.21.15.137:0/90376534 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540c004250 con 0x55977ff76080 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f5401ffb640 1 -- 172.21.15.137:0/90376534 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8528+0+0 (secure 0 0 0) 0x7f540c0043f0 con 0x55977ff76080 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5404024070 con 0x55977ff76080 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f5401ffb640 1 -- 172.21.15.137:0/90376534 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f540c0035f0 con 0x55977ff76080 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f54233bf740 10 client.26435 fetch_fsmap learned FSMap version 43 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55977fffb310 con 0x55977ff76080 2024-09-12T12:27:05.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f5401ffb640 1 -- 172.21.15.137:0/90376534 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5404024070 con 0x55977ff76080 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f5401ffb640 1 -- 172.21.15.137:0/90376534 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f540c032850 con 0x55977ff76080 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f54233bf740 10 client.26435 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f54233bf740 10 client.26435 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f54233bf740 20 client.26435 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.100+0000 7f54233bf740 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977fffbb40 0x55977fffdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.101+0000 7f54233bf740 4 client.26435 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.101+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55977fee2d20 con 0x55977fffbb40 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.101+0000 7f5411d9b640 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977fffbb40 0x55977fffdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.101+0000 7f5401ffb640 10 client.26435 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:05.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:05.101+0000 7f5411d9b640 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977fffbb40 0x55977fffdf60 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4efac6cea83a7acd server_cookie=317f971419319dda in_seq=0 out_seq=0 2024-09-12T12:27:05.213 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3323 9.95 MB/sec warmup 37 sec latency 107.085 ms 2024-09-12T12:27:05.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:05 smithi189 ceph-mon[35475]: pgmap v437: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 72 op/s 2024-09-12T12:27:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:05 smithi171 ceph-mon[35098]: pgmap v437: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 72 op/s 2024-09-12T12:27:05.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:05 smithi137 ceph-mon[30950]: pgmap v437: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 12 MiB/s wr, 72 op/s 2024-09-12T12:27:06.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f5401ffb640 1 -- 172.21.15.137:0/90376534 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55977fee2d20 con 0x55977fffbb40 2024-09-12T12:27:06.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f5401ffb640 10 client.26435 handle_command_reply: tid=0 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 client.26435 shutdown 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 msgr2=0x55977ff844e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff844e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f540c004700 tx=0x7f540c01f7c0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 shutdown_connections 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55977fffbb40 0x55977fffdf60 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55977ff80940 0x55977ff84c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55977ff76080 0x55977ff844e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 --2- 172.21.15.137:0/90376534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55977ff75760 0x55977ff891b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f5401ffb640 0 client.26435 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 >> 172.21.15.137:0/90376534 conn(0x55977ff71cf0 msgr2=0x55977ff779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 shutdown_connections 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.020+0000 7f54233bf740 1 -- 172.21.15.137:0/90376534 wait complete. 2024-09-12T12:27:06.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f54233bf740 20 client.26435 trim_cache size 0 max 16384 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 43, 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 54, 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "63946e6c": { 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 54, 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:27:06.023 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 150.0, 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:06.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:06.025 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:06.025 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:06.025 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:27:06.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54080a2cf0 msgr2=0x7f54080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54080a2cf0 0x7f54080a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5414009bf0 tx=0x7f541402a8c0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 msgr2=0x7f541c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 0x7f541c19cd40 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f541402ade0 tx=0x7f54140319f0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 shutdown_connections 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f54080a2cf0 0x7f54080a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f541c106f10 0x7f541c19cd40 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f541c102790 0x7f541c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 --2- 172.21.15.137:0/1593438773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f541c101de0 0x7f541c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 >> 172.21.15.137:0/1593438773 conn(0x7f541c0fd8c0 msgr2=0x7f541c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 shutdown_connections 2024-09-12T12:27:06.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:06.021+0000 7f5420fe6640 1 -- 172.21.15.137:0/1593438773 wait complete. 2024-09-12T12:27:06.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 9.72 MB/sec warmup 38 sec latency 1086.990 ms 2024-09-12T12:27:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 -- 172.21.15.137:0/3631474177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501069c0 msgr2=0x7f9f50106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/3631474177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501069c0 0x7f9f50106da0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009930 tx=0x7f9f4002a420 comp rx=0 tx=0).stop 2024-09-12T12:27:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 -- 172.21.15.137:0/3631474177 shutdown_connections 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/3631474177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f50101350 0x7f9f5010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/3631474177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f501009b0 0x7f9f50100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/3631474177 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501069c0 0x7f9f50106da0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 -- 172.21.15.137:0/3631474177 >> 172.21.15.137:0/3631474177 conn(0x7f9f50077990 msgr2=0x7f9f500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 -- 172.21.15.137:0/3631474177 shutdown_connections 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.325+0000 7f9f55c1e640 1 -- 172.21.15.137:0/3631474177 wait complete. 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 Processor -- start 2024-09-12T12:27:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 -- start start 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501009b0 0x7f9f501991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f50101350 0x7f9f50199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f501069c0 0x7f9f501a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50110dc0 con 0x7f9f501009b0 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50110c40 con 0x7f9f50101350 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f55c1e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f50110f40 con 0x7f9f501069c0 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501009b0 0x7f9f501991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501009b0 0x7f9f501991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57020/0 (socket says 172.21.15.137:57020) 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/999316418 learned_addr learned my addr 172.21.15.137:0/999316418 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f4effd640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f50101350 0x7f9f50199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.326+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f501069c0 0x7f9f501a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/999316418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f501069c0 msgr2=0x7f9f501a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f501069c0 0x7f9f501a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/999316418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f50101350 msgr2=0x7f9f50199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f50101350 0x7f9f50199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/999316418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f501a4780 con 0x7f9f501009b0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501009b0 0x7f9f501991c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009230 tx=0x7f9f40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4cff9640 1 -- 172.21.15.137:0/999316418 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4003f030 con 0x7f9f501009b0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4cff9640 1 -- 172.21.15.137:0/999316418 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f40003ad0 con 0x7f9f501009b0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f501a4950 con 0x7f9f501009b0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f4cff9640 1 -- 172.21.15.137:0/999316418 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40035070 con 0x7f9f501009b0 2024-09-12T12:27:06.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.327+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f501a4e10 con 0x7f9f501009b0 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.329+0000 7f9f4cff9640 1 -- 172.21.15.137:0/999316418 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9f40042030 con 0x7f9f501009b0 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 Processor -- start 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 -- start start 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec3ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec48770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ba9eba22f0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ba9eb4a270 con 0x55ba9ec3e410 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba9eb9e2b0 con 0x55ba9ec48cb0 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec3ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec3ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57032/0 (socket says 172.21.15.137:57032) 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 -- 172.21.15.137:0/749874903 learned_addr learned my addr 172.21.15.137:0/749874903 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4cff9640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9f380a2e20 0x7f9f380a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec48770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9f380a2e20 0x7f9f380a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4effd640 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 -- 172.21.15.137:0/749874903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec48cb0 msgr2=0x55ba9ec4b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4cff9640 1 -- 172.21.15.137:0/999316418 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8528+0+0 (secure 0 0 0) 0x7f9f400ef8d0 con 0x7f9f501009b0 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 -- 172.21.15.137:0/749874903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 msgr2=0x55ba9ec48770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec48770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 -- 172.21.15.137:0/749874903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9eb49340 con 0x55ba9ec3daf0 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec48770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.330+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec3ded0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9f48014940 tx=0x7f9f48014e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511d68217ecce6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f35ffb640 1 -- 172.21.15.137:0/749874903 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48014050 con 0x55ba9ec3daf0 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f35ffb640 1 -- 172.21.15.137:0/749874903 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f480141f0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f35ffb640 1 -- 172.21.15.137:0/749874903 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f48013640 con 0x55ba9ec3daf0 2024-09-12T12:27:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 -- 172.21.15.137:0/749874903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 msgr2=0x55ba9ec3ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9f380a2e20 0x7f9f380a52e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f48021000 tx=0x7f9f48020000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec3ded0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9f48014940 tx=0x7f9f48014e00 comp rx=0 tx=0).stop 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 -- 172.21.15.137:0/749874903 shutdown_connections 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec48770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/749874903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec3ded0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 -- 172.21.15.137:0/749874903 >> 172.21.15.137:0/749874903 conn(0x55ba9ec3a060 msgr2=0x55ba9ec3a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 -- 172.21.15.137:0/749874903 shutdown_connections 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.331+0000 7f9f58a8d740 1 -- 172.21.15.137:0/749874903 wait complete. 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 -- start start 2024-09-12T12:27:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 Processor -- start 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec45b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec46080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ba9ec4b880 con 0x55ba9ec3daf0 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ba9eb2b290 con 0x55ba9ec48cb0 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ba9ec4ba00 con 0x55ba9ec3e410 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f58a8d740 10 client.0 _ll_register_callbacks cb 0x55ba9eb5c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec45b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec45b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57034/0 (socket says 172.21.15.137:57034) 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/4051950292 learned_addr learned my addr 172.21.15.137:0/4051950292 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f4effd640 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec46080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.332+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/4051950292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 msgr2=0x55ba9ec46080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec46080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec48cb0 msgr2=0x55ba9ec465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f4f7fe640 1 -- 172.21.15.137:0/4051950292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9e8b5210 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec45b40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9f400b87c0 tx=0x7f9f40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f69b4282bfa2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f2effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f2effd640 1 -- 172.21.15.137:0/4051950292 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40049040 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f2effd640 1 -- 172.21.15.137:0/4051950292 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8528+0+0 (secure 0 0 0) 0x7f9f400b8460 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f480141f0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f2effd640 1 -- 172.21.15.137:0/4051950292 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9f400bbdf0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 10 client.17145 fetch_fsmap learned FSMap version 43 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ba9e981620 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f2effd640 1 -- 172.21.15.137:0/4051950292 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f480141f0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f2effd640 1 -- 172.21.15.137:0/4051950292 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f9f40031bd0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 10 client.17145 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 20 client.17145 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ec58c50 0x55ba9ec5b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 4 client.17145 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.333+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba9ebafa20 con 0x55ba9ec58c50 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.334+0000 7f9f4effd640 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ec58c50 0x55ba9ec5b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.334+0000 7f9f4effd640 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ec58c50 0x55ba9ec5b0d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a14947091e9dd04 server_cookie=b412e558b8d59ca1 in_seq=0 out_seq=0 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.334+0000 7f9f2effd640 10 client.17145 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f2effd640 1 -- 172.21.15.137:0/4051950292 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba9ebafa20 con 0x55ba9ec58c50 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f2effd640 10 client.17145 handle_command_reply: tid=0 2024-09-12T12:27:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 client.17145 shutdown 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 msgr2=0x55ba9ec45b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec45b40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9f400b87c0 tx=0x7f9f40009260 comp rx=0 tx=0).stop 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 shutdown_connections 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec46080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ec58c50 0x55ba9ec5b0d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/4051950292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec45b40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f2effd640 0 client.17145 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 >> 172.21.15.137:0/4051950292 conn(0x55ba9ec3a060 msgr2=0x55ba9ec3fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 shutdown_connections 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 1 -- 172.21.15.137:0/4051950292 wait complete. 2024-09-12T12:27:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.335+0000 7f9f58a8d740 20 client.17145 trim_cache size 0 max 16384 2024-09-12T12:27:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 Processor -- start 2024-09-12T12:27:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 -- start start 2024-09-12T12:27:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec60f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501b0090 con 0x55ba9ec48cb0 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501ad8a0 con 0x55ba9ec3e410 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.350+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501ab6f0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec60f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec60f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57050/0 (socket says 172.21.15.137:57050) 2024-09-12T12:27:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 -- 172.21.15.137:0/3517799409 learned_addr learned my addr 172.21.15.137:0/3517799409 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4effd640 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 -- 172.21.15.137:0/3517799409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3daf0 msgr2=0x55ba9ec463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 -- 172.21.15.137:0/3517799409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 msgr2=0x55ba9ec609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 -- 172.21.15.137:0/3517799409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9ec39e90 con 0x55ba9ec48cb0 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec60f20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9f48005b40 tx=0x7f9f48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925e377f05a46964 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f357fa640 1 -- 172.21.15.137:0/3517799409 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f480120c0 con 0x55ba9ec48cb0 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f357fa640 1 -- 172.21.15.137:0/3517799409 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f48013400 con 0x55ba9ec48cb0 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.351+0000 7f9f357fa640 1 -- 172.21.15.137:0/3517799409 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f480054b0 con 0x55ba9ec48cb0 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 -- 172.21.15.137:0/3517799409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 msgr2=0x55ba9ec60f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec60f20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9f48005b40 tx=0x7f9f48017000 comp rx=0 tx=0).stop 2024-09-12T12:27:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 -- 172.21.15.137:0/3517799409 shutdown_connections 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec60f20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3e410 0x55ba9ec609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/3517799409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 -- 172.21.15.137:0/3517799409 >> 172.21.15.137:0/3517799409 conn(0x55ba9ec3a060 msgr2=0x55ba9ec47bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 -- 172.21.15.137:0/3517799409 shutdown_connections 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 -- 172.21.15.137:0/3517799409 wait complete. 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 -- start start 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.352+0000 7f9f58a8d740 1 Processor -- start 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec50cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec51230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501abd60 con 0x55ba9ec48cb0 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501ad1d0 con 0x55ba9ec3daf0 2024-09-12T12:27:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f501ac430 con 0x55ba9ec3e410 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 10 client.0 _ll_register_callbacks cb 0x7f9f400b8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4f7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec50cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57056/0 (socket says 172.21.15.137:57056) 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 -- 172.21.15.137:0/2837492775 learned_addr learned my addr 172.21.15.137:0/2837492775 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4effd640 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec51230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 -- 172.21.15.137:0/2837492775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 msgr2=0x55ba9ec51230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec51230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 -- 172.21.15.137:0/2837492775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3daf0 msgr2=0x55ba9ec50cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec50cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 -- 172.21.15.137:0/2837492775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9ec352d0 con 0x55ba9ec48cb0 2024-09-12T12:27:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4effd640 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec51230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f4ffff640 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4dcc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9f48014710 tx=0x7f9f48005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbf1b03b8136b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f2effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f2effd640 1 -- 172.21.15.137:0/2837492775 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f480059a0 con 0x55ba9ec48cb0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f2effd640 1 -- 172.21.15.137:0/2837492775 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8528+0+0 (secure 0 0 0) 0x7f9f48023040 con 0x55ba9ec48cb0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.353+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f48013400 con 0x55ba9ec48cb0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f2effd640 1 -- 172.21.15.137:0/2837492775 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9f48009d10 con 0x55ba9ec48cb0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 10 client.17151 fetch_fsmap learned FSMap version 43 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ba9ecc3cf0 con 0x55ba9ec48cb0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f2effd640 1 -- 172.21.15.137:0/2837492775 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f48013400 con 0x55ba9ec48cb0 2024-09-12T12:27:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f2effd640 1 -- 172.21.15.137:0/2837492775 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 43) ==== 8370+0+0 (secure 0 0 0) 0x7f9f4800f040 con 0x55ba9ec48cb0 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 10 client.17151 fetch_fsmap finished waiting for FSMap version 43 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 20 client.17151 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ecc42c0 0x55ba9ecc66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 4 client.17151 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba9eba9d70 con 0x55ba9ecc42c0 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ecc42c0 0x55ba9ecc66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f2effd640 10 client.17151 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.354+0000 7f9f4f7fe640 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ecc42c0 0x55ba9ecc66e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e3d9c44a4de214 server_cookie=f3aa7a032bba99f1 in_seq=0 out_seq=0 2024-09-12T12:27:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f2effd640 1 -- 172.21.15.137:0/2837492775 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba9eba9d70 con 0x55ba9ecc42c0 2024-09-12T12:27:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f2effd640 10 client.17151 handle_command_reply: tid=0 2024-09-12T12:27:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 client.17151 shutdown 2024-09-12T12:27:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 msgr2=0x55ba9ec4dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4dcc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9f48014710 tx=0x7f9f48005cf0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 shutdown_connections 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ba9ecc42c0 0x55ba9ecc66e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f2effd640 0 client.17151 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba9ec48cb0 0x55ba9ec4dcc0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ba9ec3e410 0x55ba9ec51230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 --2- 172.21.15.137:0/2837492775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba9ec3daf0 0x55ba9ec50cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 >> 172.21.15.137:0/2837492775 conn(0x55ba9ec3a060 msgr2=0x55ba9ec389a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 shutdown_connections 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 1 -- 172.21.15.137:0/2837492775 wait complete. 2024-09-12T12:27:06.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.591+0000 7f9f58a8d740 20 client.17151 trim_cache size 0 max 16384 2024-09-12T12:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:27:06.594 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9f380a2e20 msgr2=0x7f9f380a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9f380a2e20 0x7f9f380a52e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f48021000 tx=0x7f9f48020000 comp rx=0 tx=0).stop 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501009b0 msgr2=0x7f9f501991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501009b0 0x7f9f501991c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009230 tx=0x7f9f40004270 comp rx=0 tx=0).stop 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 shutdown_connections 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9f380a2e20 0x7f9f380a52e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f501069c0 0x7f9f501a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.595 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f50101350 0x7f9f50199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 --2- 172.21.15.137:0/999316418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f501009b0 0x7f9f501991c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 >> 172.21.15.137:0/999316418 conn(0x7f9f50077990 msgr2=0x7f9f50103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 shutdown_connections 2024-09-12T12:27:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:06.592+0000 7f9f55c1e640 1 -- 172.21.15.137:0/999316418 wait complete. 2024-09-12T12:27:06.692 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '63946e6c', quiesce took 4.5 seconds. Will release after: 25.5 2024-09-12T12:27:07.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 9.47 MB/sec warmup 39 sec latency 2087.068 ms 2024-09-12T12:27:07.319 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:27:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:27:07.319 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:27:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:27:07.319 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:27:06 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/3629954008 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55560a929860 con 0x55560a906800 2024-09-12T12:27:07.319 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:27:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:27:07.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:07 smithi189 ceph-mon[35475]: pgmap v438: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-12T12:27:07.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:07 smithi189 ceph-mon[35475]: daemon mds.f restarted 2024-09-12T12:27:07.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:07 smithi171 ceph-mon[35098]: pgmap v438: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-12T12:27:07.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:07 smithi171 ceph-mon[35098]: daemon mds.f restarted 2024-09-12T12:27:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:07 smithi137 ceph-mon[30950]: pgmap v438: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-12T12:27:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:07 smithi137 ceph-mon[30950]: daemon mds.f restarted 2024-09-12T12:27:08.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 9.23 MB/sec warmup 40 sec latency 3087.166 ms 2024-09-12T12:27:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:08 smithi189 ceph-mon[35475]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T12:27:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:08 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/3604136981,v1:172.21.15.189:6835/3604136981] up:boot 2024-09-12T12:27:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:08 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:27:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:08 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:27:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:08 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:27:08.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:08 smithi189 ceph-mon[35475]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T12:27:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:08 smithi171 ceph-mon[35098]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T12:27:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:08 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/3604136981,v1:172.21.15.189:6835/3604136981] up:boot 2024-09-12T12:27:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:08 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:27:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:08 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:27:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:08 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:27:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:08 smithi171 ceph-mon[35098]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T12:27:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:08 smithi137 ceph-mon[30950]: osdmap e105: 12 total, 12 up, 12 in 2024-09-12T12:27:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:08 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/3604136981,v1:172.21.15.189:6835/3604136981] up:boot 2024-09-12T12:27:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:08 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:27:08.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:08 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:27:08.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:08 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:27:08.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:08 smithi137 ceph-mon[30950]: osdmap e106: 12 total, 12 up, 12 in 2024-09-12T12:27:09.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 9.00 MB/sec warmup 41 sec latency 4087.289 ms 2024-09-12T12:27:09.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:09 smithi189 ceph-mon[35475]: pgmap v440: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-12T12:27:09.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:09 smithi171 ceph-mon[35098]: pgmap v440: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-12T12:27:09.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:09 smithi137 ceph-mon[30950]: pgmap v440: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-12T12:27:10.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 8.79 MB/sec warmup 42 sec latency 5087.369 ms 2024-09-12T12:27:11.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 8.59 MB/sec warmup 43 sec latency 6087.446 ms 2024-09-12T12:27:11.592 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:11 smithi189 ceph-mon[35475]: pgmap v442: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-12T12:27:11.601 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:11 smithi137 ceph-mon[30950]: pgmap v442: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-12T12:27:11.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:11 smithi171 ceph-mon[35098]: pgmap v442: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.1 MiB/s wr, 412 op/s 2024-09-12T12:27:12.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 8.39 MB/sec warmup 44 sec latency 7087.487 ms 2024-09-12T12:27:13.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 8.20 MB/sec warmup 45 sec latency 8087.520 ms 2024-09-12T12:27:13.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:13 smithi189 ceph-mon[35475]: pgmap v443: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 374 op/s 2024-09-12T12:27:13.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:13 smithi171 ceph-mon[35098]: pgmap v443: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 374 op/s 2024-09-12T12:27:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:13 smithi137 ceph-mon[30950]: pgmap v443: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 374 op/s 2024-09-12T12:27:14.214 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 8.03 MB/sec warmup 46 sec latency 9087.803 ms 2024-09-12T12:27:15.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 7.86 MB/sec warmup 47 sec latency 10087.913 ms 2024-09-12T12:27:15.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:15 smithi189 ceph-mon[35475]: pgmap v444: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 669 KiB/s wr, 247 op/s 2024-09-12T12:27:15.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:15 smithi171 ceph-mon[35098]: pgmap v444: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 669 KiB/s wr, 247 op/s 2024-09-12T12:27:15.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:15 smithi137 ceph-mon[30950]: pgmap v444: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 669 KiB/s wr, 247 op/s 2024-09-12T12:27:16.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 7.69 MB/sec warmup 48 sec latency 11087.979 ms 2024-09-12T12:27:17.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 7.53 MB/sec warmup 49 sec latency 12088.021 ms 2024-09-12T12:27:17.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:17 smithi189 ceph-mon[35475]: pgmap v445: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 697 KiB/s wr, 253 op/s 2024-09-12T12:27:17.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:17 smithi171 ceph-mon[35098]: pgmap v445: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 697 KiB/s wr, 253 op/s 2024-09-12T12:27:17.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:17 smithi137 ceph-mon[30950]: pgmap v445: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 697 KiB/s wr, 253 op/s 2024-09-12T12:27:18.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 7.38 MB/sec warmup 50 sec latency 13088.092 ms 2024-09-12T12:27:19.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 7.24 MB/sec warmup 51 sec latency 14088.185 ms 2024-09-12T12:27:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/3785082822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 msgr2=0x7f4a58106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/3785082822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a58106da0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009cd0 tx=0x7f4a4802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/3785082822 shutdown_connections 2024-09-12T12:27:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/3785082822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a58101350 0x7f4a5810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/3785082822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a581009b0 0x7f4a58100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/3785082822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a58106da0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/3785082822 >> 172.21.15.137:0/3785082822 conn(0x7f4a58077990 msgr2=0x7f4a580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/3785082822 shutdown_connections 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.688+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/3785082822 wait complete. 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 Processor -- start 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 -- start start 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a581009b0 0x7f4a580fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a58101350 0x7f4a580fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a580fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58110e50 con 0x7f4a581069c0 2024-09-12T12:27:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58110cd0 con 0x7f4a581009b0 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5fb4b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58110fd0 con 0x7f4a58101350 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5e0e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a580fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5e0e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a580fd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37442/0 (socket says 172.21.15.137:37442) 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5e0e8640 1 -- 172.21.15.137:0/2544621539 learned_addr learned my addr 172.21.15.137:0/2544621539 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.689+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a58101350 0x7f4a580fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a581009b0 0x7f4a580fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5e0e8640 1 -- 172.21.15.137:0/2544621539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a58101350 msgr2=0x7f4a580fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5e0e8640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a58101350 0x7f4a580fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5e0e8640 1 -- 172.21.15.137:0/2544621539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a581009b0 msgr2=0x7f4a580fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5e0e8640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a581009b0 0x7f4a580fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5e0e8640 1 -- 172.21.15.137:0/2544621539 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581a6870 con 0x7f4a581069c0 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5e0e8640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a580fd860 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000d950 tx=0x7f4a5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a56ffd640 1 -- 172.21.15.137:0/2544621539 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50017030 con 0x7f4a581069c0 2024-09-12T12:27:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a56ffd640 1 -- 172.21.15.137:0/2544621539 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a50004020 con 0x7f4a581069c0 2024-09-12T12:27:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a581a6b60 con 0x7f4a581069c0 2024-09-12T12:27:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a56ffd640 1 -- 172.21.15.137:0/2544621539 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50003c00 con 0x7f4a581069c0 2024-09-12T12:27:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.690+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a7060 con 0x7f4a581069c0 2024-09-12T12:27:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.693+0000 7f4a56ffd640 1 -- 172.21.15.137:0/2544621539 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4a5001a070 con 0x7f4a581069c0 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.693+0000 7f4a629c0740 1 Processor -- start 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.693+0000 7f4a629c0740 1 -- start start 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc69770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a629c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecbc32f0 con 0x55b8ecc5f410 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a629c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecb6b270 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a629c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecbbf2b0 con 0x55b8ecc69cb0 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a56ffd640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4a400a2da0 0x7f4a400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d8e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc69770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d8e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc69770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37450/0 (socket says 172.21.15.137:37450) 2024-09-12T12:27:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d8e7640 1 -- 172.21.15.137:0/2810112254 learned_addr learned my addr 172.21.15.137:0/2810112254 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5e0e8640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4a400a2da0 0x7f4a400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a56ffd640 1 -- 172.21.15.137:0/2544621539 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f4a500cebd0 con 0x7f4a581069c0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/2810112254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 msgr2=0x55b8ecc6c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc6c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/2810112254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 msgr2=0x55b8ecc69770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc69770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/2810112254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ecb6a340 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc69770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc5eed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ffce0 tx=0x7f4a4c005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385bf9f1db7dd310 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.694+0000 7f4a5e0e8640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4a400a2da0 0x7f4a400a5260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cc3f0 tx=0x7f4a500661e0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a3ffff640 1 -- 172.21.15.137:0/2810112254 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c004220 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a3ffff640 1 -- 172.21.15.137:0/2810112254 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4c0043c0 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a3ffff640 1 -- 172.21.15.137:0/2810112254 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0035f0 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a629c0740 1 -- 172.21.15.137:0/2810112254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 msgr2=0x55b8ecc5eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc5eed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4a580ffce0 tx=0x7f4a4c005c90 comp rx=0 tx=0).stop 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a629c0740 1 -- 172.21.15.137:0/2810112254 shutdown_connections 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc6c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc69770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2810112254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc5eed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.695+0000 7f4a629c0740 1 -- 172.21.15.137:0/2810112254 >> 172.21.15.137:0/2810112254 conn(0x55b8ecc5b060 msgr2=0x55b8ecc5b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 -- 172.21.15.137:0/2810112254 shutdown_connections 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 -- 172.21.15.137:0/2810112254 wait complete. 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 -- start start 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 Processor -- start 2024-09-12T12:27:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc66bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc67130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc67670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecc6c880 con 0x55b8ecc5f410 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecb4c290 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b8ecc6ca00 con 0x55b8ecc69cb0 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.696+0000 7f4a629c0740 10 client.0 _ll_register_callbacks cb 0x55b8ecb7d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc67130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc67130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37454/0 (socket says 172.21.15.137:37454) 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 -- 172.21.15.137:0/3910707493 learned_addr learned my addr 172.21.15.137:0/3910707493 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc66bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc67670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 -- 172.21.15.137:0/3910707493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 msgr2=0x55b8ecc67670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc67670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 -- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 msgr2=0x55b8ecc66bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc66bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 -- 172.21.15.137:0/3910707493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ec8d6210 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc67130 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4a4802aed0 tx=0x7f4a48031b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f08803ab226ced server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/3910707493 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4804a060 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/3910707493 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f4a48003970 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4c0043c0 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/3910707493 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4a48003b70 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a629c0740 10 client.17169 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b8ec9a2620 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.697+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/3910707493 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4c0043c0 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/3910707493 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4a48030070 con 0x55b8ecc5f410 2024-09-12T12:27:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a629c0740 10 client.17169 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a629c0740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a629c0740 20 client.17169 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a629c0740 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecc79c80 0x55b8ecc7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a629c0740 4 client.17169 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8ecbd0a20 con 0x55b8ecc79c80 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecc79c80 0x55b8ecc7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a3d7fa640 10 client.17169 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.698+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecc79c80 0x55b8ecc7c100 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4930fe09ba299ceb server_cookie=f9b029795228fbe0 in_seq=0 out_seq=0 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.699+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/3910707493 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8ecbd0a20 con 0x55b8ecc79c80 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.699+0000 7f4a3d7fa640 10 client.17169 handle_command_reply: tid=0 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.699+0000 7f4a629c0740 1 client.17169 shutdown 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.699+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 msgr2=0x55b8ecc67130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.699+0000 7f4a629c0740 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc67130 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4a4802aed0 tx=0x7f4a48031b80 comp rx=0 tx=0).stop 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 shutdown_connections 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc67670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecc79c80 0x55b8ecc7c100 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5f410 0x55b8ecc67130 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 --2- 172.21.15.137:0/3910707493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc66bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a3d7fa640 0 client.17169 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 >> 172.21.15.137:0/3910707493 conn(0x55b8ecc5b060 msgr2=0x55b8ecc60dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 shutdown_connections 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 1 -- 172.21.15.137:0/3910707493 wait complete. 2024-09-12T12:27:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.700+0000 7f4a629c0740 20 client.17169 trim_cache size 0 max 16384 2024-09-12T12:27:19.708 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:19 smithi171 ceph-mon[35098]: pgmap v446: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 576 KiB/s wr, 208 op/s 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 Processor -- start 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 -- start start 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc61560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581b2340 con 0x55b8ecc69cb0 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581afb50 con 0x55b8ecc5f410 2024-09-12T12:27:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a629c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581ad9c0 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a5d0e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a5d0e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc81a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37456/0 (socket says 172.21.15.137:37456) 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.714+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/2082110802 learned_addr learned my addr 172.21.15.137:0/2082110802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc61560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/2082110802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc5eaf0 msgr2=0x55b8ecc61560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc61560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/2082110802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 msgr2=0x55b8ecc674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/2082110802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ecc5c110 con 0x55b8ecc69cb0 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:27:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc81a10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00c010 tx=0x7f4a4c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1912345f9390e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a3f7fe640 1 -- 172.21.15.137:0/2082110802 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c004830 con 0x55b8ecc69cb0 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a3f7fe640 1 -- 172.21.15.137:0/2082110802 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4c004e20 con 0x55b8ecc69cb0 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.715+0000 7f4a3f7fe640 1 -- 172.21.15.137:0/2082110802 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c003850 con 0x55b8ecc69cb0 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 -- 172.21.15.137:0/2082110802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc69cb0 msgr2=0x55b8ecc81a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc81a10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00c010 tx=0x7f4a4c00bf20 comp rx=0 tx=0).stop 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 -- 172.21.15.137:0/2082110802 shutdown_connections 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc81a10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 --2- 172.21.15.137:0/2082110802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc61560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 -- 172.21.15.137:0/2082110802 >> 172.21.15.137:0/2082110802 conn(0x55b8ecc5b060 msgr2=0x55b8ecc68c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 -- 172.21.15.137:0/2082110802 shutdown_connections 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 -- 172.21.15.137:0/2082110802 wait complete. 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 -- start start 2024-09-12T12:27:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.716+0000 7f4a629c0740 1 Processor -- start 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc6dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581ae010 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581af480 con 0x55b8ecc5f410 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581ae6e0 con 0x55b8ecc69cb0 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 10 client.0 _ll_register_callbacks cb 0x7f4a480099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc72440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37466/0 (socket says 172.21.15.137:37466) 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/218467339 learned_addr learned my addr 172.21.15.137:0/218467339 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/218467339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 msgr2=0x55b8ecc6dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc6dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/218467339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 msgr2=0x55b8ecc6d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 -- 172.21.15.137:0/218467339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ecc562d0 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a5d0e6640 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc72440 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00bbe0 tx=0x7f4a4c004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4355dd846755e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/218467339 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c03a060 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/218467339 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f4a4c004450 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4c004e20 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/218467339 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4a4c004fd0 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 10 client.17175 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.717+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b8ecce47c0 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/218467339 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4c004e20 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/218467339 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4a4c01c030 con 0x55b8ecc5eaf0 2024-09-12T12:27:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a629c0740 10 client.17175 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a629c0740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a629c0740 20 client.17175 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a629c0740 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecce4cc0 0x55b8ecce7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a629c0740 4 client.17175 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8ecbcad70 con 0x55b8ecce4cc0 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.718+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecce4cc0 0x55b8ecce7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.719+0000 7f4a3d7fa640 10 client.17175 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.719+0000 7f4a5d8e7640 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecce4cc0 0x55b8ecce7100 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=214a9e334e867930 server_cookie=2b746cf0a3673ee4 in_seq=0 out_seq=0 2024-09-12T12:27:19.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:19 smithi137 ceph-mon[30950]: pgmap v446: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 576 KiB/s wr, 208 op/s 2024-09-12T12:27:19.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:19 smithi189 ceph-mon[35475]: pgmap v446: 97 pgs: 97 active+clean; 468 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 576 KiB/s wr, 208 op/s 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a3d7fa640 1 -- 172.21.15.137:0/218467339 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8ecbcad70 con 0x55b8ecce4cc0 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a3d7fa640 10 client.17175 handle_command_reply: tid=0 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 client.17175 shutdown 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5eaf0 msgr2=0x55b8ecc72440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc72440 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00bbe0 tx=0x7f4a4c004830 comp rx=0 tx=0).stop 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 shutdown_connections 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b8ecc69cb0 0x55b8ecc6dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8ecc5f410 0x55b8ecc6d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b8ecce4cc0 0x55b8ecce7100 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 --2- 172.21.15.137:0/218467339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b8ecc5eaf0 0x55b8ecc72440 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a3d7fa640 0 client.17175 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 >> 172.21.15.137:0/218467339 conn(0x55b8ecc5b060 msgr2=0x55b8ecc599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 shutdown_connections 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 1 -- 172.21.15.137:0/218467339 wait complete. 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.867+0000 7f4a629c0740 20 client.17175 trim_cache size 0 max 16384 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:27:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:27:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4a400a2da0 msgr2=0x7f4a400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4a400a2da0 0x7f4a400a5260 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cc3f0 tx=0x7f4a500661e0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 msgr2=0x7f4a580fd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a580fd860 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000d950 tx=0x7f4a5000de10 comp rx=0 tx=0).stop 2024-09-12T12:27:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 shutdown_connections 2024-09-12T12:27:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4a400a2da0 0x7f4a400a5260 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4a581069c0 0x7f4a580fd860 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a58101350 0x7f4a580fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 --2- 172.21.15.137:0/2544621539 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a581009b0 0x7f4a580fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 >> 172.21.15.137:0/2544621539 conn(0x7f4a58077990 msgr2=0x7f4a580699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 shutdown_connections 2024-09-12T12:27:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:19.868+0000 7f4a5fb4b640 1 -- 172.21.15.137:0/2544621539 wait complete. 2024-09-12T12:27:20.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 7.10 MB/sec warmup 52 sec latency 15088.310 ms 2024-09-12T12:27:21.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.97 MB/sec warmup 53 sec latency 16088.437 ms 2024-09-12T12:27:21.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:21 smithi171 ceph-mon[35098]: pgmap v447: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T12:27:21.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:21 smithi137 ceph-mon[30950]: pgmap v447: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T12:27:21.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:21 smithi189 ceph-mon[35475]: pgmap v447: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-12T12:27:22.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.84 MB/sec warmup 54 sec latency 17088.546 ms 2024-09-12T12:27:23.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.71 MB/sec warmup 55 sec latency 18088.615 ms 2024-09-12T12:27:23.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:23 smithi171 ceph-mon[35098]: pgmap v448: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 736 KiB/s wr, 16 op/s 2024-09-12T12:27:23.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:23 smithi137 ceph-mon[30950]: pgmap v448: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 736 KiB/s wr, 16 op/s 2024-09-12T12:27:23.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:23 smithi189 ceph-mon[35475]: pgmap v448: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 736 KiB/s wr, 16 op/s 2024-09-12T12:27:24.215 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.59 MB/sec warmup 56 sec latency 19088.735 ms 2024-09-12T12:27:25.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.48 MB/sec warmup 57 sec latency 20088.835 ms 2024-09-12T12:27:25.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:25 smithi171 ceph-mon[35098]: pgmap v449: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 736 KiB/s wr, 16 op/s 2024-09-12T12:27:25.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:25 smithi137 ceph-mon[30950]: pgmap v449: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 736 KiB/s wr, 16 op/s 2024-09-12T12:27:25.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:25 smithi189 ceph-mon[35475]: pgmap v449: 97 pgs: 97 active+clean; 476 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 736 KiB/s wr, 16 op/s 2024-09-12T12:27:26.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.37 MB/sec warmup 58 sec latency 21088.925 ms 2024-09-12T12:27:27.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.26 MB/sec warmup 59 sec latency 22089.026 ms 2024-09-12T12:27:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:27 smithi171 ceph-mon[35098]: pgmap v450: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 825 KiB/s wr, 42 op/s 2024-09-12T12:27:27.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:27 smithi137 ceph-mon[30950]: pgmap v450: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 825 KiB/s wr, 42 op/s 2024-09-12T12:27:27.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:27 smithi189 ceph-mon[35475]: pgmap v450: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 825 KiB/s wr, 42 op/s 2024-09-12T12:27:28.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.15 MB/sec warmup 60 sec latency 23089.103 ms 2024-09-12T12:27:28.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:28 smithi171 ceph-mon[35098]: pgmap v451: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 792 KiB/s wr, 33 op/s 2024-09-12T12:27:28.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:27:28.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:27:28.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:27:28.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:28 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:27:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:28 smithi137 ceph-mon[30950]: pgmap v451: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 792 KiB/s wr, 33 op/s 2024-09-12T12:27:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:27:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:27:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:27:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:28 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:27:29.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:28 smithi189 ceph-mon[35475]: pgmap v451: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 792 KiB/s wr, 33 op/s 2024-09-12T12:27:29.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:27:29.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:27:29.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:27:29.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:28 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:27:29.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 6.05 MB/sec warmup 61 sec latency 24089.225 ms 2024-09-12T12:27:30.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 5.95 MB/sec warmup 62 sec latency 25089.323 ms 2024-09-12T12:27:30.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:30 smithi171 ceph-mon[35098]: pgmap v452: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 802 KiB/s wr, 37 op/s 2024-09-12T12:27:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:30 smithi137 ceph-mon[30950]: pgmap v452: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 802 KiB/s wr, 37 op/s 2024-09-12T12:27:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:30 smithi189 ceph-mon[35475]: pgmap v452: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 802 KiB/s wr, 37 op/s 2024-09-12T12:27:31.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 5.86 MB/sec warmup 63 sec latency 26089.416 ms 2024-09-12T12:27:32.194 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:27:32.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 5.77 MB/sec warmup 64 sec latency 27089.461 ms 2024-09-12T12:27:32.443 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:27:32.834 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:32 smithi137 ceph-mon[30950]: pgmap v453: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 99 KiB/s wr, 30 op/s 2024-09-12T12:27:32.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:32 smithi171 ceph-mon[35098]: pgmap v453: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 99 KiB/s wr, 30 op/s 2024-09-12T12:27:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 -- 172.21.15.137:0/12823911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea4101350 msgr2=0x7efea410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 --2- 172.21.15.137:0/12823911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea4101350 0x7efea410d880 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009930 tx=0x7efe8c02a420 comp rx=0 tx=0).stop 2024-09-12T12:27:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 -- 172.21.15.137:0/12823911 shutdown_connections 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 --2- 172.21.15.137:0/12823911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea4101350 0x7efea410d880 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 --2- 172.21.15.137:0/12823911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efea41009b0 0x7efea4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 --2- 172.21.15.137:0/12823911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efea41069c0 0x7efea4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 -- 172.21.15.137:0/12823911 >> 172.21.15.137:0/12823911 conn(0x7efea4077990 msgr2=0x7efea40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 -- 172.21.15.137:0/12823911 shutdown_connections 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.960+0000 7efeaadd8640 1 -- 172.21.15.137:0/12823911 wait complete. 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 Processor -- start 2024-09-12T12:27:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 -- start start 2024-09-12T12:27:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efea41009b0 0x7efea4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efea4101350 0x7efea4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea41069c0 0x7efea41a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efea4110e00 con 0x7efea41069c0 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efea4110c80 con 0x7efea4101350 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efeaadd8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efea4110f80 con 0x7efea41009b0 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efea9375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea41069c0 0x7efea41a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efea9375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea41069c0 0x7efea41a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47586/0 (socket says 172.21.15.137:47586) 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efea9375640 1 -- 172.21.15.137:0/3918042951 learned_addr learned my addr 172.21.15.137:0/3918042951 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.961+0000 7efea3fff640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efea4101350 0x7efea4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea8b74640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efea41009b0 0x7efea4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea9375640 1 -- 172.21.15.137:0/3918042951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efea41009b0 msgr2=0x7efea4199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea9375640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efea41009b0 0x7efea4199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea9375640 1 -- 172.21.15.137:0/3918042951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efea4101350 msgr2=0x7efea4199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea9375640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efea4101350 0x7efea4199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea9375640 1 -- 172.21.15.137:0/3918042951 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea41a4700 con 0x7efea41069c0 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea9375640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea41069c0 0x7efea41a41c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efe8c005ab0 tx=0x7efe8c031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea1ffb640 1 -- 172.21.15.137:0/3918042951 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c03f030 con 0x7efea41069c0 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea1ffb640 1 -- 172.21.15.137:0/3918042951 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8c0043d0 con 0x7efea41069c0 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea41a48d0 con 0x7efea41069c0 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efea1ffb640 1 -- 172.21.15.137:0/3918042951 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c035070 con 0x7efea41069c0 2024-09-12T12:27:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.962+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea41a4d90 con 0x7efea41069c0 2024-09-12T12:27:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.964+0000 7efea1ffb640 1 -- 172.21.15.137:0/3918042951 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efe8c042030 con 0x7efea41069c0 2024-09-12T12:27:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.965+0000 7efea1ffb640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe900a2e20 0x7efe900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.965+0000 7efea8b74640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe900a2e20 0x7efe900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.965+0000 7efea1ffb640 1 -- 172.21.15.137:0/3918042951 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7efe8c0ef650 con 0x7efea41069c0 2024-09-12T12:27:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efea8b74640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe900a2e20 0x7efe900a52e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009b50 tx=0x7efe9c005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 Processor -- start 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 -- start start 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca45eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca469770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca46c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5647ca3c32f0 con 0x5647ca469cb0 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5647ca36b270 con 0x5647ca45eaf0 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efeadc4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5647ca3bf2b0 con 0x5647ca45f410 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efea9375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca46c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efea8b74640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca45eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efea8b74640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca45eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:45022/0 (socket says 172.21.15.137:45022) 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efea8b74640 1 -- 172.21.15.137:0/3086205558 learned_addr learned my addr 172.21.15.137:0/3086205558 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.966+0000 7efea3fff640 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca469770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efea8b74640 1 -- 172.21.15.137:0/3086205558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 msgr2=0x5647ca469770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efea8b74640 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca469770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efea8b74640 1 -- 172.21.15.137:0/3086205558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 msgr2=0x5647ca46c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efea8b74640 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca46c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efea8b74640 1 -- 172.21.15.137:0/3086205558 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ca36a340 con 0x5647ca45eaf0 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efea8b74640 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca45eed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005bc0 tx=0x7efe9c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b051b758c88c67c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efe9affd640 1 -- 172.21.15.137:0/3086205558 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c01e420 con 0x5647ca45eaf0 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efe9affd640 1 -- 172.21.15.137:0/3086205558 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9c01eaa0 con 0x5647ca45eaf0 2024-09-12T12:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.967+0000 7efe9affd640 1 -- 172.21.15.137:0/3086205558 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c01db70 con 0x5647ca45eaf0 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 -- 172.21.15.137:0/3086205558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 msgr2=0x5647ca45eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca45eed0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005bc0 tx=0x7efe9c014000 comp rx=0 tx=0).stop 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 -- 172.21.15.137:0/3086205558 shutdown_connections 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca46c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca469770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 --2- 172.21.15.137:0/3086205558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca45eed0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 -- 172.21.15.137:0/3086205558 >> 172.21.15.137:0/3086205558 conn(0x5647ca45b060 msgr2=0x5647ca45b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 -- 172.21.15.137:0/3086205558 shutdown_connections 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.968+0000 7efeadc4b740 1 -- 172.21.15.137:0/3086205558 wait complete. 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 -- start start 2024-09-12T12:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 Processor -- start 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca466be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca467120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca467660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5647ca46c880 con 0x5647ca469cb0 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5647ca34c290 con 0x5647ca45eaf0 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5647ca46ca00 con 0x5647ca45f410 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efeadc4b740 10 client.0 _ll_register_callbacks cb 0x5647ca37d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efea9375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca467660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efea8b74640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca466be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efea9375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca467660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47604/0 (socket says 172.21.15.137:47604) 2024-09-12T12:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efea9375640 1 -- 172.21.15.137:0/1601757254 learned_addr learned my addr 172.21.15.137:0/1601757254 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.969+0000 7efea3fff640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca467120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea8b74640 1 -- 172.21.15.137:0/1601757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 msgr2=0x5647ca467120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea8b74640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca467120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea8b74640 1 -- 172.21.15.137:0/1601757254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 msgr2=0x5647ca467660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea8b74640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca467660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea8b74640 1 -- 172.21.15.137:0/1601757254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647ca0d6210 con 0x5647ca45eaf0 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea9375640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca467660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea3fff640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca467120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efea8b74640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca466be0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005b90 tx=0x7efe9c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91a07ee7395744b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efe83fff640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efe83fff640 1 -- 172.21.15.137:0/1601757254 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c035060 con 0x5647ca45eaf0 2024-09-12T12:27:32.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe9c01eaa0 con 0x5647ca45eaf0 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efe83fff640 1 -- 172.21.15.137:0/1601757254 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7efe9c009050 con 0x5647ca45eaf0 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efe83fff640 1 -- 172.21.15.137:0/1601757254 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efe9c01d8f0 con 0x5647ca45eaf0 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efeadc4b740 10 client.26486 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5647ca1a2620 con 0x5647ca45eaf0 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.970+0000 7efe83fff640 1 -- 172.21.15.137:0/1601757254 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9c01eaa0 con 0x5647ca45eaf0 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efe83fff640 1 -- 172.21.15.137:0/1601757254 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7efe9c026a50 con 0x5647ca45eaf0 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efeadc4b740 10 client.26486 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efeadc4b740 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efeadc4b740 20 client.26486 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efeadc4b740 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca479c50 0x5647ca47c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efeadc4b740 4 client.26486 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647ca3d0a20 con 0x5647ca479c50 2024-09-12T12:27:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efea3fff640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca479c50 0x5647ca47c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efea3fff640 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca479c50 0x5647ca47c040 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96482c89dfa9e396 server_cookie=e2602fdaa84e5f00 in_seq=0 out_seq=0 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.971+0000 7efe83fff640 10 client.26486 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efe83fff640 1 -- 172.21.15.137:0/1601757254 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647ca3d0a20 con 0x5647ca479c50 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efe83fff640 10 client.26486 handle_command_reply: tid=0 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 client.26486 shutdown 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 msgr2=0x5647ca466be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca466be0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005b90 tx=0x7efe9c0056d0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 shutdown_connections 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca479c50 0x5647ca47c040 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca467660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efe83fff640 0 client.26486 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45f410 0x5647ca467120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 --2- 172.21.15.137:0/1601757254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca466be0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 >> 172.21.15.137:0/1601757254 conn(0x5647ca45b060 msgr2=0x5647ca45a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 shutdown_connections 2024-09-12T12:27:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.972+0000 7efeadc4b740 1 -- 172.21.15.137:0/1601757254 wait complete. 2024-09-12T12:27:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:32.973+0000 7efeadc4b740 20 client.26486 trim_cache size 0 max 16384 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 -- 172.21.15.137:0/4205702498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 msgr2=0x7f583c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 --2- 172.21.15.137:0/4205702498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c1067f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5830009900 tx=0x7f583002a410 comp rx=0 tx=0).stop 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 -- 172.21.15.137:0/4205702498 shutdown_connections 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 --2- 172.21.15.137:0/4205702498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c106f20 0x7f583c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 --2- 172.21.15.137:0/4205702498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c1067f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 --2- 172.21.15.137:0/4205702498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f583c101df0 0x7f583c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 -- 172.21.15.137:0/4205702498 >> 172.21.15.137:0/4205702498 conn(0x7f583c0fd8f0 msgr2=0x7f583c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 -- 172.21.15.137:0/4205702498 shutdown_connections 2024-09-12T12:27:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.918+0000 7f5843e9a640 1 -- 172.21.15.137:0/4205702498 wait complete. 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 Processor -- start 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 -- start start 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f583c101df0 0x7f583c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c106f20 0x7f583c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f583c111fc0 con 0x7f583c1027a0 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f583c111e40 con 0x7f583c106f20 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5843e9a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f583c112140 con 0x7f583c101df0 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5841435640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5841435640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47570/0 (socket says 172.21.15.137:47570) 2024-09-12T12:27:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5841435640 1 -- 172.21.15.137:0/75859440 learned_addr learned my addr 172.21.15.137:0/75859440 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.919+0000 7f5842437640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c106f20 0x7f583c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841c36640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f583c101df0 0x7f583c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841435640 1 -- 172.21.15.137:0/75859440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f583c101df0 msgr2=0x7f583c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841435640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f583c101df0 0x7f583c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841435640 1 -- 172.21.15.137:0/75859440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c106f20 msgr2=0x7f583c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841435640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c106f20 0x7f583c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841435640 1 -- 172.21.15.137:0/75859440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c19d470 con 0x7f583c1027a0 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841c36640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f583c101df0 0x7f583c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5841435640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c198a00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5830006010 tx=0x7f58300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f582effd640 1 -- 172.21.15.137:0/75859440 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583003f030 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f582effd640 1 -- 172.21.15.137:0/75859440 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5830003ce0 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583c19d700 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f582effd640 1 -- 172.21.15.137:0/75859440 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830035070 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.920+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583c1a5040 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.921+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f583c1021d0 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.922+0000 7f582effd640 1 -- 172.21.15.137:0/75859440 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5830042030 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.923+0000 7f582effd640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58280a2d70 0x7f58280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.923+0000 7f5841c36640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58280a2d70 0x7f58280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.924+0000 7f582effd640 1 -- 172.21.15.137:0/75859440 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f58300efa20 con 0x7f583c1027a0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.924+0000 7f5841c36640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58280a2d70 0x7f58280a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58380058c0 tx=0x7f5838005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:32.927+0000 7f582effd640 1 -- 172.21.15.137:0/75859440 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58300bc6c0 con 0x7f583c1027a0 2024-09-12T12:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 Processor -- start 2024-09-12T12:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 -- start start 2024-09-12T12:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45eaf0 0x5647ca4674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45f410 0x5647ca468b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca481670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efea41b0110 con 0x5647ca469cb0 2024-09-12T12:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efea41ad920 con 0x5647ca45f410 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efeadc4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efea41ab790 con 0x5647ca45eaf0 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.001+0000 7efea9375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca481670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca481670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47620/0 (socket says 172.21.15.137:47620) 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 -- 172.21.15.137:0/352257091 learned_addr learned my addr 172.21.15.137:0/352257091 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea3fff640 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45f410 0x5647ca468b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea8b74640 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45eaf0 0x5647ca4674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 -- 172.21.15.137:0/352257091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45eaf0 msgr2=0x5647ca4674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45eaf0 0x5647ca4674b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 -- 172.21.15.137:0/352257091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45f410 msgr2=0x5647ca468b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45f410 0x5647ca468b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 -- 172.21.15.137:0/352257091 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ca45e920 con 0x5647ca469cb0 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea3fff640 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45f410 0x5647ca468b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efea9375640 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca481670 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0bb940 tx=0x7efe8c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d32bd2f67b6e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efe9a7fc640 1 -- 172.21.15.137:0/352257091 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c049040 con 0x5647ca469cb0 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efe9a7fc640 1 -- 172.21.15.137:0/352257091 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8c0a6040 con 0x5647ca469cb0 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.002+0000 7efe9a7fc640 1 -- 172.21.15.137:0/352257091 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c031e30 con 0x5647ca469cb0 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 -- 172.21.15.137:0/352257091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 msgr2=0x5647ca481670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca481670 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0bb940 tx=0x7efe8c02a420 comp rx=0 tx=0).stop 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 -- 172.21.15.137:0/352257091 shutdown_connections 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca469cb0 0x5647ca481670 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45f410 0x5647ca468b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 --2- 172.21.15.137:0/352257091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca45eaf0 0x5647ca4674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 -- 172.21.15.137:0/352257091 >> 172.21.15.137:0/352257091 conn(0x5647ca45b060 msgr2=0x5647ca468440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 -- 172.21.15.137:0/352257091 shutdown_connections 2024-09-12T12:27:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 -- 172.21.15.137:0/352257091 wait complete. 2024-09-12T12:27:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 -- start start 2024-09-12T12:27:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.003+0000 7efeadc4b740 1 Processor -- start 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca471c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca45f410 0x5647ca4721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efeadc4b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca469cb0 0x5647ca463280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efeadc4b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efea41abde0 con 0x5647ca45f410 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efeadc4b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efea41ad250 con 0x5647ca45eaf0 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efeadc4b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efea41ac4b0 con 0x5647ca469cb0 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efeadc4b740 10 client.0 _ll_register_callbacks cb 0x7efe8c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca45f410 0x5647ca4721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca45f410 0x5647ca4721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47632/0 (socket says 172.21.15.137:47632) 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 -- 172.21.15.137:0/396182644 learned_addr learned my addr 172.21.15.137:0/396182644 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea8b74640 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca471c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 -- 172.21.15.137:0/396182644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca469cb0 msgr2=0x5647ca463280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca469cb0 0x5647ca463280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 -- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 msgr2=0x5647ca471c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca471c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.004+0000 7efea3fff640 1 -- 172.21.15.137:0/396182644 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647ca4637c0 con 0x5647ca45f410 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efea3fff640 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca45f410 0x5647ca4721d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x5647ca45bec0 tx=0x7efe94003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec614ae61d7f938 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efe83fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efe83fff640 1 -- 172.21.15.137:0/396182644 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9402a060 con 0x5647ca45f410 2024-09-12T12:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efe83fff640 1 -- 172.21.15.137:0/396182644 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7efe94004450 con 0x5647ca45f410 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe8c0a6040 con 0x5647ca45f410 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efe83fff640 1 -- 172.21.15.137:0/396182644 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efe94004650 con 0x5647ca45f410 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 10 client.17199 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5647ca4e3e30 con 0x5647ca45f410 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efe83fff640 1 -- 172.21.15.137:0/396182644 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8c0a6040 con 0x5647ca45f410 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efe83fff640 1 -- 172.21.15.137:0/396182644 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7efe94024030 con 0x5647ca45f410 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 10 client.17199 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 20 client.17199 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca4e4640 0x5647ca4e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 4 client.17199 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.005+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647ca3cad70 con 0x5647ca4e4640 2024-09-12T12:27:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.006+0000 7efea8b74640 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca4e4640 0x5647ca4e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.006+0000 7efea8b74640 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca4e4640 0x5647ca4e6a60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dfbfa121bce2142 server_cookie=27e00ffdaef403fa in_seq=0 out_seq=0 2024-09-12T12:27:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.006+0000 7efe83fff640 10 client.17199 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:32 smithi189 ceph-mon[35475]: pgmap v453: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 99 KiB/s wr, 30 op/s 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.104+0000 7efe83fff640 1 -- 172.21.15.137:0/396182644 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647ca3cad70 con 0x5647ca4e4640 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.104+0000 7efe83fff640 10 client.17199 handle_command_reply: tid=0 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.104+0000 7efeadc4b740 1 client.17199 shutdown 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca45f410 msgr2=0x5647ca4721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca45f410 0x5647ca4721d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x5647ca45bec0 tx=0x7efe94003e00 comp rx=0 tx=0).stop 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 shutdown_connections 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5647ca469cb0 0x5647ca463280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5647ca4e4640 0x5647ca4e6a60 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647ca45f410 0x5647ca4721d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 --2- 172.21.15.137:0/396182644 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5647ca45eaf0 0x5647ca471c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efe83fff640 0 client.17199 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:33.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 >> 172.21.15.137:0/396182644 conn(0x5647ca45b060 msgr2=0x5647ca4599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 shutdown_connections 2024-09-12T12:27:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 1 -- 172.21.15.137:0/396182644 wait complete. 2024-09-12T12:27:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeadc4b740 20 client.17199 trim_cache size 0 max 16384 2024-09-12T12:27:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:27:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:27:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:27:33.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:27:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe900a2e20 msgr2=0x7efe900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeaadd8640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe900a2e20 0x7efe900a52e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009b50 tx=0x7efe9c005f90 comp rx=0 tx=0).stop 2024-09-12T12:27:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea41069c0 msgr2=0x7efea41a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.105+0000 7efeaadd8640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea41069c0 0x7efea41a41c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efe8c005ab0 tx=0x7efe8c031b50 comp rx=0 tx=0).stop 2024-09-12T12:27:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 shutdown_connections 2024-09-12T12:27:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe900a2e20 0x7efe900a52e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efea41069c0 0x7efea41a41c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efea4101350 0x7efea4199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 --2- 172.21.15.137:0/3918042951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efea41009b0 0x7efea4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 >> 172.21.15.137:0/3918042951 conn(0x7efea4077990 msgr2=0x7efea4103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 shutdown_connections 2024-09-12T12:27:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:33.106+0000 7efeaadd8640 1 -- 172.21.15.137:0/3918042951 wait complete. 2024-09-12T12:27:33.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.169+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f583c199820 con 0x7f583c1027a0 2024-09-12T12:27:33.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.170+0000 7f582effd640 1 -- 172.21.15.137:0/75859440 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11945 (secure 0 0 0) 0x7f583c199820 con 0x7f583c1027a0 2024-09-12T12:27:33.170 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:27:33.170 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":45,"btime":"2024-09-12T12:27:07:316870+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":45,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:27:07.316855+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26518":{"gid":26518,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3604136981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3604136981},{"type":"v1","addr":"172.21.15.189:6835","nonce":3604136981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:27:33.170 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 45 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58280a2d70 msgr2=0x7f58280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58280a2d70 0x7f58280a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58380058c0 tx=0x7f5838005850 comp rx=0 tx=0).stop 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 msgr2=0x7f583c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c198a00 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5830006010 tx=0x7f58300042f0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 shutdown_connections 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58280a2d70 0x7f58280a5230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c106f20 0x7f583c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f583c1027a0 0x7f583c198a00 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 --2- 172.21.15.137:0/75859440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f583c101df0 0x7f583c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 >> 172.21.15.137:0/75859440 conn(0x7f583c0fd8f0 msgr2=0x7f583c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:33.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 shutdown_connections 2024-09-12T12:27:33.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:33.172+0000 7f5843e9a640 1 -- 172.21.15.137:0/75859440 wait complete. 2024-09-12T12:27:33.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 5.68 MB/sec warmup 65 sec latency 28089.489 ms 2024-09-12T12:27:33.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:33 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/75859440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:33.878 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 63946e6c --release --await' 2024-09-12T12:27:33.878 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 63946e6c --release --await 2024-09-12T12:27:33.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:33 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/75859440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:34.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:33 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/75859440' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:27:34.171 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:27:34.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3357 5.59 MB/sec warmup 66 sec latency 29089.767 ms 2024-09-12T12:27:34.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 -- 172.21.15.137:0/358428663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0106f10 msgr2=0x7f0ec010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 --2- 172.21.15.137:0/358428663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0106f10 0x7f0ec010ea90 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009900 tx=0x7f0eb802a410 comp rx=0 tx=0).stop 2024-09-12T12:27:34.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 -- 172.21.15.137:0/358428663 shutdown_connections 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 --2- 172.21.15.137:0/358428663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0106f10 0x7f0ec010ea90 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 --2- 172.21.15.137:0/358428663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0102790 0x7f0ec01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 --2- 172.21.15.137:0/358428663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ec0101de0 0x7f0ec01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 -- 172.21.15.137:0/358428663 >> 172.21.15.137:0/358428663 conn(0x7f0ec00fd8c0 msgr2=0x7f0ec00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 -- 172.21.15.137:0/358428663 shutdown_connections 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.707+0000 7f0ec499b640 1 -- 172.21.15.137:0/358428663 wait complete. 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 Processor -- start 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 -- start start 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101de0 0x7f0ec00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ec0102790 0x7f0ec0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0106f10 0x7f0ec0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0111f70 con 0x7f0ec0101de0 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0111df0 con 0x7f0ec0102790 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ec499b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01120f0 con 0x7f0ec0106f10 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101de0 0x7f0ec00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101de0 0x7f0ec00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47654/0 (socket says 172.21.15.137:47654) 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.708+0000 7f0ebe59c640 1 -- 172.21.15.137:0/3413644498 learned_addr learned my addr 172.21.15.137:0/3413644498 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ebed9d640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0106f10 0x7f0ec0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0eb7fff640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ec0102790 0x7f0ec0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ebe59c640 1 -- 172.21.15.137:0/3413644498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0106f10 msgr2=0x7f0ec0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0106f10 0x7f0ec0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ebe59c640 1 -- 172.21.15.137:0/3413644498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ec0102790 msgr2=0x7f0ec0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ec0102790 0x7f0ec0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ebe59c640 1 -- 172.21.15.137:0/3413644498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec01a6910 con 0x7f0ec0101de0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101de0 0x7f0ec00744f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000eab0 tx=0x7f0eb000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0eb77fe640 1 -- 172.21.15.137:0/3413644498 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000ce40 con 0x7f0ec0101de0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0eb77fe640 1 -- 172.21.15.137:0/3413644498 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0011950 con 0x7f0ec0101de0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0eb77fe640 1 -- 172.21.15.137:0/3413644498 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0010ae0 con 0x7f0ec0101de0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01a6c00 con 0x7f0ec0101de0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.709+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a70f0 con 0x7f0ec0101de0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0eb77fe640 1 -- 172.21.15.137:0/3413644498 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0eb0011b00 con 0x7f0ec0101de0 2024-09-12T12:27:34.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 Processor -- start 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 -- start start 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7060 0x5586d3cc7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7980 0x5586d3cd1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3cd4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5586d3c2c620 con 0x5586d3cc7980 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5586d3bd3e50 con 0x5586d3cd2240 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586d3c285e0 con 0x5586d3cc7060 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7980 0x5586d3cd1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7980 0x5586d3cd1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47666/0 (socket says 172.21.15.137:47666) 2024-09-12T12:27:34.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1773934716 learned_addr learned my addr 172.21.15.137:0/1773934716 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0eb77fe640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2cf0 0x7f0eac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.712+0000 7f0ebed9d640 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7060 0x5586d3cc7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0eb7fff640 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3cd4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1773934716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7060 msgr2=0x5586d3cc7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7060 0x5586d3cc7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebed9d640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2cf0 0x7f0eac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1773934716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 msgr2=0x5586d3cd4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3cd4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1773934716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d3bc4d40 con 0x5586d3cc7980 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0eb7fff640 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3cd4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:27:34.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0eb77fe640 1 -- 172.21.15.137:0/3413644498 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f0eb00cfa50 con 0x7f0ec0101de0 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7980 0x5586d3cd1d00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0098960 tx=0x7f0eb0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8ded1ded5fa0cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0ebed9d640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2cf0 0x7f0eac0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8031550 tx=0x7f0eb802a410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0eb4ff9640 1 -- 172.21.15.137:0/1773934716 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb001c0c0 con 0x5586d3cc7980 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0eb4ff9640 1 -- 172.21.15.137:0/1773934716 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb009c690 con 0x5586d3cc7980 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.713+0000 7f0eb4ff9640 1 -- 172.21.15.137:0/1773934716 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00a0340 con 0x5586d3cc7980 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1773934716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7980 msgr2=0x5586d3cd1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7980 0x5586d3cd1d00 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0098960 tx=0x7f0eb0021000 comp rx=0 tx=0).stop 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1773934716 shutdown_connections 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3cd4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7980 0x5586d3cd1d00 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1773934716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7060 0x5586d3cc7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1773934716 >> 172.21.15.137:0/1773934716 conn(0x5586d3cc35f0 msgr2=0x5586d3cc39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1773934716 shutdown_connections 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1773934716 wait complete. 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 -- start start 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.714+0000 7f0ec6d74740 1 Processor -- start 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3ccf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3ccf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3ccfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5586d3cd4e10 con 0x5586d3cc7060 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5586d3bb4f10 con 0x5586d3cd2240 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5586d3cd4f90 con 0x5586d3cc7980 2024-09-12T12:27:34.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 10 client.0 _ll_register_callbacks cb 0x5586d3bd8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3ccf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3ccf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47676/0 (socket says 172.21.15.137:47676) 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 -- 172.21.15.137:0/696807379 learned_addr learned my addr 172.21.15.137:0/696807379 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0eb7fff640 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3ccf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 -- 172.21.15.137:0/696807379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 msgr2=0x5586d3ccf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3ccf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 -- 172.21.15.137:0/696807379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 msgr2=0x5586d3ccfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3ccfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 -- 172.21.15.137:0/696807379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d393b580 con 0x5586d3cc7060 2024-09-12T12:27:34.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3ccf160 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00cd270 tx=0x7f0eb00988c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f4649892206cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/696807379 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00a4c00 con 0x5586d3cc7060 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/696807379 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f0eb0010c80 con 0x5586d3cc7060 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb009c690 con 0x5586d3cc7060 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/696807379 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0eb00a0340 con 0x5586d3cc7060 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 10 client.17211 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.715+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5586d3a05960 con 0x5586d3cc7060 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/696807379 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb009c690 con 0x5586d3cc7060 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/696807379 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f0eb000f040 con 0x5586d3cc7060 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ec6d74740 10 client.17211 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:34.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ec6d74740 10 client.17211 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ec6d74740 20 client.17211 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3ce1c10 0x5586d3ce4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ec6d74740 4 client.17211 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586d3c39b90 con 0x5586d3ce1c10 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.716+0000 7f0eb7fff640 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3ce1c10 0x5586d3ce4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ea5ffb640 10 client.17211 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0eb7fff640 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3ce1c10 0x5586d3ce4000 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d397997fbd69fa0 server_cookie=54bb50e6588cb3a4 in_seq=0 out_seq=0 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/696807379 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586d3c39b90 con 0x5586d3ce1c10 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ea5ffb640 10 client.17211 handle_command_reply: tid=0 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 client.17211 shutdown 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 msgr2=0x5586d3ccf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3ccf160 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00cd270 tx=0x7f0eb00988c0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 shutdown_connections 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3ccfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3ccf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3ce1c10 0x5586d3ce4000 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/696807379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3ccf160 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ea5ffb640 0 client.17211 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.717+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 >> 172.21.15.137:0/696807379 conn(0x5586d3cc35f0 msgr2=0x5586d3cc9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.718+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 shutdown_connections 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.718+0000 7f0ec6d74740 1 -- 172.21.15.137:0/696807379 wait complete. 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.718+0000 7f0ec6d74740 20 client.17211 trim_cache size 0 max 16384 2024-09-12T12:27:34.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.734+0000 7f0ec6d74740 1 Processor -- start 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.734+0000 7f0ec6d74740 1 -- start start 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3cc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3cc9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3ce95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01b22b0 con 0x5586d3cc7060 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01afac0 con 0x5586d3cd2240 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01ad930 con 0x5586d3cc7980 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3cc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3cc2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47682/0 (socket says 172.21.15.137:47682) 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1301291316 learned_addr learned my addr 172.21.15.137:0/1301291316 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:34.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1301291316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 msgr2=0x5586d3cc9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/1301291316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3cc9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1301291316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 msgr2=0x5586d3ce95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/1301291316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3ce95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 -- 172.21.15.137:0/1301291316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586d3cc9f70 con 0x5586d3cc7060 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/1301291316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3cc2a10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00cd7c0 tx=0x7f0eb009adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1364ea283efebec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ea7fff640 1 -- 172.21.15.137:0/1301291316 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb00a4c00 con 0x5586d3cc7060 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ea7fff640 1 -- 172.21.15.137:0/1301291316 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb009bf90 con 0x5586d3cc7060 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.735+0000 7f0ea7fff640 1 -- 172.21.15.137:0/1301291316 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb009c1f0 con 0x5586d3cc7060 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1301291316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 msgr2=0x5586d3cc2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1301291316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3cc2a10 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0eb00cd7c0 tx=0x7f0eb009adc0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1301291316 shutdown_connections 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1301291316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cd2240 0x5586d3ce95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1301291316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3cc9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/1301291316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cc7060 0x5586d3cc2a10 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1301291316 >> 172.21.15.137:0/1301291316 conn(0x5586d3cc35f0 msgr2=0x5586d3cd09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1301291316 shutdown_connections 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 -- 172.21.15.137:0/1301291316 wait complete. 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 -- start start 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.736+0000 7f0ec6d74740 1 Processor -- start 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cc7060 0x5586d3cda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3cda930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cd2240 0x5586d3cd71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01adf80 con 0x5586d3cd2240 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01af3f0 con 0x5586d3cc7060 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec01ae650 con 0x5586d3cc7980 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 10 client.0 _ll_register_callbacks cb 0x7f0eb009ad10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cd2240 0x5586d3cd71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cd2240 0x5586d3cd71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47690/0 (socket says 172.21.15.137:47690) 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 -- 172.21.15.137:0/634218279 learned_addr learned my addr 172.21.15.137:0/634218279 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0eb7fff640 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3cda930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cc7060 0x5586d3cda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 -- 172.21.15.137:0/634218279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 msgr2=0x5586d3cda930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3cda930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 -- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cc7060 msgr2=0x5586d3cda3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cc7060 0x5586d3cda3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 -- 172.21.15.137:0/634218279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586d3cd7a10 con 0x5586d3cd2240 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ebed9d640 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cd2240 0x5586d3cd71e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8033000 tx=0x7f0eb80042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9eacd64c507d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ea5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/634218279 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb804c030 con 0x5586d3cd2240 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb009bf90 con 0x5586d3cd2240 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/634218279 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f0eb8005020 con 0x5586d3cd2240 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/634218279 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0eb8056050 con 0x5586d3cd2240 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 10 client.17217 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:34.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.737+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5586d3d4d0e0 con 0x5586d3cd2240 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/634218279 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb009bf90 con 0x5586d3cd2240 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/634218279 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f0eb80047a0 con 0x5586d3cd2240 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ec6d74740 10 client.17217 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ec6d74740 10 client.17217 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ec6d74740 20 client.17217 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3d4d940 0x5586d3d4fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ec6d74740 4 client.17217 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "63946e6c", "release": true, "await": true}] 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "63946e6c", "release": true, "await": true}) -- 0x5586d3c340a0 con 0x5586d3d4d940 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3d4d940 0x5586d3d4fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ebe59c640 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3d4d940 0x5586d3d4fd30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd48cd05782cb71 server_cookie=f35d5df520155396 in_seq=0 out_seq=0 2024-09-12T12:27:34.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.738+0000 7f0ea5ffb640 10 client.17217 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:34.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ea5ffb640 1 -- 172.21.15.137:0/634218279 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5586d3c340a0 con 0x5586d3d4d940 2024-09-12T12:27:34.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ea5ffb640 10 client.17217 handle_command_reply: tid=0 2024-09-12T12:27:34.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 client.17217 shutdown 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cd2240 msgr2=0x5586d3cd71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cd2240 0x5586d3cd71e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8033000 tx=0x7f0eb80042c0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 shutdown_connections 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5586d3d4d940 0x5586d3d4fd30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586d3cd2240 0x5586d3cd71e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5586d3cc7980 0x5586d3cda930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 --2- 172.21.15.137:0/634218279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586d3cc7060 0x5586d3cda3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ea5ffb640 0 client.17217 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 >> 172.21.15.137:0/634218279 conn(0x5586d3cc35f0 msgr2=0x5586d3ccf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 shutdown_connections 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 1 -- 172.21.15.137:0/634218279 wait complete. 2024-09-12T12:27:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.818+0000 7f0ec6d74740 20 client.17217 trim_cache size 0 max 16384 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 45, 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 56, 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "63946e6c": { 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 56, 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:27:34.820 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 150.0, 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:34.821 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:34.822 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:34.822 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:34.822 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:27:34.822 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:27:34.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2cf0 msgr2=0x7f0eac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2cf0 0x7f0eac0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8031550 tx=0x7f0eb802a410 comp rx=0 tx=0).stop 2024-09-12T12:27:34.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101de0 msgr2=0x7f0ec00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101de0 0x7f0ec00744f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000eab0 tx=0x7f0eb000ef70 comp rx=0 tx=0).stop 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 shutdown_connections 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0eac0a2cf0 0x7f0eac0a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ec0106f10 0x7f0ec0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ec0102790 0x7f0ec0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 --2- 172.21.15.137:0/3413644498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ec0101de0 0x7f0ec00744f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 >> 172.21.15.137:0/3413644498 conn(0x7f0ec00fd8c0 msgr2=0x7f0ec00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 shutdown_connections 2024-09-12T12:27:34.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:27:34.819+0000 7f0ec499b640 1 -- 172.21.15.137:0/3413644498 wait complete. 2024-09-12T12:27:34.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:34 smithi171 ceph-mon[35098]: pgmap v454: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 99 KiB/s wr, 30 op/s 2024-09-12T12:27:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:34 smithi137 ceph-mon[30950]: pgmap v454: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 99 KiB/s wr, 30 op/s 2024-09-12T12:27:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:34 smithi189 ceph-mon[35475]: pgmap v454: 97 pgs: 97 active+clean; 477 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 99 KiB/s wr, 30 op/s 2024-09-12T12:27:35.216 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3446 5.54 MB/sec warmup 67 sec latency 29743.237 ms 2024-09-12T12:27:35.578 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '63946e6c', total seconds elapsed: 33.3 2024-09-12T12:27:36.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3685 5.68 MB/sec warmup 68 sec latency 96.334 ms 2024-09-12T12:27:36.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:36 smithi171 ceph-mon[35098]: pgmap v455: 97 pgs: 97 active+clean; 480 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 416 KiB/s wr, 65 op/s 2024-09-12T12:27:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:36 smithi137 ceph-mon[30950]: pgmap v455: 97 pgs: 97 active+clean; 480 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 416 KiB/s wr, 65 op/s 2024-09-12T12:27:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:36 smithi189 ceph-mon[35475]: pgmap v455: 97 pgs: 97 active+clean; 480 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 416 KiB/s wr, 65 op/s 2024-09-12T12:27:37.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 3887 5.61 MB/sec warmup 69 sec latency 119.969 ms 2024-09-12T12:27:38.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 4096 5.58 MB/sec warmup 70 sec latency 330.292 ms 2024-09-12T12:27:38.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:38 smithi171 ceph-mon[35098]: pgmap v456: 97 pgs: 97 active+clean; 480 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 328 KiB/s wr, 38 op/s 2024-09-12T12:27:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:38 smithi137 ceph-mon[30950]: pgmap v456: 97 pgs: 97 active+clean; 480 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 328 KiB/s wr, 38 op/s 2024-09-12T12:27:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:38 smithi189 ceph-mon[35475]: pgmap v456: 97 pgs: 97 active+clean; 480 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 328 KiB/s wr, 38 op/s 2024-09-12T12:27:39.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 4513 5.74 MB/sec warmup 71 sec latency 123.700 ms 2024-09-12T12:27:40.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 5101 6.13 MB/sec warmup 72 sec latency 65.459 ms 2024-09-12T12:27:40.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:40 smithi171 ceph-mon[35098]: pgmap v457: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 183 op/s 2024-09-12T12:27:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:40 smithi137 ceph-mon[30950]: pgmap v457: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 183 op/s 2024-09-12T12:27:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:40 smithi189 ceph-mon[35475]: pgmap v457: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 183 op/s 2024-09-12T12:27:41.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 5796 6.41 MB/sec warmup 73 sec latency 56.062 ms 2024-09-12T12:27:42.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 6415 7.04 MB/sec warmup 74 sec latency 196.641 ms 2024-09-12T12:27:42.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:42 smithi171 ceph-mon[35098]: pgmap v458: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 179 op/s 2024-09-12T12:27:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:42 smithi137 ceph-mon[30950]: pgmap v458: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 179 op/s 2024-09-12T12:27:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:42 smithi189 ceph-mon[35475]: pgmap v458: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 179 op/s 2024-09-12T12:27:43.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 6797 7.37 MB/sec warmup 75 sec latency 1196.667 ms 2024-09-12T12:27:44.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 6983 7.37 MB/sec warmup 76 sec latency 2196.717 ms 2024-09-12T12:27:44.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:44 smithi137 ceph-mon[30950]: pgmap v459: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 179 op/s 2024-09-12T12:27:44.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:44 smithi171 ceph-mon[35098]: pgmap v459: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 179 op/s 2024-09-12T12:27:45.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:44 smithi189 ceph-mon[35475]: pgmap v459: 97 pgs: 97 active+clean; 502 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 179 op/s 2024-09-12T12:27:45.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 7167 7.29 MB/sec warmup 77 sec latency 3196.744 ms 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 -- 172.21.15.137:0/2335836579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c6010c0f0 msgr2=0x7f4c6010e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 --2- 172.21.15.137:0/2335836579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c6010c0f0 0x7f4c6010e570 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009ca0 tx=0x7f4c5802a950 comp rx=0 tx=0).stop 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 -- 172.21.15.137:0/2335836579 shutdown_connections 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 --2- 172.21.15.137:0/2335836579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c6010c0f0 0x7f4c6010e570 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 --2- 172.21.15.137:0/2335836579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c601035c0 0x7f4c6010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 --2- 172.21.15.137:0/2335836579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c60102ca0 0x7f4c60103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 -- 172.21.15.137:0/2335836579 >> 172.21.15.137:0/2335836579 conn(0x7f4c600fc6b0 msgr2=0x7f4c600fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 -- 172.21.15.137:0/2335836579 shutdown_connections 2024-09-12T12:27:46.205 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.202+0000 7f4c65991640 1 -- 172.21.15.137:0/2335836579 wait complete. 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 Processor -- start 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 -- start start 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c60102ca0 0x7f4c601982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c601035c0 0x7f4c60198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c6010c0f0 0x7f4c6019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60111ed0 con 0x7f4c601035c0 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60111d50 con 0x7f4c6010c0f0 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c65991640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c60112050 con 0x7f4c60102ca0 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c601035c0 0x7f4c60198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c601035c0 0x7f4c60198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33152/0 (socket says 172.21.15.137:33152) 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c60102ca0 0x7f4c601982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 -- 172.21.15.137:0/2822192874 learned_addr learned my addr 172.21.15.137:0/2822192874 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c5f7fe640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c6010c0f0 0x7f4c6019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 -- 172.21.15.137:0/2822192874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c60102ca0 msgr2=0x7f4c601982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c60102ca0 0x7f4c601982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 -- 172.21.15.137:0/2822192874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c6010c0f0 msgr2=0x7f4c6019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c6010c0f0 0x7f4c6019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.203+0000 7f4c57fff640 1 -- 172.21.15.137:0/2822192874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6019d2d0 con 0x7f4c601035c0 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.204+0000 7f4c57fff640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c601035c0 0x7f4c60198820 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e9f0 tx=0x7f4c4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.204+0000 7f4c6498f640 1 -- 172.21.15.137:0/2822192874 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4800ccc0 con 0x7f4c601035c0 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.204+0000 7f4c6498f640 1 -- 172.21.15.137:0/2822192874 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c48010910 con 0x7f4c601035c0 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.204+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6019d5c0 con 0x7f4c601035c0 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.204+0000 7f4c6498f640 1 -- 172.21.15.137:0/2822192874 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48018920 con 0x7f4c601035c0 2024-09-12T12:27:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.204+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c601a4e50 con 0x7f4c601035c0 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6498f640 1 -- 172.21.15.137:0/2822192874 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4c48010430 con 0x7f4c601035c0 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 Processor -- start 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 -- start start 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf872f0 0x55a6daf95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf960c0 0x55a6daf984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6daeeb2f0 con 0x55a6daf960c0 2024-09-12T12:27:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a6dae93270 con 0x55a6daf869d0 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c6880a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a6daee72b0 con 0x55a6daf872f0 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf960c0 0x55a6daf984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf872f0 0x55a6daf95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf872f0 0x55a6daf95b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:33372/0 (socket says 172.21.15.137:33372) 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c5effd640 1 -- 172.21.15.137:0/313579685 learned_addr learned my addr 172.21.15.137:0/313579685 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.207+0000 7f4c5f7fe640 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c5effd640 1 -- 172.21.15.137:0/313579685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 msgr2=0x55a6daf86db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c5effd640 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf86db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c5effd640 1 -- 172.21.15.137:0/313579685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf960c0 msgr2=0x55a6daf984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c5effd640 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf960c0 0x55a6daf984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c5effd640 1 -- 172.21.15.137:0/313579685 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6dae92340 con 0x55a6daf872f0 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c6498f640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4c4c0a2e20 0x7f4c4c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c5effd640 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf872f0 0x55a6daf95b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d20 tx=0x7f4c50011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7ae4e71ea8a6044 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c5f7fe640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4c4c0a2e20 0x7f4c4c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c6498f640 1 -- 172.21.15.137:0/2822192874 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f4c480cf570 con 0x7f4c601035c0 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c55ffb640 1 -- 172.21.15.137:0/313579685 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001e070 con 0x55a6daf872f0 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c55ffb640 1 -- 172.21.15.137:0/313579685 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50019b10 con 0x55a6daf872f0 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.208+0000 7f4c55ffb640 1 -- 172.21.15.137:0/313579685 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000eb80 con 0x55a6daf872f0 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 -- 172.21.15.137:0/313579685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf872f0 msgr2=0x55a6daf95b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c5f7fe640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4c4c0a2e20 0x7f4c4c0a52e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c58005e30 tx=0x7f4c58005c50 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf872f0 0x55a6daf95b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009d20 tx=0x7f4c50011810 comp rx=0 tx=0).stop 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 -- 172.21.15.137:0/313579685 shutdown_connections 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf960c0 0x55a6daf984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf872f0 0x55a6daf95b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 --2- 172.21.15.137:0/313579685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf86db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 -- 172.21.15.137:0/313579685 >> 172.21.15.137:0/313579685 conn(0x55a6daf83060 msgr2=0x55a6daf83440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 -- 172.21.15.137:0/313579685 shutdown_connections 2024-09-12T12:27:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 -- 172.21.15.137:0/313579685 wait complete. 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 -- start start 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.209+0000 7f4c6880a740 1 Processor -- start 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf93010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf93550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf93a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a6daf98cf0 con 0x55a6daf872f0 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a6dae74290 con 0x55a6daf869d0 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a6daf98e70 con 0x55a6daf960c0 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 10 client.0 _ll_register_callbacks cb 0x55a6daea5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf93550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf93010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf93550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33182/0 (socket says 172.21.15.137:33182) 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c5f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf93a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 -- 172.21.15.137:0/874093241 learned_addr learned my addr 172.21.15.137:0/874093241 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 -- 172.21.15.137:0/874093241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 msgr2=0x55a6daf93a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf93a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 -- 172.21.15.137:0/874093241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 msgr2=0x55a6daf93010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf93010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 -- 172.21.15.137:0/874093241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6dabfe210 con 0x55a6daf872f0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c57fff640 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf93550 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4c4809b860 tx=0x7f4c480d2cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b774dd15d500711 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c3effd640 1 -- 172.21.15.137:0/874093241 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4809b4c0 con 0x55a6daf872f0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c3effd640 1 -- 172.21.15.137:0/874093241 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f4c480d1b60 con 0x55a6daf872f0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c50019b10 con 0x55a6daf872f0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c3effd640 1 -- 172.21.15.137:0/874093241 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4c4809bb60 con 0x55a6daf872f0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 10 client.17229 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.210+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a6dacca620 con 0x55a6daf872f0 2024-09-12T12:27:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c3effd640 1 -- 172.21.15.137:0/874093241 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c50019b10 con 0x55a6daf872f0 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c3effd640 1 -- 172.21.15.137:0/874093241 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4c480cf0d0 con 0x55a6daf872f0 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c6880a740 10 client.17229 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c6880a740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c6880a740 20 client.17229 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c6880a740 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6dafa6130 0x55a6dafa8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c6880a740 4 client.17229 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6daef8a20 con 0x55a6dafa6130 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c5effd640 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6dafa6130 0x55a6dafa8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c3effd640 10 client.17229 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.211+0000 7f4c5effd640 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6dafa6130 0x55a6dafa8580 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35817599369a450e server_cookie=9b3db5ed73f189ef in_seq=0 out_seq=0 2024-09-12T12:27:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c3effd640 1 -- 172.21.15.137:0/874093241 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6daef8a20 con 0x55a6dafa6130 2024-09-12T12:27:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c3effd640 10 client.17229 handle_command_reply: tid=0 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 client.17229 shutdown 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 msgr2=0x55a6daf93550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf93550 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4c4809b860 tx=0x7f4c480d2cb0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 shutdown_connections 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf93a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6dafa6130 0x55a6dafa8580 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf93550 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 --2- 172.21.15.137:0/874093241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf93010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c3effd640 0 client.17229 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 >> 172.21.15.137:0/874093241 conn(0x55a6daf83060 msgr2=0x55a6daf88dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 shutdown_connections 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 1 -- 172.21.15.137:0/874093241 wait complete. 2024-09-12T12:27:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.214+0000 7f4c6880a740 20 client.17229 trim_cache size 0 max 16384 2024-09-12T12:27:46.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 7527 7.36 MB/sec warmup 78 sec latency 4196.771 ms 2024-09-12T12:27:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 Processor -- start 2024-09-12T12:27:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 -- start start 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf869d0 0x55a6dafade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf872f0 0x55a6dafae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf94870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601ade60 con 0x55a6daf869d0 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601ab670 con 0x55a6daf872f0 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.229+0000 7f4c6880a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a9500 con 0x55a6daf960c0 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf869d0 0x55a6dafade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf869d0 0x55a6dafade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33198/0 (socket says 172.21.15.137:33198) 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 -- 172.21.15.137:0/2660089095 learned_addr learned my addr 172.21.15.137:0/2660089095 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c57fff640 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf872f0 0x55a6dafae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5f7fe640 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf94870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 -- 172.21.15.137:0/2660089095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 msgr2=0x55a6daf94870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf94870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 -- 172.21.15.137:0/2660089095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf872f0 msgr2=0x55a6dafae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf872f0 0x55a6dafae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 -- 172.21.15.137:0/2660089095 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6dafa3110 con 0x55a6daf869d0 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c5effd640 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf869d0 0x55a6dafade90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009ab0 tx=0x7f4c5000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9fa0edc9811f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c557fa640 1 -- 172.21.15.137:0/2660089095 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50033070 con 0x55a6daf869d0 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c557fa640 1 -- 172.21.15.137:0/2660089095 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5001f040 con 0x55a6daf869d0 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.230+0000 7f4c557fa640 1 -- 172.21.15.137:0/2660089095 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001e040 con 0x55a6daf869d0 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 -- 172.21.15.137:0/2660089095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf869d0 msgr2=0x55a6dafade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf869d0 0x55a6dafade90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009ab0 tx=0x7f4c5000e820 comp rx=0 tx=0).stop 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 -- 172.21.15.137:0/2660089095 shutdown_connections 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf94870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf872f0 0x55a6dafae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 --2- 172.21.15.137:0/2660089095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf869d0 0x55a6dafade90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 -- 172.21.15.137:0/2660089095 >> 172.21.15.137:0/2660089095 conn(0x55a6daf83060 msgr2=0x55a6daf8e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 -- 172.21.15.137:0/2660089095 shutdown_connections 2024-09-12T12:27:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 -- 172.21.15.137:0/2660089095 wait complete. 2024-09-12T12:27:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 -- start start 2024-09-12T12:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.231+0000 7f4c6880a740 1 Processor -- start 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf9e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf8fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf8ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601a9b50 con 0x55a6daf872f0 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601aafa0 con 0x55a6daf869d0 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4c601aa200 con 0x55a6daf960c0 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 10 client.0 _ll_register_callbacks cb 0x7f4c480d2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf8fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf8fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33214/0 (socket says 172.21.15.137:33214) 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 -- 172.21.15.137:0/3447321563 learned_addr learned my addr 172.21.15.137:0/3447321563 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c5f7fe640 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf8ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c5effd640 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf9e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 -- 172.21.15.137:0/3447321563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 msgr2=0x55a6daf8ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf8ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 -- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 msgr2=0x55a6daf9e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf9e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 -- 172.21.15.137:0/3447321563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6daf90490 con 0x55a6daf872f0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c57fff640 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf8fa10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800c6a0 tx=0x7f4c4800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7893b79671dbac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c3effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c3effd640 1 -- 172.21.15.137:0/3447321563 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4809b4c0 con 0x55a6daf872f0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c3effd640 1 -- 172.21.15.137:0/3447321563 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7f4c48097ab0 con 0x55a6daf872f0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5001f040 con 0x55a6daf872f0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c3effd640 1 -- 172.21.15.137:0/3447321563 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4c480a4720 con 0x55a6daf872f0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 10 client.17235 fetch_fsmap learned FSMap version 45 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.232+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a6db010b20 con 0x55a6daf872f0 2024-09-12T12:27:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c3effd640 1 -- 172.21.15.137:0/3447321563 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c5001f040 con 0x55a6daf872f0 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c3effd640 1 -- 172.21.15.137:0/3447321563 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f4c4809ab90 con 0x55a6daf872f0 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c6880a740 10 client.17235 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c6880a740 10 client.17235 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c6880a740 20 client.17235 populate_metadata read hostname 'smithi137' 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c6880a740 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6db011130 0x55a6db013550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c6880a740 4 client.17235 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6daef2d70 con 0x55a6db011130 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c5effd640 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6db011130 0x55a6db013550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c3effd640 10 client.17235 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:46.233+0000 7f4c5effd640 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6db011130 0x55a6db013550 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832cf18e6a9bcf24 server_cookie=b221319579fa999f in_seq=0 out_seq=0 2024-09-12T12:27:46.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:46 smithi171 ceph-mon[35098]: pgmap v460: 97 pgs: 97 active+clean; 514 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 293 op/s 2024-09-12T12:27:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:46 smithi137 ceph-mon[30950]: pgmap v460: 97 pgs: 97 active+clean; 514 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 293 op/s 2024-09-12T12:27:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:46 smithi189 ceph-mon[35475]: pgmap v460: 97 pgs: 97 active+clean; 514 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.7 MiB/s wr, 293 op/s 2024-09-12T12:27:47.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 8014 7.63 MB/sec warmup 79 sec latency 5196.797 ms 2024-09-12T12:27:48.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 8670 7.87 MB/sec warmup 80 sec latency 6196.823 ms 2024-09-12T12:27:48.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:48 smithi171 ceph-mon[35098]: pgmap v461: 97 pgs: 97 active+clean; 514 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.4 MiB/s wr, 258 op/s 2024-09-12T12:27:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:48 smithi137 ceph-mon[30950]: pgmap v461: 97 pgs: 97 active+clean; 514 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.4 MiB/s wr, 258 op/s 2024-09-12T12:27:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:48 smithi189 ceph-mon[35475]: pgmap v461: 97 pgs: 97 active+clean; 514 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.4 MiB/s wr, 258 op/s 2024-09-12T12:27:49.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 9241 8.29 MB/sec warmup 81 sec latency 7196.850 ms 2024-09-12T12:27:50.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 9604 8.58 MB/sec warmup 82 sec latency 8196.875 ms 2024-09-12T12:27:50.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:50 smithi171 ceph-mon[35098]: pgmap v462: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 503 op/s 2024-09-12T12:27:51.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:50 smithi137 ceph-mon[30950]: pgmap v462: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 503 op/s 2024-09-12T12:27:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:50 smithi189 ceph-mon[35475]: pgmap v462: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.8 MiB/s wr, 503 op/s 2024-09-12T12:27:51.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 9825 8.58 MB/sec warmup 83 sec latency 9196.911 ms 2024-09-12T12:27:52.160 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.159+0000 7f4c3effd640 1 -- 172.21.15.137:0/3447321563 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6daef2d70 con 0x55a6db011130 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.159+0000 7f4c3effd640 10 client.17235 handle_command_reply: tid=0 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.159+0000 7f4c6880a740 1 client.17235 shutdown 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 msgr2=0x55a6daf8fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf8fa10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800c6a0 tx=0x7f4c4800ced0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 shutdown_connections 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6daf960c0 0x55a6daf8ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a6db011130 0x55a6db013550 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6daf872f0 0x55a6daf8fa10 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.161 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 --2- 172.21.15.137:0/3447321563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6daf869d0 0x55a6daf9e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 >> 172.21.15.137:0/3447321563 conn(0x55a6daf83060 msgr2=0x55a6daf819a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c3effd640 0 client.17235 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:27:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 shutdown_connections 2024-09-12T12:27:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 1 -- 172.21.15.137:0/3447321563 wait complete. 2024-09-12T12:27:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.160+0000 7f4c6880a740 20 client.17235 trim_cache size 0 max 16384 2024-09-12T12:27:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:27:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:27:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:27:52.163 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:27:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4c4c0a2e20 msgr2=0x7f4c4c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4c4c0a2e20 0x7f4c4c0a52e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4c58005e30 tx=0x7f4c58005c50 comp rx=0 tx=0).stop 2024-09-12T12:27:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c601035c0 msgr2=0x7f4c60198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:27:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c601035c0 0x7f4c60198820 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800e9f0 tx=0x7f4c4800eeb0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 shutdown_connections 2024-09-12T12:27:52.164 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4c4c0a2e20 0x7f4c4c0a52e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c6010c0f0 0x7f4c6019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c601035c0 0x7f4c60198820 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 --2- 172.21.15.137:0/2822192874 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c60102ca0 0x7f4c601982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:27:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 >> 172.21.15.137:0/2822192874 conn(0x7f4c600fc6b0 msgr2=0x7f4c600fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:27:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 shutdown_connections 2024-09-12T12:27:52.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:27:52.161+0000 7f4c65991640 1 -- 172.21.15.137:0/2822192874 wait complete. 2024-09-12T12:27:52.217 INFO:tasks.workunit.client.0.smithi137.stdout: 10 10009 8.51 MB/sec warmup 84 sec latency 10196.959 ms 2024-09-12T12:27:53.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:52 smithi137 ceph-mon[30950]: pgmap v463: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-12T12:27:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:52 smithi189 ceph-mon[35475]: pgmap v463: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-12T12:27:53.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:52 smithi171 ceph-mon[35098]: pgmap v463: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-12T12:27:53.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 10665 8.79 MB/sec warmup 85 sec latency 10204.712 ms 2024-09-12T12:27:54.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 11543 9.19 MB/sec warmup 86 sec latency 72.924 ms 2024-09-12T12:27:55.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:54 smithi137 ceph-mon[30950]: pgmap v464: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-12T12:27:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:54 smithi189 ceph-mon[35475]: pgmap v464: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-12T12:27:55.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:54 smithi171 ceph-mon[35098]: pgmap v464: 97 pgs: 97 active+clean; 527 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-12T12:27:55.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 12338 9.59 MB/sec warmup 87 sec latency 44.608 ms 2024-09-12T12:27:56.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 13144 10.10 MB/sec warmup 88 sec latency 42.756 ms 2024-09-12T12:27:57.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:56 smithi137 ceph-mon[30950]: pgmap v465: 97 pgs: 97 active+clean; 524 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 495 op/s 2024-09-12T12:27:57.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:56 smithi189 ceph-mon[35475]: pgmap v465: 97 pgs: 97 active+clean; 524 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 495 op/s 2024-09-12T12:27:57.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:56 smithi171 ceph-mon[35098]: pgmap v465: 97 pgs: 97 active+clean; 524 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 495 op/s 2024-09-12T12:27:57.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 13826 10.33 MB/sec warmup 89 sec latency 35.514 ms 2024-09-12T12:27:58.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 14670 10.66 MB/sec warmup 90 sec latency 51.699 ms 2024-09-12T12:27:59.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:27:58 smithi137 ceph-mon[30950]: pgmap v466: 97 pgs: 97 active+clean; 524 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 381 op/s 2024-09-12T12:27:59.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:27:58 smithi189 ceph-mon[35475]: pgmap v466: 97 pgs: 97 active+clean; 524 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 381 op/s 2024-09-12T12:27:59.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:27:58 smithi171 ceph-mon[35098]: pgmap v466: 97 pgs: 97 active+clean; 524 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 381 op/s 2024-09-12T12:27:59.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 15561 11.16 MB/sec warmup 91 sec latency 42.330 ms 2024-09-12T12:28:00.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16251 11.42 MB/sec warmup 92 sec latency 132.360 ms 2024-09-12T12:28:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:00 smithi137 ceph-mon[30950]: pgmap v467: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2024-09-12T12:28:01.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:00 smithi189 ceph-mon[35475]: pgmap v467: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2024-09-12T12:28:01.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:00 smithi171 ceph-mon[35098]: pgmap v467: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 598 op/s 2024-09-12T12:28:01.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16251 11.30 MB/sec warmup 93 sec latency 1132.394 ms 2024-09-12T12:28:02.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16251 11.18 MB/sec warmup 94 sec latency 2132.447 ms 2024-09-12T12:28:03.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:02 smithi171 ceph-mon[35098]: pgmap v468: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-12T12:28:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:02 smithi137 ceph-mon[30950]: pgmap v468: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-12T12:28:03.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:02 smithi189 ceph-mon[35475]: pgmap v468: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-12T12:28:03.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16295 11.08 MB/sec warmup 95 sec latency 2823.319 ms 2024-09-12T12:28:04.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16296 10.97 MB/sec warmup 96 sec latency 1296.012 ms 2024-09-12T12:28:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:04 smithi137 ceph-mon[30950]: pgmap v469: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-12T12:28:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:04 smithi189 ceph-mon[35475]: pgmap v469: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-12T12:28:05.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:04 smithi171 ceph-mon[35098]: pgmap v469: 97 pgs: 97 active+clean; 553 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 353 op/s 2024-09-12T12:28:05.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16296 10.86 MB/sec warmup 97 sec latency 2296.089 ms 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 -- 172.21.15.137:0/2933323724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 msgr2=0x7fbf0410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/2933323724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0410e550 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009ca0 tx=0x7fbefc02a950 comp rx=0 tx=0).stop 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 -- 172.21.15.137:0/2933323724 shutdown_connections 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/2933323724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0410e550 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/2933323724 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf041035a0 0x7fbf0410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/2933323724 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf04102c80 0x7fbf04103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 -- 172.21.15.137:0/2933323724 >> 172.21.15.137:0/2933323724 conn(0x7fbf04077990 msgr2=0x7fbf040fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 -- 172.21.15.137:0/2933323724 shutdown_connections 2024-09-12T12:28:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.259+0000 7fbf0a14b640 1 -- 172.21.15.137:0/2933323724 wait complete. 2024-09-12T12:28:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 Processor -- start 2024-09-12T12:28:05.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 -- start start 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf04102c80 0x7fbf041982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf041035a0 0x7fbf04198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04111ee0 con 0x7fbf0410c0d0 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04111d60 con 0x7fbf041035a0 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf0a14b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04112060 con 0x7fbf04102c80 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40414/0 (socket says 172.21.15.137:40414) 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf03fff640 1 -- 172.21.15.137:0/518133885 learned_addr learned my addr 172.21.15.137:0/518133885 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbefaffd640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf041035a0 0x7fbf04198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.260+0000 7fbf037fe640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf04102c80 0x7fbf041982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf03fff640 1 -- 172.21.15.137:0/518133885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf04102c80 msgr2=0x7fbf041982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf03fff640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf04102c80 0x7fbf041982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf03fff640 1 -- 172.21.15.137:0/518133885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf041035a0 msgr2=0x7fbf04198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf03fff640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf041035a0 0x7fbf04198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf03fff640 1 -- 172.21.15.137:0/518133885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0419d2b0 con 0x7fbf0410c0d0 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf037fe640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf04102c80 0x7fbf041982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf03fff640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0419cb80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009dc0 tx=0x7fbefc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf017fa640 1 -- 172.21.15.137:0/518133885 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc034040 con 0x7fbf0410c0d0 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf017fa640 1 -- 172.21.15.137:0/518133885 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbefc02fe20 con 0x7fbf0410c0d0 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0419d540 con 0x7fbf0410c0d0 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf017fa640 1 -- 172.21.15.137:0/518133885 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0047f0 con 0x7fbf0410c0d0 2024-09-12T12:28:05.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.261+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a4e30 con 0x7fbf0410c0d0 2024-09-12T12:28:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 Processor -- start 2024-09-12T12:28:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf017fa640 1 -- 172.21.15.137:0/518133885 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbefc037020 con 0x7fbf0410c0d0 2024-09-12T12:28:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 -- start start 2024-09-12T12:28:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e86397ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863989b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e86398c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e8638df2f0 con 0x55e86398a0c0 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e863887270 con 0x55e86397b2d0 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.264+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e8638db2b0 con 0x55e86397a9b0 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e86398c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e86398c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40418/0 (socket says 172.21.15.137:40418) 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 -- 172.21.15.137:0/2727638345 learned_addr learned my addr 172.21.15.137:0/2727638345 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863989b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e86397ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbf017fa640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbef00a2e20 0x7fbef00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 -- 172.21.15.137:0/2727638345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 msgr2=0x55e86397ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e86397ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 -- 172.21.15.137:0/2727638345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 msgr2=0x55e863989b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863989b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 -- 172.21.15.137:0/2727638345 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e863886340 con 0x55e86398a0c0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863989b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbf03fff640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbef00a2e20 0x7fbef00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e86398c4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0126f0 tx=0x7fbeec00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f697a2ace0175c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbf017fa640 1 -- 172.21.15.137:0/518133885 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7fbefc0f01d0 con 0x7fbf0410c0d0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbef9ffb640 1 -- 172.21.15.137:0/2727638345 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec012c70 con 0x55e86398a0c0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbef9ffb640 1 -- 172.21.15.137:0/2727638345 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeec012e10 con 0x55e86398a0c0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.265+0000 7fbef9ffb640 1 -- 172.21.15.137:0/2727638345 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004a00 con 0x55e86398a0c0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2727638345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 msgr2=0x55e86398c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf03fff640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbef00a2e20 0x7fbef00a52e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0b8c80 tx=0x7fbefc0b8420 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e86398c4b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0126f0 tx=0x7fbeec00b6d0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2727638345 shutdown_connections 2024-09-12T12:28:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e86398c4b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863989b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2727638345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e86397ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2727638345 >> 172.21.15.137:0/2727638345 conn(0x55e863977060 msgr2=0x55e863977440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2727638345 shutdown_connections 2024-09-12T12:28:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.266+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2727638345 wait complete. 2024-09-12T12:28:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 -- start start 2024-09-12T12:28:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 Processor -- start 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e863987030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863987570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863987ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e86398cc90 con 0x55e86398a0c0 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e863868290 con 0x55e86397b2d0 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e86398ce10 con 0x55e86397a9b0 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf0cfc1740 10 client.0 _ll_register_callbacks cb 0x55e863899140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863987ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863987ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40428/0 (socket says 172.21.15.137:40428) 2024-09-12T12:28:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbefaffd640 1 -- 172.21.15.137:0/2659521568 learned_addr learned my addr 172.21.15.137:0/2659521568 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e863987030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.267+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863987570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbefaffd640 1 -- 172.21.15.137:0/2659521568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 msgr2=0x55e863987030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e863987030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbefaffd640 1 -- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 msgr2=0x55e863987570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863987570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbefaffd640 1 -- 172.21.15.137:0/2659521568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8635f2210 con 0x55e86398a0c0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863987ab0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbeec001eb0 tx=0x7fbeec014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee803a6d0efa7149 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2659521568 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004050 con 0x55e86398a0c0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2659521568 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7fbeec0041f0 con 0x55e86398a0c0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeec012e10 con 0x55e86398a0c0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2659521568 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbeec033050 con 0x55e86398a0c0 2024-09-12T12:28:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbf0cfc1740 10 client.17247 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e8636be620 con 0x55e86398a0c0 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.268+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2659521568 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeec012e10 con 0x55e86398a0c0 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2659521568 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fbeec02b020 con 0x55e86398a0c0 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf0cfc1740 10 client.17247 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf0cfc1740 10 client.17247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf0cfc1740 20 client.17247 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863999f40 0x55e86399c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf0cfc1740 4 client.17247 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8638eca20 con 0x55e863999f40 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863999f40 0x55e86399c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbee2ffd640 10 client.17247 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.269+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863999f40 0x55e86399c330 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0c568f61e047a1 server_cookie=96c92e4c88daedd3 in_seq=0 out_seq=0 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2659521568 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8638eca20 con 0x55e863999f40 2024-09-12T12:28:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbee2ffd640 10 client.17247 handle_command_reply: tid=0 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 client.17247 shutdown 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 msgr2=0x55e863987ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863987ab0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbeec001eb0 tx=0x7fbeec014040 comp rx=0 tx=0).stop 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 shutdown_connections 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863999f40 0x55e86399c330 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863987ab0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397b2d0 0x55e863987570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbee2ffd640 0 client.17247 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2659521568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397a9b0 0x55e863987030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 >> 172.21.15.137:0/2659521568 conn(0x55e863977060 msgr2=0x55e86397cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 shutdown_connections 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.270+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2659521568 wait complete. 2024-09-12T12:28:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.271+0000 7fbf0cfc1740 20 client.17247 trim_cache size 0 max 16384 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 Processor -- start 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 -- start start 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e8639a1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e8639a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863984270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ade30 con 0x55e86398a0c0 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ab640 con 0x55e86397a9b0 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a94b0 con 0x55e86397b2d0 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863984270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863984270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40440/0 (socket says 172.21.15.137:40440) 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.285+0000 7fbefaffd640 1 -- 172.21.15.137:0/2065066289 learned_addr learned my addr 172.21.15.137:0/2065066289 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e8639a1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e8639a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbefaffd640 1 -- 172.21.15.137:0/2065066289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 msgr2=0x55e8639a20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e8639a20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbefaffd640 1 -- 172.21.15.137:0/2065066289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 msgr2=0x55e8639a1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e8639a1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbefaffd640 1 -- 172.21.15.137:0/2065066289 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86397b100 con 0x55e86398a0c0 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863984270 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbeec006010 tx=0x7fbeec0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d041fbcff7049a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbef97fa640 1 -- 172.21.15.137:0/2065066289 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00ba00 con 0x55e86398a0c0 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbef97fa640 1 -- 172.21.15.137:0/2065066289 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeec005610 con 0x55e86398a0c0 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.286+0000 7fbef97fa640 1 -- 172.21.15.137:0/2065066289 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec012110 con 0x55e86398a0c0 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2065066289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 msgr2=0x55e863984270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863984270 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fbeec006010 tx=0x7fbeec0055a0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2065066289 shutdown_connections 2024-09-12T12:28:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863984270 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e8639a20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2065066289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e8639a1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2065066289 >> 172.21.15.137:0/2065066289 conn(0x55e863977060 msgr2=0x55e8639890b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2065066289 shutdown_connections 2024-09-12T12:28:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2065066289 wait complete. 2024-09-12T12:28:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 -- start start 2024-09-12T12:28:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.287+0000 7fbf0cfc1740 1 Processor -- start 2024-09-12T12:28:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e86398d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e86398db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863a02b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041a9b00 con 0x55e86398a0c0 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041aaf70 con 0x55e86397a9b0 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041aa1d0 con 0x55e86397b2d0 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 10 client.0 _ll_register_callbacks cb 0x7fbef400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863a02b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863a02b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40450/0 (socket says 172.21.15.137:40450) 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 -- 172.21.15.137:0/2454006790 learned_addr learned my addr 172.21.15.137:0/2454006790 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e86398d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e86398db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 -- 172.21.15.137:0/2454006790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 msgr2=0x55e86398db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e86398db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 -- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 msgr2=0x55e86398d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e86398d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 -- 172.21.15.137:0/2454006790 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e863a03130 con 0x55e86398a0c0 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbefaffd640 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863a02b20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00bc10 tx=0x7fbeec017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a62412767be68a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2454006790 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec03d040 con 0x55e86398a0c0 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2454006790 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8484+0+0 (secure 0 0 0) 0x7fbeec010740 con 0x55e86398a0c0 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.288+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeec005610 con 0x55e86398a0c0 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2454006790 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbeec012310 con 0x55e86398a0c0 2024-09-12T12:28:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 10 client.17253 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e863a04420 con 0x55e86398a0c0 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2454006790 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeec005610 con 0x55e86398a0c0 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2454006790 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fbeec02b070 con 0x55e86398a0c0 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 10 client.17253 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 20 client.17253 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863a04a50 0x55e863a06e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 4 client.17253 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8638e6d70 con 0x55e863a04a50 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.289+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863a04a50 0x55e863a06e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.290+0000 7fbf037fe640 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863a04a50 0x55e863a06e70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d0eb01d4a7b8c0 server_cookie=48c543c87ab682b3 in_seq=0 out_seq=0 2024-09-12T12:28:05.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.290+0000 7fbee2ffd640 10 client.17253 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbee2ffd640 1 -- 172.21.15.137:0/2454006790 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8638e6d70 con 0x55e863a04a50 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbee2ffd640 10 client.17253 handle_command_reply: tid=0 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 client.17253 shutdown 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 msgr2=0x55e863a02b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863a02b20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00bc10 tx=0x7fbeec017040 comp rx=0 tx=0).stop 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 shutdown_connections 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e863a04a50 0x55e863a06e70 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e86398a0c0 0x55e863a02b20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbee2ffd640 0 client.17253 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e86397b2d0 0x55e86398db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 --2- 172.21.15.137:0/2454006790 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e86397a9b0 0x55e86398d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 >> 172.21.15.137:0/2454006790 conn(0x55e863977060 msgr2=0x55e86397c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 shutdown_connections 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 1 -- 172.21.15.137:0/2454006790 wait complete. 2024-09-12T12:28:05.502 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.500+0000 7fbf0cfc1740 20 client.17253 trim_cache size 0 max 16384 2024-09-12T12:28:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:28:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:28:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:28:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbef00a2e20 msgr2=0x7fbef00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbef00a2e20 0x7fbef00a52e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0b8c80 tx=0x7fbefc0b8420 comp rx=0 tx=0).stop 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 msgr2=0x7fbf0419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0419cb80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbefc009dc0 tx=0x7fbefc003f10 comp rx=0 tx=0).stop 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 shutdown_connections 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fbef00a2e20 0x7fbef00a52e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf0410c0d0 0x7fbf0419cb80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf041035a0 0x7fbf04198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.504 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 --2- 172.21.15.137:0/518133885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf04102c80 0x7fbf041982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 >> 172.21.15.137:0/518133885 conn(0x7fbf04077990 msgr2=0x7fbf040fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 shutdown_connections 2024-09-12T12:28:05.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:05.501+0000 7fbf0a14b640 1 -- 172.21.15.137:0/518133885 wait complete. 2024-09-12T12:28:06.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16296 10.74 MB/sec warmup 98 sec latency 3296.181 ms 2024-09-12T12:28:07.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:06 smithi137 ceph-mon[30950]: pgmap v470: 97 pgs: 97 active+clean; 558 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.0 MiB/s wr, 425 op/s 2024-09-12T12:28:07.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:06 smithi189 ceph-mon[35475]: pgmap v470: 97 pgs: 97 active+clean; 558 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.0 MiB/s wr, 425 op/s 2024-09-12T12:28:07.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:06 smithi171 ceph-mon[35098]: pgmap v470: 97 pgs: 97 active+clean; 558 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.0 MiB/s wr, 425 op/s 2024-09-12T12:28:07.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16296 10.64 MB/sec warmup 99 sec latency 4296.289 ms 2024-09-12T12:28:08.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16315 10.53 MB/sec warmup 100 sec latency 4982.047 ms 2024-09-12T12:28:09.052 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:08 smithi137 ceph-mon[30950]: pgmap v471: 97 pgs: 97 active+clean; 558 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-09-12T12:28:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:08 smithi189 ceph-mon[35475]: pgmap v471: 97 pgs: 97 active+clean; 558 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-09-12T12:28:09.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:08 smithi171 ceph-mon[35098]: pgmap v471: 97 pgs: 97 active+clean; 558 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-09-12T12:28:09.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 16461 10.52 MB/sec warmup 101 sec latency 1254.118 ms 2024-09-12T12:28:10.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 17030 10.69 MB/sec warmup 102 sec latency 83.346 ms 2024-09-12T12:28:11.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:10 smithi137 ceph-mon[30950]: pgmap v472: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 13 MiB/s wr, 656 op/s 2024-09-12T12:28:11.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:10 smithi189 ceph-mon[35475]: pgmap v472: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 13 MiB/s wr, 656 op/s 2024-09-12T12:28:11.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:10 smithi171 ceph-mon[35098]: pgmap v472: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 13 MiB/s wr, 656 op/s 2024-09-12T12:28:11.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 17978 11.06 MB/sec warmup 103 sec latency 68.810 ms 2024-09-12T12:28:12.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 18911 11.50 MB/sec warmup 104 sec latency 43.785 ms 2024-09-12T12:28:13.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:12 smithi137 ceph-mon[30950]: pgmap v473: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 439 op/s 2024-09-12T12:28:13.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:12 smithi189 ceph-mon[35475]: pgmap v473: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 439 op/s 2024-09-12T12:28:13.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:12 smithi171 ceph-mon[35098]: pgmap v473: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 439 op/s 2024-09-12T12:28:13.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 19414 11.69 MB/sec warmup 105 sec latency 76.499 ms 2024-09-12T12:28:14.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:13 smithi137 ceph-mon[30950]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T12:28:14.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:13 smithi189 ceph-mon[35475]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T12:28:14.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:13 smithi171 ceph-mon[35098]: osdmap e107: 12 total, 12 up, 12 in 2024-09-12T12:28:14.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 19878 11.73 MB/sec warmup 106 sec latency 87.971 ms 2024-09-12T12:28:15.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:14 smithi137 ceph-mon[30950]: pgmap v475: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 527 op/s 2024-09-12T12:28:15.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:14 smithi189 ceph-mon[35475]: pgmap v475: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 527 op/s 2024-09-12T12:28:15.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:14 smithi171 ceph-mon[35098]: pgmap v475: 97 pgs: 97 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 527 op/s 2024-09-12T12:28:15.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 20925 12.10 MB/sec warmup 107 sec latency 30.731 ms 2024-09-12T12:28:16.218 INFO:tasks.workunit.client.0.smithi137.stdout: 10 21860 12.53 MB/sec warmup 108 sec latency 39.609 ms 2024-09-12T12:28:17.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:16 smithi137 ceph-mon[30950]: pgmap v476: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 809 op/s 2024-09-12T12:28:17.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:16 smithi189 ceph-mon[35475]: pgmap v476: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 809 op/s 2024-09-12T12:28:17.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:16 smithi171 ceph-mon[35098]: pgmap v476: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 809 op/s 2024-09-12T12:28:17.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 22592 12.79 MB/sec warmup 109 sec latency 43.009 ms 2024-09-12T12:28:18.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 23421 13.05 MB/sec warmup 110 sec latency 45.054 ms 2024-09-12T12:28:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.609+0000 7f7e8f528640 1 -- 172.21.15.137:0/3106258389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e881069e0 msgr2=0x7f7e88106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.610 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.609+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3106258389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e881069e0 0x7f7e88106dc0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400d940 tx=0x7f7e8402c3b0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.609+0000 7f7e8f528640 1 -- 172.21.15.137:0/3106258389 shutdown_connections 2024-09-12T12:28:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.609+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3106258389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e88101370 0x7f7e8810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.609+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3106258389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e881009d0 0x7f7e88100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.609+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3106258389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e881069e0 0x7f7e88106dc0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.609+0000 7f7e8f528640 1 -- 172.21.15.137:0/3106258389 >> 172.21.15.137:0/3106258389 conn(0x7f7e880fc6f0 msgr2=0x7f7e880feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.610+0000 7f7e8f528640 1 -- 172.21.15.137:0/3106258389 shutdown_connections 2024-09-12T12:28:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.610+0000 7f7e8f528640 1 -- 172.21.15.137:0/3106258389 wait complete. 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.610+0000 7f7e8f528640 1 Processor -- start 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.610+0000 7f7e8f528640 1 -- start start 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8f528640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e881009d0 0x7f7e8819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8f528640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e88101370 0x7f7e8819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8f528640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e881069e0 0x7f7e8819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8f528640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8810fd60 con 0x7f7e88101370 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8f528640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8810fbe0 con 0x7f7e881009d0 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8f528640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8810fee0 con 0x7f7e881069e0 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e88101370 0x7f7e8819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e881009d0 0x7f7e8819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e881009d0 0x7f7e8819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:42764/0 (socket says 172.21.15.137:42764) 2024-09-12T12:28:18.613 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e88101370 0x7f7e8819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48100/0 (socket says 172.21.15.137:48100) 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 -- 172.21.15.137:0/3101440216 learned_addr learned my addr 172.21.15.137:0/3101440216 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 -- 172.21.15.137:0/3101440216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e881069e0 msgr2=0x7f7e8819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e881069e0 0x7f7e8819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 -- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e88101370 msgr2=0x7f7e8819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e88101370 0x7f7e8819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.611+0000 7f7e8d2c4640 1 -- 172.21.15.137:0/3101440216 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8819f530 con 0x7f7e881009d0 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.612+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e881009d0 0x7f7e8819a540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005ec0 tx=0x7f7e840346d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.612+0000 7f7e7e7fc640 1 -- 172.21.15.137:0/3101440216 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84034ad0 con 0x7f7e881009d0 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.612+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8819f7c0 con 0x7f7e881009d0 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.612+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881a70b0 con 0x7f7e881009d0 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.612+0000 7f7e7e7fc640 1 -- 172.21.15.137:0/3101440216 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84041070 con 0x7f7e881009d0 2024-09-12T12:28:18.614 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.612+0000 7f7e7e7fc640 1 -- 172.21.15.137:0/3101440216 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84033620 con 0x7f7e881009d0 2024-09-12T12:28:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.615+0000 7f7e922b6740 1 Processor -- start 2024-09-12T12:28:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e922b6740 1 -- start start 2024-09-12T12:28:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2654df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2667f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e7e7fc640 1 -- 172.21.15.137:0/3101440216 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7e8400c050 con 0x7f7e881009d0 2024-09-12T12:28:18.616 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d266a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e922b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653d25b92f0 con 0x5653d26684c0 2024-09-12T12:28:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e922b6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5653d2561270 con 0x5653d2655330 2024-09-12T12:28:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e922b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5653d25b52b0 con 0x5653d2654a10 2024-09-12T12:28:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d266a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e8d2c4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2667f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d266a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48114/0 (socket says 172.21.15.137:48114) 2024-09-12T12:28:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e8cac3640 1 -- 172.21.15.137:0/1855645554 learned_addr learned my addr 172.21.15.137:0/1855645554 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:18.618 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.616+0000 7f7e8dac5640 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2654df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8cac3640 1 -- 172.21.15.137:0/1855645554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 msgr2=0x5653d2654df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2654df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8cac3640 1 -- 172.21.15.137:0/1855645554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 msgr2=0x5653d2667f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2667f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8cac3640 1 -- 172.21.15.137:0/1855645554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d2560340 con 0x5653d26684c0 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8dac5640 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2654df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e7e7fc640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7e740a2ea0 0x7f7e740a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d266a8b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800bcb0 tx=0x7f7e780107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f43809b346cd67 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e8dac5640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7e740a2ea0 0x7f7e740a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.617+0000 7f7e7e7fc640 1 -- 172.21.15.137:0/3101440216 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7f7e840eeac0 con 0x7f7e881009d0 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.618+0000 7f7e737fe640 1 -- 172.21.15.137:0/1855645554 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78004030 con 0x5653d26684c0 2024-09-12T12:28:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.618+0000 7f7e737fe640 1 -- 172.21.15.137:0/1855645554 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e780041d0 con 0x5653d26684c0 2024-09-12T12:28:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.618+0000 7f7e737fe640 1 -- 172.21.15.137:0/1855645554 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78011800 con 0x5653d26684c0 2024-09-12T12:28:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 -- 172.21.15.137:0/1855645554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 msgr2=0x5653d266a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e8dac5640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7e740a2ea0 0x7f7e740a5360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e80005fd0 tx=0x7f7e80009b10 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d266a8b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800bcb0 tx=0x7f7e780107a0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 -- 172.21.15.137:0/1855645554 shutdown_connections 2024-09-12T12:28:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d266a8b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.623 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2667f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 --2- 172.21.15.137:0/1855645554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2654df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 -- 172.21.15.137:0/1855645554 >> 172.21.15.137:0/1855645554 conn(0x5653d2651060 msgr2=0x5653d2651440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 -- 172.21.15.137:0/1855645554 shutdown_connections 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 -- 172.21.15.137:0/1855645554 wait complete. 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 -- start start 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.619+0000 7f7e922b6740 1 Processor -- start 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2665300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2665840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d2665d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e922b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5653d266b0e0 con 0x5653d26684c0 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e922b6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5653d2542290 con 0x5653d2655330 2024-09-12T12:28:18.624 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e922b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5653d266b260 con 0x5653d2654a10 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e922b6740 10 client.0 _ll_register_callbacks cb 0x5653d2573140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8dac5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d2665d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2665840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8dac5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d2665d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48120/0 (socket says 172.21.15.137:48120) 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8dac5640 1 -- 172.21.15.137:0/3210016036 learned_addr learned my addr 172.21.15.137:0/3210016036 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8d2c4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2665300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8cac3640 1 -- 172.21.15.137:0/3210016036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 msgr2=0x5653d2665300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2665300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8cac3640 1 -- 172.21.15.137:0/3210016036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 msgr2=0x5653d2665d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.620+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d2665d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e8cac3640 1 -- 172.21.15.137:0/3210016036 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d22cc210 con 0x5653d2655330 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e8dac5640 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d2665d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2665840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e78011010 tx=0x7f7e78003db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d0dcf71772777f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:18.625 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e70ff9640 1 -- 172.21.15.137:0/3210016036 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78004380 con 0x5653d2655330 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e780041d0 con 0x5653d2655330 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e70ff9640 1 -- 172.21.15.137:0/3210016036 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7f7e780035f0 con 0x5653d2655330 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e70ff9640 1 -- 172.21.15.137:0/3210016036 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7e780037f0 con 0x5653d2655330 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e922b6740 10 client.26546 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5653d2398620 con 0x5653d2655330 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.621+0000 7f7e70ff9640 1 -- 172.21.15.137:0/3210016036 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e780041d0 con 0x5653d2655330 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e70ff9640 1 -- 172.21.15.137:0/3210016036 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f7e780047e0 con 0x5653d2655330 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e922b6740 10 client.26546 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e922b6740 10 client.26546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e922b6740 20 client.26546 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26788f0 0x5653d267ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.626 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e922b6740 4 client.26546 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653d25c6a20 con 0x5653d26788f0 2024-09-12T12:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26788f0 0x5653d267ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e70ff9640 10 client.26546 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:18.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.622+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26788f0 0x5653d267ad50 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0dd0d643dc2cd9 server_cookie=558063c76a2d4c78 in_seq=0 out_seq=0 2024-09-12T12:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e70ff9640 1 -- 172.21.15.137:0/3210016036 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653d25c6a20 con 0x5653d26788f0 2024-09-12T12:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e70ff9640 10 client.26546 handle_command_reply: tid=0 2024-09-12T12:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 client.26546 shutdown 2024-09-12T12:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 msgr2=0x5653d2665840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2665840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e78011010 tx=0x7f7e78003db0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 shutdown_connections 2024-09-12T12:28:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26788f0 0x5653d267ad50 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d26684c0 0x5653d2665d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d2655330 0x5653d2665840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3210016036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2665300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e70ff9640 0 client.26546 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.623+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 >> 172.21.15.137:0/3210016036 conn(0x5653d2651060 msgr2=0x5653d2656d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.624+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 shutdown_connections 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.624+0000 7f7e922b6740 1 -- 172.21.15.137:0/3210016036 wait complete. 2024-09-12T12:28:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.624+0000 7f7e922b6740 20 client.26546 trim_cache size 0 max 16384 2024-09-12T12:28:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 Processor -- start 2024-09-12T12:28:18.645 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 -- start start 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2667270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d2680620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d2680b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c011060 con 0x5653d2655330 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c00e870 con 0x5653d26684c0 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e922b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c00c6e0 con 0x5653d2654a10 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d2680620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d2680620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48130/0 (socket says 172.21.15.137:48130) 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.645+0000 7f7e8cac3640 1 -- 172.21.15.137:0/3863158905 learned_addr learned my addr 172.21.15.137:0/3863158905 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:18.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e8dac5640 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d2680b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e8cac3640 1 -- 172.21.15.137:0/3863158905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 msgr2=0x5653d2667270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2667270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e8cac3640 1 -- 172.21.15.137:0/3863158905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 msgr2=0x5653d2680b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d2680b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e8cac3640 1 -- 172.21.15.137:0/3863158905 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653d2650e90 con 0x5653d2655330 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d2680620 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7e78004c00 tx=0x7f7e78010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112550090cbe10d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e72ffd640 1 -- 172.21.15.137:0/3863158905 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78032070 con 0x5653d2655330 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e72ffd640 1 -- 172.21.15.137:0/3863158905 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e780035f0 con 0x5653d2655330 2024-09-12T12:28:18.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.646+0000 7f7e72ffd640 1 -- 172.21.15.137:0/3863158905 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e780045f0 con 0x5653d2655330 2024-09-12T12:28:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.647+0000 7f7e922b6740 1 -- 172.21.15.137:0/3863158905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 msgr2=0x5653d2680620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.647+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d2680620 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7e78004c00 tx=0x7f7e78010ca0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.647+0000 7f7e922b6740 1 -- 172.21.15.137:0/3863158905 shutdown_connections 2024-09-12T12:28:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.647+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d2680b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.647+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d2680620 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.647+0000 7f7e922b6740 1 --2- 172.21.15.137:0/3863158905 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d2667270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.647+0000 7f7e922b6740 1 -- 172.21.15.137:0/3863158905 >> 172.21.15.137:0/3863158905 conn(0x5653d2651060 msgr2=0x5653d2666b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.648+0000 7f7e922b6740 1 -- 172.21.15.137:0/3863158905 shutdown_connections 2024-09-12T12:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.648+0000 7f7e922b6740 1 -- 172.21.15.137:0/3863158905 wait complete. 2024-09-12T12:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 -- start start 2024-09-12T12:28:18.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 Processor -- start 2024-09-12T12:28:18.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d26709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d266bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d266c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c00cd30 con 0x5653d2655330 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c00e1a0 con 0x5653d26684c0 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c00d400 con 0x5653d2654a10 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e922b6740 10 client.0 _ll_register_callbacks cb 0x7f7e840f23c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d266bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e8cac3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d266bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48134/0 (socket says 172.21.15.137:48134) 2024-09-12T12:28:18.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e8cac3640 1 -- 172.21.15.137:0/125185814 learned_addr learned my addr 172.21.15.137:0/125185814 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e8dac5640 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d266c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.649+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d26709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e8cac3640 1 -- 172.21.15.137:0/125185814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 msgr2=0x5653d26709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d26709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e8cac3640 1 -- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 msgr2=0x5653d266c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d266c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e8cac3640 1 -- 172.21.15.137:0/125185814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653d2660150 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e8cac3640 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d266bd10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7e780037c0 tx=0x7f7e78005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1948b2fb829acac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e70ff9640 1 -- 172.21.15.137:0/125185814 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78032050 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e70ff9640 1 -- 172.21.15.137:0/125185814 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7f7e78004480 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.650+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e780035f0 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.651+0000 7f7e70ff9640 1 -- 172.21.15.137:0/125185814 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7e78044050 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.651+0000 7f7e922b6740 10 client.17271 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.651+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5653d26e3100 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.651+0000 7f7e70ff9640 1 -- 172.21.15.137:0/125185814 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e78004e70 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e70ff9640 1 -- 172.21.15.137:0/125185814 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7f7e780269f0 con 0x5653d2655330 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e922b6740 10 client.17271 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e922b6740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:18.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e922b6740 20 client.17271 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e922b6740 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26e3810 0x5653d26e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e922b6740 4 client.17271 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:28:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7e780035f0 con 0x5653d26e3810 2024-09-12T12:28:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.652+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26e3810 0x5653d26e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.654+0000 7f7e8d2c4640 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26e3810 0x5653d26e5c30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6e4f34966c67ae server_cookie=a28573a444c30d4d in_seq=0 out_seq=0 2024-09-12T12:28:18.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.654+0000 7f7e70ff9640 10 client.17271 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.877+0000 7f7e70ff9640 1 -- 172.21.15.137:0/125185814 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7e780035f0 con 0x5653d26e3810 2024-09-12T12:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.877+0000 7f7e70ff9640 10 client.17271 handle_command_reply: tid=0 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.877+0000 7f7e922b6740 1 client.17271 shutdown 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 msgr2=0x5653d266bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d266bd10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7e780037c0 tx=0x7f7e78005570 comp rx=0 tx=0).stop 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 shutdown_connections 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653d26684c0 0x5653d266c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5653d26e3810 0x5653d26e5c30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5653d2655330 0x5653d266bd10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 --2- 172.21.15.137:0/125185814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5653d2654a10 0x5653d26709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 >> 172.21.15.137:0/125185814 conn(0x5653d2651060 msgr2=0x5653d2667380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e70ff9640 0 client.17271 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 shutdown_connections 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.878+0000 7f7e922b6740 1 -- 172.21.15.137:0/125185814 wait complete. 2024-09-12T12:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.879+0000 7f7e922b6740 20 client.17271 trim_cache size 0 max 16384 2024-09-12T12:28:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:28:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:28:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:28:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.879+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7e740a2ea0 msgr2=0x7f7e740a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.879+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7e740a2ea0 0x7f7e740a5360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7e80005fd0 tx=0x7f7e80009b10 comp rx=0 tx=0).stop 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e881009d0 msgr2=0x7f7e8819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e881009d0 0x7f7e8819a540 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7e84005ec0 tx=0x7f7e840346d0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 shutdown_connections 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7e740a2ea0 0x7f7e740a5360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e881069e0 0x7f7e8819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e88101370 0x7f7e8819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 --2- 172.21.15.137:0/3101440216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e881009d0 0x7f7e8819a540 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 >> 172.21.15.137:0/3101440216 conn(0x7f7e880fc6f0 msgr2=0x7f7e880fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 shutdown_connections 2024-09-12T12:28:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:18.880+0000 7f7e8f528640 1 -- 172.21.15.137:0/3101440216 wait complete. 2024-09-12T12:28:18.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:18 smithi171 ceph-mon[35098]: pgmap v477: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 809 op/s 2024-09-12T12:28:18.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:18 smithi171 ceph-mon[35098]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T12:28:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:18 smithi137 ceph-mon[30950]: pgmap v477: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 809 op/s 2024-09-12T12:28:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:18 smithi137 ceph-mon[30950]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T12:28:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:18 smithi189 ceph-mon[35475]: pgmap v477: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 16 MiB/s wr, 809 op/s 2024-09-12T12:28:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:18 smithi189 ceph-mon[35475]: osdmap e108: 12 total, 12 up, 12 in 2024-09-12T12:28:19.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 24246 13.30 MB/sec warmup 111 sec latency 133.858 ms 2024-09-12T12:28:20.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 25121 13.58 MB/sec warmup 112 sec latency 48.645 ms 2024-09-12T12:28:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:20 smithi171 ceph-mon[35098]: pgmap v479: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.2 MiB/s wr, 954 op/s 2024-09-12T12:28:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:20 smithi137 ceph-mon[30950]: pgmap v479: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.2 MiB/s wr, 954 op/s 2024-09-12T12:28:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:20 smithi189 ceph-mon[35475]: pgmap v479: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 9.2 MiB/s wr, 954 op/s 2024-09-12T12:28:21.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 26017 13.96 MB/sec warmup 113 sec latency 86.989 ms 2024-09-12T12:28:22.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 26849 14.24 MB/sec warmup 114 sec latency 42.059 ms 2024-09-12T12:28:22.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:22 smithi171 ceph-mon[35098]: pgmap v480: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.3 MiB/s wr, 863 op/s 2024-09-12T12:28:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:22 smithi137 ceph-mon[30950]: pgmap v480: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.3 MiB/s wr, 863 op/s 2024-09-12T12:28:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:22 smithi189 ceph-mon[35475]: pgmap v480: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 8.3 MiB/s wr, 863 op/s 2024-09-12T12:28:23.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 27548 14.43 MB/sec warmup 115 sec latency 52.197 ms 2024-09-12T12:28:24.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 28515 14.70 MB/sec warmup 116 sec latency 53.192 ms 2024-09-12T12:28:24.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:24 smithi171 ceph-mon[35098]: pgmap v481: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.4 MiB/s wr, 763 op/s 2024-09-12T12:28:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:24 smithi137 ceph-mon[30950]: pgmap v481: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.4 MiB/s wr, 763 op/s 2024-09-12T12:28:25.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:24 smithi189 ceph-mon[35475]: pgmap v481: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.4 MiB/s wr, 763 op/s 2024-09-12T12:28:25.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 29446 15.00 MB/sec warmup 117 sec latency 97.955 ms 2024-09-12T12:28:26.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 30136 15.30 MB/sec warmup 118 sec latency 316.033 ms 2024-09-12T12:28:26.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:26 smithi171 ceph-mon[35098]: pgmap v482: 97 pgs: 97 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.9 MiB/s wr, 673 op/s 2024-09-12T12:28:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:26 smithi137 ceph-mon[30950]: pgmap v482: 97 pgs: 97 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.9 MiB/s wr, 673 op/s 2024-09-12T12:28:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:26 smithi189 ceph-mon[35475]: pgmap v482: 97 pgs: 97 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.9 MiB/s wr, 673 op/s 2024-09-12T12:28:27.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 30778 15.36 MB/sec warmup 119 sec latency 98.547 ms 2024-09-12T12:28:28.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:28 smithi171 ceph-mon[35098]: pgmap v483: 97 pgs: 97 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.9 MiB/s wr, 673 op/s 2024-09-12T12:28:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:28 smithi137 ceph-mon[30950]: pgmap v483: 97 pgs: 97 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.9 MiB/s wr, 673 op/s 2024-09-12T12:28:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:28 smithi189 ceph-mon[35475]: pgmap v483: 97 pgs: 97 active+clean; 629 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.9 MiB/s wr, 673 op/s 2024-09-12T12:28:29.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 32430 37.09 MB/sec execute 1 sec latency 135.213 ms 2024-09-12T12:28:30.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:29 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:28:30.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:29 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:28:30.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:29 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:28:30.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 33261 46.55 MB/sec execute 2 sec latency 59.904 ms 2024-09-12T12:28:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:30 smithi137 ceph-mon[30950]: pgmap v484: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.6 MiB/s wr, 777 op/s 2024-09-12T12:28:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:30 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:28:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:30 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:28:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:30 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:28:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:30 smithi189 ceph-mon[35475]: pgmap v484: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.6 MiB/s wr, 777 op/s 2024-09-12T12:28:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:30 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:28:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:30 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:28:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:30 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:28:31.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:30 smithi171 ceph-mon[35098]: pgmap v484: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.6 MiB/s wr, 777 op/s 2024-09-12T12:28:31.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:30 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:28:31.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:30 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:28:31.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:30 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:28:31.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 33991 43.36 MB/sec execute 3 sec latency 106.469 ms 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.989+0000 7fed44f0b640 1 -- 172.21.15.137:0/155641696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103480 msgr2=0x7fed4010b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.989+0000 7fed44f0b640 1 --2- 172.21.15.137:0/155641696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103480 0x7fed4010b960 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fed34009900 tx=0x7fed3402a480 comp rx=0 tx=0).stop 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 -- 172.21.15.137:0/155641696 shutdown_connections 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 --2- 172.21.15.137:0/155641696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed4010bf30 0x7fed4010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 --2- 172.21.15.137:0/155641696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40103480 0x7fed4010b960 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 --2- 172.21.15.137:0/155641696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed40102b60 0x7fed40102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 -- 172.21.15.137:0/155641696 >> 172.21.15.137:0/155641696 conn(0x7fed400777c0 msgr2=0x7fed400feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 -- 172.21.15.137:0/155641696 shutdown_connections 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 -- 172.21.15.137:0/155641696 wait complete. 2024-09-12T12:28:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 Processor -- start 2024-09-12T12:28:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.990+0000 7fed44f0b640 1 -- start start 2024-09-12T12:28:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40102b60 0x7fed40198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed40103480 0x7fed40198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed4010bf30 0x7fed4019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed40111cd0 con 0x7fed40102b60 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed40111b50 con 0x7fed40103480 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed40111e50 con 0x7fed4010bf30 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40102b60 0x7fed40198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40102b60 0x7fed40198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42584/0 (socket says 172.21.15.137:42584) 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2308619674 learned_addr learned my addr 172.21.15.137:0/2308619674 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3effd640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed40103480 0x7fed40198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3ffff640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed4010bf30 0x7fed4019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2308619674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed4010bf30 msgr2=0x7fed4019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed4010bf30 0x7fed4019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2308619674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed40103480 msgr2=0x7fed40198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed40103480 0x7fed40198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2308619674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4019d230 con 0x7fed40102b60 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40102b60 0x7fed40198240 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e9f0 tx=0x7fed3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3cff9640 1 -- 172.21.15.137:0/2308619674 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3000cd80 con 0x7fed40102b60 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4019d520 con 0x7fed40102b60 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3cff9640 1 -- 172.21.15.137:0/2308619674 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed30010910 con 0x7fed40102b60 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed401a4db0 con 0x7fed40102b60 2024-09-12T12:28:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.991+0000 7fed3cff9640 1 -- 172.21.15.137:0/2308619674 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed300188f0 con 0x7fed40102b60 2024-09-12T12:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 Processor -- start 2024-09-12T12:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed3cff9640 1 -- 172.21.15.137:0/2308619674 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fed30010ac0 con 0x7fed40102b60 2024-09-12T12:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 -- start start 2024-09-12T12:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae3dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3e500 0x557b7ae4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b7ada22f0 con 0x557b7ae3e500 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557b7ad4a270 con 0x557b7ae4d290 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed48feb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557b7ad9e2b0 con 0x557b7ae3dbe0 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3e500 0x557b7ae4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed3ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae3dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.994+0000 7fed3effd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3e500 0x557b7ae4cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42588/0 (socket says 172.21.15.137:42588) 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae3dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:36800/0 (socket says 172.21.15.137:36800) 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2094157409 learned_addr learned my addr 172.21.15.137:0/2094157409 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3effd640 1 -- 172.21.15.137:0/2094157409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 msgr2=0x557b7ae3dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3effd640 1 --2- 172.21.15.137:0/2094157409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae3dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3effd640 1 -- 172.21.15.137:0/2094157409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3e500 msgr2=0x557b7ae4cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3effd640 1 --2- 172.21.15.137:0/2094157409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3e500 0x557b7ae4cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3effd640 1 -- 172.21.15.137:0/2094157409 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b7ad49340 con 0x557b7ae4d290 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3cff9640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed280a2cf0 0x7fed280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3ffff640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed280a2cf0 0x7fed280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3cff9640 1 -- 172.21.15.137:0/2308619674 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7fed300cf810 con 0x7fed40102b60 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3effd640 1 --2- 172.21.15.137:0/2094157409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4f680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fed34005940 tx=0x7fed34005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e146401d91215a3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed25ffb640 1 -- 172.21.15.137:0/2094157409 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34040070 con 0x557b7ae4d290 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed3ffff640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed280a2cf0 0x7fed280a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fed38005fd0 tx=0x7fed38009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed25ffb640 1 -- 172.21.15.137:0/2094157409 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34004030 con 0x557b7ae4d290 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.995+0000 7fed25ffb640 1 -- 172.21.15.137:0/2094157409 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035040 con 0x557b7ae4d290 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 -- 172.21.15.137:0/2094157409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 msgr2=0x557b7ae4f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 --2- 172.21.15.137:0/2094157409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4f680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fed34005940 tx=0x7fed34005be0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 -- 172.21.15.137:0/2094157409 shutdown_connections 2024-09-12T12:28:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 --2- 172.21.15.137:0/2094157409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4f680 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 --2- 172.21.15.137:0/2094157409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3e500 0x557b7ae4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 --2- 172.21.15.137:0/2094157409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae3dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 -- 172.21.15.137:0/2094157409 >> 172.21.15.137:0/2094157409 conn(0x557b7ae3a230 msgr2=0x557b7ae3a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 -- 172.21.15.137:0/2094157409 shutdown_connections 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.996+0000 7fed48feb740 1 -- 172.21.15.137:0/2094157409 wait complete. 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 -- start start 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 Processor -- start 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae4a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae4a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae4a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae4a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42594/0 (socket says 172.21.15.137:42594) 2024-09-12T12:28:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed3f7fe640 1 -- 172.21.15.137:0/902976381 learned_addr learned my addr 172.21.15.137:0/902976381 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x557b7ae4fe60 con 0x557b7ae3dbe0 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557b7ad2b290 con 0x557b7ae4d290 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.997+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557b7ae4ffe0 con 0x557b7ae3e500 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed48feb740 10 client.0 _ll_register_callbacks cb 0x557b7ad5c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3effd640 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae4a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3ffff640 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3f7fe640 1 -- 172.21.15.137:0/902976381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 msgr2=0x557b7ae4a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae4a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3f7fe640 1 -- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 msgr2=0x557b7ae4ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3f7fe640 1 -- 172.21.15.137:0/902976381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b7aab5210 con 0x557b7ae3dbe0 2024-09-12T12:28:32.004 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae4a110 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c6a0 tx=0x7fed300980a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9256bdfcb07ebeb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed1effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed1effd640 1 -- 172.21.15.137:0/902976381 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3009b760 con 0x557b7ae3dbe0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed1effd640 1 -- 172.21.15.137:0/902976381 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7fed3009b250 con 0x557b7ae3dbe0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.998+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34004030 con 0x557b7ae3dbe0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed1effd640 1 -- 172.21.15.137:0/902976381 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fed3009be00 con 0x557b7ae3dbe0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 10 client.17283 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557b7ab81620 con 0x557b7ae3dbe0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed1effd640 1 -- 172.21.15.137:0/902976381 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34004030 con 0x557b7ae3dbe0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed1effd640 1 -- 172.21.15.137:0/902976381 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fed30097cf0 con 0x557b7ae3dbe0 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 10 client.17283 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 20 client.17283 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7ae5f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 4 client.17283 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:28:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:31.999+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b7adafa20 con 0x557b7ae5d160 2024-09-12T12:28:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.000+0000 7fed3effd640 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7ae5f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.000+0000 7fed1effd640 10 client.17283 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.000+0000 7fed3effd640 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7ae5f5b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a557aff1add690 server_cookie=72fd8d78fcb3dd1 in_seq=0 out_seq=0 2024-09-12T12:28:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed1effd640 1 -- 172.21.15.137:0/902976381 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b7adafa20 con 0x557b7ae5d160 2024-09-12T12:28:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed1effd640 10 client.17283 handle_command_reply: tid=0 2024-09-12T12:28:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 client.17283 shutdown 2024-09-12T12:28:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 msgr2=0x557b7ae4a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae4a110 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c6a0 tx=0x7fed300980a0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 shutdown_connections 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae4a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7ae5f5b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed1effd640 0 client.17283 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 --2- 172.21.15.137:0/902976381 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae4a110 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.002+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 >> 172.21.15.137:0/902976381 conn(0x557b7ae3a230 msgr2=0x557b7ae3fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.003+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 shutdown_connections 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.003+0000 7fed48feb740 1 -- 172.21.15.137:0/902976381 wait complete. 2024-09-12T12:28:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.003+0000 7fed48feb740 20 client.17283 trim_cache size 0 max 16384 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 Processor -- start 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 -- start start 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae64ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae65400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ade90 con 0x557b7ae3dbe0 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed401ab6a0 con 0x557b7ae4d290 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed48feb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed401a94f0 con 0x557b7ae3e500 2024-09-12T12:28:32.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae64ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae64ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42604/0 (socket says 172.21.15.137:42604) 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2218767469 learned_addr learned my addr 172.21.15.137:0/2218767469 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed3effd640 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae65400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.021+0000 7fed3ffff640 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed3effd640 1 -- 172.21.15.137:0/2218767469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 msgr2=0x557b7ae4b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed3effd640 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed3effd640 1 -- 172.21.15.137:0/2218767469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 msgr2=0x557b7ae64ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed3effd640 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae64ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed3effd640 1 -- 172.21.15.137:0/2218767469 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b7ae48640 con 0x557b7ae3e500 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed3effd640 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae65400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed340051c0 tx=0x7fed340050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db7058226b96930d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed257fa640 1 -- 172.21.15.137:0/2218767469 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34040040 con 0x557b7ae3e500 2024-09-12T12:28:32.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed257fa640 1 -- 172.21.15.137:0/2218767469 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34004030 con 0x557b7ae3e500 2024-09-12T12:28:32.025 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.022+0000 7fed257fa640 1 -- 172.21.15.137:0/2218767469 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035040 con 0x557b7ae3e500 2024-09-12T12:28:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 -- 172.21.15.137:0/2218767469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 msgr2=0x557b7ae65400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae65400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fed340051c0 tx=0x7fed340050c0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.027 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 -- 172.21.15.137:0/2218767469 shutdown_connections 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae4d290 0x557b7ae4b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3e500 0x557b7ae65400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 --2- 172.21.15.137:0/2218767469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae3dbe0 0x557b7ae64ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 -- 172.21.15.137:0/2218767469 >> 172.21.15.137:0/2218767469 conn(0x557b7ae3a230 msgr2=0x557b7ae489c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 -- 172.21.15.137:0/2218767469 shutdown_connections 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.023+0000 7fed48feb740 1 -- 172.21.15.137:0/2218767469 wait complete. 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 -- start start 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 Processor -- start 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae55940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae3e500 0x557b7ae4e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae4d290 0x557b7ae4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fed401a9b60 con 0x557b7ae4d290 2024-09-12T12:28:32.028 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed401aafd0 con 0x557b7ae3e500 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed401aa230 con 0x557b7ae3dbe0 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed48feb740 10 client.0 _ll_register_callbacks cb 0x7fed3000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae55940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed3f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae55940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:36834/0 (socket says 172.21.15.137:36834) 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.024+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2148269947 learned_addr learned my addr 172.21.15.137:0/2148269947 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3effd640 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae3e500 0x557b7ae4e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3ffff640 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae4d290 0x557b7ae4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae3e500 msgr2=0x557b7ae4e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae3e500 0x557b7ae4e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2148269947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae4d290 msgr2=0x557b7ae4e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae4d290 0x557b7ae4e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3f7fe640 1 -- 172.21.15.137:0/2148269947 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b7ae4ee60 con 0x557b7ae3dbe0 2024-09-12T12:28:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed3f7fe640 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae55940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fed3009a0b0 tx=0x7fed3009b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f922fc146b1f2d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed1effd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed1effd640 1 -- 172.21.15.137:0/2148269947 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3009b760 con 0x557b7ae3dbe0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed1effd640 1 -- 172.21.15.137:0/2148269947 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7fed30010640 con 0x557b7ae3dbe0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34004030 con 0x557b7ae3dbe0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.025+0000 7fed1effd640 1 -- 172.21.15.137:0/2148269947 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fed30097ea0 con 0x557b7ae3dbe0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 10 client.26629 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557b7aec7e90 con 0x557b7ae3dbe0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed1effd640 1 -- 172.21.15.137:0/2148269947 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed34004030 con 0x557b7ae3dbe0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed1effd640 1 -- 172.21.15.137:0/2148269947 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7fed300ab030 con 0x557b7ae3dbe0 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 10 client.26629 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 20 client.26629 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7aeca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 4 client.26629 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:28:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b7ada9d70 con 0x557b7ae5d160 2024-09-12T12:28:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.026+0000 7fed3effd640 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7aeca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.027+0000 7fed3effd640 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7aeca640 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dcfd5781851d592 server_cookie=851621d24bb690c2 in_seq=0 out_seq=0 2024-09-12T12:28:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.027+0000 7fed1effd640 10 client.26629 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:32.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 34614 40.50 MB/sec execute 4 sec latency 134.888 ms 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed1effd640 1 -- 172.21.15.137:0/2148269947 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b7ada9d70 con 0x557b7ae5d160 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed1effd640 10 client.26629 handle_command_reply: tid=0 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 client.26629 shutdown 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 msgr2=0x557b7ae55940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae55940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fed3009a0b0 tx=0x7fed3009b480 comp rx=0 tx=0).stop 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 shutdown_connections 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x557b7ae5d160 0x557b7aeca640 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed1effd640 0 client.26629 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557b7ae4d290 0x557b7ae4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557b7ae3e500 0x557b7ae4e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 --2- 172.21.15.137:0/2148269947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557b7ae3dbe0 0x557b7ae55940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 >> 172.21.15.137:0/2148269947 conn(0x557b7ae3a230 msgr2=0x557b7ae38a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.522+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 shutdown_connections 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.523+0000 7fed48feb740 1 -- 172.21.15.137:0/2148269947 wait complete. 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.523+0000 7fed48feb740 20 client.26629 trim_cache size 0 max 16384 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:28:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:28:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed280a2cf0 msgr2=0x7fed280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed280a2cf0 0x7fed280a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fed38005fd0 tx=0x7fed38009500 comp rx=0 tx=0).stop 2024-09-12T12:28:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40102b60 msgr2=0x7fed40198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40102b60 0x7fed40198240 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e9f0 tx=0x7fed3000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 shutdown_connections 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fed280a2cf0 0x7fed280a51b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed4010bf30 0x7fed4019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed40103480 0x7fed40198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 --2- 172.21.15.137:0/2308619674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fed40102b60 0x7fed40198240 secure :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fed3000e9f0 tx=0x7fed3000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 >> 172.21.15.137:0/2308619674 conn(0x7fed400777c0 msgr2=0x7fed400fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 shutdown_connections 2024-09-12T12:28:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:32.524+0000 7fed44f0b640 1 -- 172.21.15.137:0/2308619674 wait complete. 2024-09-12T12:28:33.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:32 smithi171 ceph-mon[35098]: pgmap v485: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 440 op/s 2024-09-12T12:28:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:32 smithi137 ceph-mon[30950]: pgmap v485: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 440 op/s 2024-09-12T12:28:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:32 smithi189 ceph-mon[35475]: pgmap v485: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 440 op/s 2024-09-12T12:28:33.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 35531 42.60 MB/sec execute 5 sec latency 241.721 ms 2024-09-12T12:28:34.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 35825 37.76 MB/sec execute 6 sec latency 689.506 ms 2024-09-12T12:28:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:34 smithi137 ceph-mon[30950]: pgmap v486: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 440 op/s 2024-09-12T12:28:35.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:34 smithi189 ceph-mon[35475]: pgmap v486: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 440 op/s 2024-09-12T12:28:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:34 smithi171 ceph-mon[35098]: pgmap v486: 97 pgs: 97 active+clean; 659 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.2 MiB/s wr, 440 op/s 2024-09-12T12:28:35.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 35848 32.81 MB/sec execute 7 sec latency 904.894 ms 2024-09-12T12:28:36.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 36260 31.27 MB/sec execute 8 sec latency 868.765 ms 2024-09-12T12:28:37.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:36 smithi137 ceph-mon[30950]: pgmap v487: 97 pgs: 97 active+clean; 805 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 21 MiB/s wr, 964 op/s 2024-09-12T12:28:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:36 smithi189 ceph-mon[35475]: pgmap v487: 97 pgs: 97 active+clean; 805 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 21 MiB/s wr, 964 op/s 2024-09-12T12:28:37.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:36 smithi171 ceph-mon[35098]: pgmap v487: 97 pgs: 97 active+clean; 805 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 21 MiB/s wr, 964 op/s 2024-09-12T12:28:37.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 36909 31.26 MB/sec execute 9 sec latency 60.613 ms 2024-09-12T12:28:38.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 37671 33.10 MB/sec execute 10 sec latency 135.930 ms 2024-09-12T12:28:39.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:38 smithi137 ceph-mon[30950]: pgmap v488: 97 pgs: 97 active+clean; 805 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 733 op/s 2024-09-12T12:28:39.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:38 smithi189 ceph-mon[35475]: pgmap v488: 97 pgs: 97 active+clean; 805 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 733 op/s 2024-09-12T12:28:39.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:38 smithi171 ceph-mon[35098]: pgmap v488: 97 pgs: 97 active+clean; 805 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 733 op/s 2024-09-12T12:28:39.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 38332 33.12 MB/sec execute 11 sec latency 172.477 ms 2024-09-12T12:28:40.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 39069 33.53 MB/sec execute 12 sec latency 35.783 ms 2024-09-12T12:28:41.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:40 smithi137 ceph-mon[30950]: pgmap v489: 97 pgs: 97 active+clean; 685 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-09-12T12:28:41.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:40 smithi189 ceph-mon[35475]: pgmap v489: 97 pgs: 97 active+clean; 685 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-09-12T12:28:41.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:40 smithi171 ceph-mon[35098]: pgmap v489: 97 pgs: 97 active+clean; 685 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 1.11k op/s 2024-09-12T12:28:41.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 39826 33.69 MB/sec execute 13 sec latency 63.209 ms 2024-09-12T12:28:42.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 40656 34.66 MB/sec execute 14 sec latency 119.402 ms 2024-09-12T12:28:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:42 smithi137 ceph-mon[30950]: pgmap v490: 97 pgs: 97 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 932 op/s 2024-09-12T12:28:43.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:42 smithi189 ceph-mon[35475]: pgmap v490: 97 pgs: 97 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 932 op/s 2024-09-12T12:28:43.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:42 smithi171 ceph-mon[35098]: pgmap v490: 97 pgs: 97 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 932 op/s 2024-09-12T12:28:43.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 41378 35.43 MB/sec execute 15 sec latency 181.948 ms 2024-09-12T12:28:44.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 42082 35.54 MB/sec execute 16 sec latency 91.278 ms 2024-09-12T12:28:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:44 smithi137 ceph-mon[30950]: pgmap v491: 97 pgs: 97 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 932 op/s 2024-09-12T12:28:45.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:44 smithi189 ceph-mon[35475]: pgmap v491: 97 pgs: 97 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 932 op/s 2024-09-12T12:28:45.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:44 smithi171 ceph-mon[35098]: pgmap v491: 97 pgs: 97 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 932 op/s 2024-09-12T12:28:45.219 INFO:tasks.workunit.client.0.smithi137.stdout: 10 42610 35.01 MB/sec execute 17 sec latency 333.032 ms 2024-09-12T12:28:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 -- 172.21.15.137:0/361596239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff10100830 msgr2=0x7eff10100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 --2- 172.21.15.137:0/361596239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff10100830 0x7eff10100c10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7eff00009900 tx=0x7eff0002a410 comp rx=0 tx=0).stop 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 -- 172.21.15.137:0/361596239 shutdown_connections 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 --2- 172.21.15.137:0/361596239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff10105520 0x7eff1010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 --2- 172.21.15.137:0/361596239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff101011e0 0x7eff10104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 --2- 172.21.15.137:0/361596239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff10100830 0x7eff10100c10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 -- 172.21.15.137:0/361596239 >> 172.21.15.137:0/361596239 conn(0x7eff10075ec0 msgr2=0x7eff100762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 -- 172.21.15.137:0/361596239 shutdown_connections 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 -- 172.21.15.137:0/361596239 wait complete. 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 Processor -- start 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.625+0000 7eff179bd640 1 -- start start 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff179bd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff10100830 0x7eff10101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff179bd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff101011e0 0x7eff101023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.628 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff179bd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff10105520 0x7eff101028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff179bd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7eff10110e60 con 0x7eff101011e0 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff179bd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7eff10110ce0 con 0x7eff10105520 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff179bd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff10110fe0 con 0x7eff10100830 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff101011e0 0x7eff101023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff101011e0 0x7eff101023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34042/0 (socket says 172.21.15.137:34042) 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 -- 172.21.15.137:0/1857476840 learned_addr learned my addr 172.21.15.137:0/1857476840 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff15f5a640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff10105520 0x7eff101028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff15759640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff10100830 0x7eff10101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 -- 172.21.15.137:0/1857476840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff10100830 msgr2=0x7eff10101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff10100830 0x7eff10101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 -- 172.21.15.137:0/1857476840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff10105520 msgr2=0x7eff101028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.629 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff10105520 0x7eff101028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 -- 172.21.15.137:0/1857476840 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff101a6860 con 0x7eff101011e0 2024-09-12T12:28:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff14f58640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff101011e0 0x7eff101023a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d740 tx=0x7eff0c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff067fc640 1 -- 172.21.15.137:0/1857476840 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c017070 con 0x7eff101011e0 2024-09-12T12:28:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff067fc640 1 -- 172.21.15.137:0/1857476840 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0c00de50 con 0x7eff101011e0 2024-09-12T12:28:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff101a6b50 con 0x7eff101011e0 2024-09-12T12:28:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.626+0000 7eff067fc640 1 -- 172.21.15.137:0/1857476840 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00f6b0 con 0x7eff101011e0 2024-09-12T12:28:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.627+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff101a70a0 con 0x7eff101011e0 2024-09-12T12:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff067fc640 1 -- 172.21.15.137:0/1857476840 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7eff0c01a070 con 0x7eff101011e0 2024-09-12T12:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 Processor -- start 2024-09-12T12:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 -- start start 2024-09-12T12:28:45.635 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72e1410 0x563ec72eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72ebcb0 0x563ec72ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ec72452f0 con 0x563ec72e0af0 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563ec71ed270 con 0x563ec72ebcb0 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff1a832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563ec72412b0 con 0x563ec72e1410 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff15f5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff15f5a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34052/0 (socket says 172.21.15.137:34052) 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff15f5a640 1 -- 172.21.15.137:0/639770664 learned_addr learned my addr 172.21.15.137:0/639770664 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff067fc640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efefc0a2da0 0x7efefc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff14f58640 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72ebcb0 0x563ec72ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.636 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.629+0000 7eff15759640 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72e1410 0x563ec72eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efefc0a2da0 0x7efefc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff067fc640 1 -- 172.21.15.137:0/1857476840 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7eff0c0cee30 con 0x7eff101011e0 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 -- 172.21.15.137:0/639770664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72e1410 msgr2=0x563ec72eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72e1410 0x563ec72eb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 -- 172.21.15.137:0/639770664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72ebcb0 msgr2=0x563ec72ee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72ebcb0 0x563ec72ee0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 -- 172.21.15.137:0/639770664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec71ec340 con 0x563ec72e0af0 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15759640 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72e1410 0x563ec72eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e0ed0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7eff0800a980 tx=0x7eff0800ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac5ca9453d8a4ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7efefb7fe640 1 -- 172.21.15.137:0/639770664 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0801d030 con 0x563ec72e0af0 2024-09-12T12:28:45.637 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7efefb7fe640 1 -- 172.21.15.137:0/639770664 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0800a420 con 0x563ec72e0af0 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7efefb7fe640 1 -- 172.21.15.137:0/639770664 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff080213c0 con 0x563ec72e0af0 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.630+0000 7eff15f5a640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efefc0a2da0 0x7efefc0a5260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7eff08020000 tx=0x7eff0801e000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 -- 172.21.15.137:0/639770664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 msgr2=0x563ec72e0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e0ed0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7eff0800a980 tx=0x7eff0800ae40 comp rx=0 tx=0).stop 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 -- 172.21.15.137:0/639770664 shutdown_connections 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72ebcb0 0x563ec72ee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72e1410 0x563ec72eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 --2- 172.21.15.137:0/639770664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e0ed0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 -- 172.21.15.137:0/639770664 >> 172.21.15.137:0/639770664 conn(0x563ec72dd060 msgr2=0x563ec72dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:45.638 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 -- 172.21.15.137:0/639770664 shutdown_connections 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 -- 172.21.15.137:0/639770664 wait complete. 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 -- start start 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.631+0000 7eff1a832740 1 Processor -- start 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72e90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec72e9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x563ec72ee880 con 0x563ec72e1410 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563ec71ce290 con 0x563ec72e0af0 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563ec72eea00 con 0x563ec72ebcb0 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 10 client.0 _ll_register_callbacks cb 0x563ec71ff140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72e90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72e90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34066/0 (socket says 172.21.15.137:34066) 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 -- 172.21.15.137:0/2510447930 learned_addr learned my addr 172.21.15.137:0/2510447930 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff15759640 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 -- 172.21.15.137:0/2510447930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 msgr2=0x563ec72e9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec72e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 -- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 msgr2=0x563ec72e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72e8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 -- 172.21.15.137:0/2510447930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec6f58210 con 0x563ec72e1410 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff14f58640 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72e90c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0977d0 tx=0x7eff0c09afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ed20b7d649f96f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7efef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7efef8ff9640 1 -- 172.21.15.137:0/2510447930 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c021070 con 0x563ec72e1410 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7efef8ff9640 1 -- 172.21.15.137:0/2510447930 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7eff0c09a7c0 con 0x563ec72e1410 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.632+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff0800a420 con 0x563ec72e1410 2024-09-12T12:28:45.640 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7efef8ff9640 1 -- 172.21.15.137:0/2510447930 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7eff0c09f720 con 0x563ec72e1410 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 10 client.17298 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563ec7024620 con 0x563ec72e1410 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7efef8ff9640 1 -- 172.21.15.137:0/2510447930 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0800a420 con 0x563ec72e1410 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7efef8ff9640 1 -- 172.21.15.137:0/2510447930 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7eff0c09a430 con 0x563ec72e1410 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 10 client.17298 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 20 client.17298 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec72fbcb0 0x563ec72fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 4 client.17298 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:28:45.641 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.633+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ec7252a20 con 0x563ec72fbcb0 2024-09-12T12:28:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.646+0000 7eff15759640 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec72fbcb0 0x563ec72fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.647+0000 7eff15759640 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec72fbcb0 0x563ec72fe100 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c581a77f70340470 server_cookie=de31c023afb76ad0 in_seq=0 out_seq=0 2024-09-12T12:28:45.647 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.647+0000 7efef8ff9640 10 client.17298 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7efef8ff9640 1 -- 172.21.15.137:0/2510447930 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ec7252a20 con 0x563ec72fbcb0 2024-09-12T12:28:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7efef8ff9640 10 client.17298 handle_command_reply: tid=0 2024-09-12T12:28:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 client.17298 shutdown 2024-09-12T12:28:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 msgr2=0x563ec72e90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72e90c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0977d0 tx=0x7eff0c09afe0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.650 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 shutdown_connections 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec72e9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec72fbcb0 0x563ec72fe100 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72e90c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7efef8ff9640 0 client.17298 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 --2- 172.21.15.137:0/2510447930 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 >> 172.21.15.137:0/2510447930 conn(0x563ec72dd060 msgr2=0x563ec72dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 shutdown_connections 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 1 -- 172.21.15.137:0/2510447930 wait complete. 2024-09-12T12:28:45.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.649+0000 7eff1a832740 20 client.17298 trim_cache size 0 max 16384 2024-09-12T12:28:45.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 Processor -- start 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 -- start start 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e1410 0x563ec72eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec73039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7eff101b2310 con 0x563ec72e0af0 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7eff101afb20 con 0x563ec72e1410 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff1a832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff101ad990 con 0x563ec72ebcb0 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34072/0 (socket says 172.21.15.137:34072) 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 -- 172.21.15.137:0/3677065324 learned_addr learned my addr 172.21.15.137:0/3677065324 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff14f58640 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e1410 0x563ec72eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15f5a640 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec73039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 -- 172.21.15.137:0/3677065324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 msgr2=0x563ec73039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec73039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 -- 172.21.15.137:0/3677065324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e1410 msgr2=0x563ec72eaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e1410 0x563ec72eaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.667+0000 7eff15759640 1 -- 172.21.15.137:0/3677065324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec72eb6d0 con 0x563ec72e0af0 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.668+0000 7eff15759640 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e9430 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7eff00009900 tx=0x7eff00009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b1359ddbaf7ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.668+0000 7efefaffd640 1 -- 172.21.15.137:0/3677065324 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00035040 con 0x563ec72e0af0 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.668+0000 7efefaffd640 1 -- 172.21.15.137:0/3677065324 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff00005520 con 0x563ec72e0af0 2024-09-12T12:28:45.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.668+0000 7efefaffd640 1 -- 172.21.15.137:0/3677065324 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00004d00 con 0x563ec72e0af0 2024-09-12T12:28:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 -- 172.21.15.137:0/3677065324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 msgr2=0x563ec72e9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.673 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e9430 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7eff00009900 tx=0x7eff00009660 comp rx=0 tx=0).stop 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 -- 172.21.15.137:0/3677065324 shutdown_connections 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec73039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e1410 0x563ec72eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 --2- 172.21.15.137:0/3677065324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e0af0 0x563ec72e9430 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 -- 172.21.15.137:0/3677065324 >> 172.21.15.137:0/3677065324 conn(0x563ec72dd060 msgr2=0x563ec72ea3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 -- 172.21.15.137:0/3677065324 shutdown_connections 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 -- 172.21.15.137:0/3677065324 wait complete. 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 -- start start 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.669+0000 7eff1a832740 1 Processor -- start 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72ef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.674 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec72f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7eff101adfe0 con 0x563ec72e1410 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7eff101af450 con 0x563ec72e0af0 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff101ae6b0 con 0x563ec72ebcb0 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 10 client.0 _ll_register_callbacks cb 0x7eff0c09b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72ef9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34084/0 (socket says 172.21.15.137:34084) 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 -- 172.21.15.137:0/987959262 learned_addr learned my addr 172.21.15.137:0/987959262 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff15759640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff15f5a640 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec72f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 -- 172.21.15.137:0/987959262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 msgr2=0x563ec72f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec72f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 -- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 msgr2=0x563ec72ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 -- 172.21.15.137:0/987959262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec72f4190 con 0x563ec72e1410 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff15759640 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff14f58640 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72ef9c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009610 tx=0x7eff0c0ccae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8553b9e463c537 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7efef8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7efef8ff9640 1 -- 172.21.15.137:0/987959262 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c021040 con 0x563ec72e1410 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7efef8ff9640 1 -- 172.21.15.137:0/987959262 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8484+0+0 (secure 0 0 0) 0x7eff0c09a9c0 con 0x563ec72e1410 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff00005520 con 0x563ec72e1410 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.670+0000 7efef8ff9640 1 -- 172.21.15.137:0/987959262 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7eff0c0bb460 con 0x563ec72e1410 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 10 client.17304 fetch_fsmap learned FSMap version 45 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563ec7366560 con 0x563ec72e1410 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7efef8ff9640 1 -- 172.21.15.137:0/987959262 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff00005520 con 0x563ec72e1410 2024-09-12T12:28:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7efef8ff9640 1 -- 172.21.15.137:0/987959262 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 45) ==== 8370+0+0 (secure 0 0 0) 0x7eff0c09a230 con 0x563ec72e1410 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 10 client.17304 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 20 client.17304 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec7366ac0 0x563ec7368ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 4 client.17304 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ec72e4760 con 0x563ec7366ac0 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff15759640 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec7366ac0 0x563ec7368ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7eff15759640 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec7366ac0 0x563ec7368ee0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d2bf1de6b7a2bfc server_cookie=2a7a960c02383257 in_seq=0 out_seq=0 2024-09-12T12:28:45.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.671+0000 7efef8ff9640 10 client.17304 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7efef8ff9640 1 -- 172.21.15.137:0/987959262 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ec72e4760 con 0x563ec7366ac0 2024-09-12T12:28:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7efef8ff9640 10 client.17304 handle_command_reply: tid=0 2024-09-12T12:28:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 client.17304 shutdown 2024-09-12T12:28:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 msgr2=0x563ec72ef9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72ef9c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009610 tx=0x7eff0c0ccae0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 shutdown_connections 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ec72ebcb0 0x563ec72f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x563ec7366ac0 0x563ec7368ee0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ec72e1410 0x563ec72ef9c0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 --2- 172.21.15.137:0/987959262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ec72e0af0 0x563ec72ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7efef8ff9640 0 client.17304 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 >> 172.21.15.137:0/987959262 conn(0x563ec72dd060 msgr2=0x563ec72e2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 shutdown_connections 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 1 -- 172.21.15.137:0/987959262 wait complete. 2024-09-12T12:28:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.906+0000 7eff1a832740 20 client.17304 trim_cache size 0 max 16384 2024-09-12T12:28:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:28:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:28:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:28:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efefc0a2da0 msgr2=0x7efefc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efefc0a2da0 0x7efefc0a5260 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7eff08020000 tx=0x7eff0801e000 comp rx=0 tx=0).stop 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff101011e0 msgr2=0x7eff101023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff101011e0 0x7eff101023a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d740 tx=0x7eff0c00dc00 comp rx=0 tx=0).stop 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 shutdown_connections 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efefc0a2da0 0x7efefc0a5260 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff10105520 0x7eff101028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff101011e0 0x7eff101023a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 --2- 172.21.15.137:0/1857476840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff10100830 0x7eff10101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 >> 172.21.15.137:0/1857476840 conn(0x7eff10075ec0 msgr2=0x7eff1010db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 shutdown_connections 2024-09-12T12:28:45.911 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:45.907+0000 7eff179bd640 1 -- 172.21.15.137:0/1857476840 wait complete. 2024-09-12T12:28:46.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 43277 34.85 MB/sec execute 18 sec latency 351.992 ms 2024-09-12T12:28:46.736 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:28:46 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:28:46.736 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:28:46 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:28:46.736 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:28:46 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/2412330208 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7ac871860 con 0x55e7ac84f400 2024-09-12T12:28:46.736 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:28:46 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:28:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:46 smithi137 ceph-mon[30950]: pgmap v492: 97 pgs: 97 active+clean; 709 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-09-12T12:28:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:46 smithi137 ceph-mon[30950]: daemon mds.f restarted 2024-09-12T12:28:47.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:46 smithi189 ceph-mon[35475]: pgmap v492: 97 pgs: 97 active+clean; 709 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-09-12T12:28:47.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:46 smithi189 ceph-mon[35475]: daemon mds.f restarted 2024-09-12T12:28:47.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:46 smithi171 ceph-mon[35098]: pgmap v492: 97 pgs: 97 active+clean; 709 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 22 MiB/s wr, 1.15k op/s 2024-09-12T12:28:47.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:46 smithi171 ceph-mon[35098]: daemon mds.f restarted 2024-09-12T12:28:47.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 43969 34.75 MB/sec execute 19 sec latency 248.640 ms 2024-09-12T12:28:48.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:47 smithi137 ceph-mon[30950]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T12:28:48.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:47 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/3747343886,v1:172.21.15.189:6835/3747343886] up:boot 2024-09-12T12:28:48.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:47 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:28:48.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:47 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:28:48.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:47 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:28:48.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:47 smithi189 ceph-mon[35475]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T12:28:48.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:47 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/3747343886,v1:172.21.15.189:6835/3747343886] up:boot 2024-09-12T12:28:48.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:47 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:28:48.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:47 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:28:48.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:47 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:28:48.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:47 smithi171 ceph-mon[35098]: osdmap e109: 12 total, 12 up, 12 in 2024-09-12T12:28:48.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:47 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/3747343886,v1:172.21.15.189:6835/3747343886] up:boot 2024-09-12T12:28:48.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:47 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-12T12:28:48.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:47 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:28:48.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:47 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:28:48.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 44801 35.70 MB/sec execute 20 sec latency 42.714 ms 2024-09-12T12:28:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:48 smithi137 ceph-mon[30950]: pgmap v494: 97 pgs: 97 active+clean; 709 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 749 op/s 2024-09-12T12:28:49.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:48 smithi189 ceph-mon[35475]: pgmap v494: 97 pgs: 97 active+clean; 709 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 749 op/s 2024-09-12T12:28:49.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:48 smithi171 ceph-mon[35098]: pgmap v494: 97 pgs: 97 active+clean; 709 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 749 op/s 2024-09-12T12:28:49.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 45404 35.53 MB/sec execute 21 sec latency 157.971 ms 2024-09-12T12:28:50.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 46192 35.76 MB/sec execute 22 sec latency 46.964 ms 2024-09-12T12:28:51.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:50 smithi137 ceph-mon[30950]: pgmap v495: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 642 op/s 2024-09-12T12:28:51.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:50 smithi189 ceph-mon[35475]: pgmap v495: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 642 op/s 2024-09-12T12:28:51.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:50 smithi171 ceph-mon[35098]: pgmap v495: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 7.3 MiB/s wr, 642 op/s 2024-09-12T12:28:51.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 46984 36.00 MB/sec execute 23 sec latency 61.574 ms 2024-09-12T12:28:52.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 47755 36.42 MB/sec execute 24 sec latency 89.685 ms 2024-09-12T12:28:53.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:52 smithi137 ceph-mon[30950]: pgmap v496: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-12T12:28:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:52 smithi189 ceph-mon[35475]: pgmap v496: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-12T12:28:53.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:52 smithi171 ceph-mon[35098]: pgmap v496: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-12T12:28:53.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 48401 36.28 MB/sec execute 25 sec latency 66.291 ms 2024-09-12T12:28:54.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 49254 36.58 MB/sec execute 26 sec latency 43.165 ms 2024-09-12T12:28:55.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:54 smithi137 ceph-mon[30950]: pgmap v497: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-12T12:28:55.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:54 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:28:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:54 smithi189 ceph-mon[35475]: pgmap v497: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-12T12:28:55.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:54 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:28:55.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:54 smithi171 ceph-mon[35098]: pgmap v497: 97 pgs: 97 active+clean; 719 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 6.5 MiB/s wr, 619 op/s 2024-09-12T12:28:55.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:54 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:28:55.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 50207 37.24 MB/sec execute 27 sec latency 43.451 ms 2024-09-12T12:28:56.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 50888 37.21 MB/sec execute 28 sec latency 219.859 ms 2024-09-12T12:28:57.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:56 smithi137 ceph-mon[30950]: pgmap v498: 97 pgs: 97 active+clean; 749 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 633 op/s 2024-09-12T12:28:57.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:56 smithi189 ceph-mon[35475]: pgmap v498: 97 pgs: 97 active+clean; 749 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 633 op/s 2024-09-12T12:28:57.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:56 smithi171 ceph-mon[35098]: pgmap v498: 97 pgs: 97 active+clean; 749 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 633 op/s 2024-09-12T12:28:57.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 51532 37.26 MB/sec execute 29 sec latency 152.659 ms 2024-09-12T12:28:58.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 52269 36.99 MB/sec execute 30 sec latency 97.461 ms 2024-09-12T12:28:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.005+0000 7f5d1705e640 1 -- 172.21.15.137:0/162172810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 msgr2=0x7f5d10101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.005+0000 7f5d1705e640 1 --2- 172.21.15.137:0/162172810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10101f30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009930 tx=0x7f5cfc02a420 comp rx=0 tx=0).stop 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.005+0000 7f5d1705e640 1 -- 172.21.15.137:0/162172810 shutdown_connections 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 --2- 172.21.15.137:0/162172810 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d10102470 0x7f5d1010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 --2- 172.21.15.137:0/162172810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10101f30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 --2- 172.21.15.137:0/162172810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10107b70 0x7f5d10107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 -- 172.21.15.137:0/162172810 >> 172.21.15.137:0/162172810 conn(0x7f5d100fd7d0 msgr2=0x7f5d100ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 -- 172.21.15.137:0/162172810 shutdown_connections 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 -- 172.21.15.137:0/162172810 wait complete. 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 Processor -- start 2024-09-12T12:28:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 -- start start 2024-09-12T12:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10102470 0x7f5d101988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d10107b70 0x7f5d1019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.006+0000 7f5d1705e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10111f30 con 0x7f5d10101ad0 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d1705e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10111db0 con 0x7f5d10102470 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d1705e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101120b0 con 0x7f5d10107b70 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10102470 0x7f5d101988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57224/0 (socket says 172.21.15.137:57224) 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10102470 0x7f5d101988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:51526/0 (socket says 172.21.15.137:51526) 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1936029820 learned_addr learned my addr 172.21.15.137:0/1936029820 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d155fb640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d10107b70 0x7f5d1019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1936029820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d10107b70 msgr2=0x7f5d1019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d10107b70 0x7f5d1019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1936029820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10102470 msgr2=0x7f5d101988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10102470 0x7f5d101988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1936029820 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1019d370 con 0x7f5d10101ad0 2024-09-12T12:28:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10102470 0x7f5d101988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:28:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10198380 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e9f0 tx=0x7f5d0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d0dffb640 1 -- 172.21.15.137:0/1936029820 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400cd80 con 0x7f5d10101ad0 2024-09-12T12:28:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d0dffb640 1 -- 172.21.15.137:0/1936029820 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d04010910 con 0x7f5d10101ad0 2024-09-12T12:28:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1019d660 con 0x7f5d10101ad0 2024-09-12T12:28:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.007+0000 7f5d0dffb640 1 -- 172.21.15.137:0/1936029820 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d04018990 con 0x7f5d10101ad0 2024-09-12T12:28:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.008+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d101a4ef0 con 0x7f5d10101ad0 2024-09-12T12:28:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.010+0000 7f5d0dffb640 1 -- 172.21.15.137:0/1936029820 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5d04010ac0 con 0x7f5d10101ad0 2024-09-12T12:28:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.010+0000 7f5d19dfb740 1 Processor -- start 2024-09-12T12:28:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d19dfb740 1 -- start start 2024-09-12T12:28:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b327db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b336b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b3394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55950b28c2f0 con 0x55950b3279d0 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55950b234270 con 0x55950b3370c0 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55950b2882b0 con 0x55950b3282f0 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b327db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b327db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57230/0 (socket says 172.21.15.137:57230) 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 -- 172.21.15.137:0/321049758 learned_addr learned my addr 172.21.15.137:0/321049758 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b336b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b3394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d0dffb640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5d000a2da0 0x7f5d000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 -- 172.21.15.137:0/321049758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 msgr2=0x55950b336b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b336b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 -- 172.21.15.137:0/321049758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 msgr2=0x55950b3394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b3394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 -- 172.21.15.137:0/321049758 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950b233340 con 0x55950b3279d0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d0dffb640 1 -- 172.21.15.137:0/1936029820 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8440+0+0 (secure 0 0 0) 0x7f5d040cf690 con 0x7f5d10101ad0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5d000a2da0 0x7f5d000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.011+0000 7f5d155fb640 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b327db0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5d08011440 tx=0x7f5d08011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d9d984eb4bad5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.012+0000 7f5cfaffd640 1 -- 172.21.15.137:0/321049758 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08004250 con 0x55950b3279d0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.012+0000 7f5cfaffd640 1 -- 172.21.15.137:0/321049758 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d08003d10 con 0x55950b3279d0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.012+0000 7f5cfaffd640 1 -- 172.21.15.137:0/321049758 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0801bcf0 con 0x55950b3279d0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 -- 172.21.15.137:0/321049758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 msgr2=0x55950b327db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b327db0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5d08011440 tx=0x7f5d08011900 comp rx=0 tx=0).stop 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d155fb640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5d000a2da0 0x7f5d000a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d08020000 tx=0x7f5d08021000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 -- 172.21.15.137:0/321049758 shutdown_connections 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b3394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b336b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/321049758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b327db0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 -- 172.21.15.137:0/321049758 >> 172.21.15.137:0/321049758 conn(0x55950b324060 msgr2=0x55950b324440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 -- 172.21.15.137:0/321049758 shutdown_connections 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 -- 172.21.15.137:0/321049758 wait complete. 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 -- start start 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 Processor -- start 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3279d0 0x55950b334000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b334540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.013+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3370c0 0x55950b334a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55950b339cf0 con 0x55950b3370c0 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55950b215290 con 0x55950b3279d0 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55950b339e70 con 0x55950b3282f0 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d19dfb740 10 client.0 _ll_register_callbacks cb 0x55950b246140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3279d0 0x55950b334000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b334540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3279d0 0x55950b334000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:51534/0 (socket says 172.21.15.137:51534) 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d14dfa640 1 -- 172.21.15.137:0/2793007611 learned_addr learned my addr 172.21.15.137:0/2793007611 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d155fb640 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3370c0 0x55950b334a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d0ffff640 1 -- 172.21.15.137:0/2793007611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3279d0 msgr2=0x55950b334000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3279d0 0x55950b334000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d0ffff640 1 -- 172.21.15.137:0/2793007611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3370c0 msgr2=0x55950b334a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3370c0 0x55950b334a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d0ffff640 1 -- 172.21.15.137:0/2793007611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950af9f210 con 0x55950b3282f0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b334540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009230 tx=0x7f5cfc005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd82782fab3a6f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5ceffff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5ceffff640 1 -- 172.21.15.137:0/2793007611 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc035040 con 0x55950b3282f0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5ceffff640 1 -- 172.21.15.137:0/2793007611 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8440+0+0 (secure 0 0 0) 0x7f5cfc004760 con 0x55950b3282f0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.014+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d08003d10 con 0x55950b3282f0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5ceffff640 1 -- 172.21.15.137:0/2793007611 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5cfc04a020 con 0x55950b3282f0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 10 client.26659 fetch_fsmap learned FSMap version 48 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55950b06b620 con 0x55950b3282f0 2024-09-12T12:28:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5ceffff640 1 -- 172.21.15.137:0/2793007611 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d08003d10 con 0x55950b3282f0 2024-09-12T12:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5ceffff640 1 -- 172.21.15.137:0/2793007611 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f5cfc031bd0 con 0x55950b3282f0 2024-09-12T12:28:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 10 client.26659 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 10 client.26659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 20 client.26659 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3470d0 0x55950b3494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 4 client.26659 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.015+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55950b299a20 con 0x55950b3470d0 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.016+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3470d0 0x55950b3494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.016+0000 7f5ceffff640 10 client.26659 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.016+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3470d0 0x55950b3494c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6a3e29548802f4 server_cookie=221dd22d1f2c6e4d in_seq=0 out_seq=0 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5ceffff640 1 -- 172.21.15.137:0/2793007611 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55950b299a20 con 0x55950b3470d0 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5ceffff640 10 client.26659 handle_command_reply: tid=0 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 client.26659 shutdown 2024-09-12T12:28:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 msgr2=0x55950b334540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b334540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009230 tx=0x7f5cfc005270 comp rx=0 tx=0).stop 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 shutdown_connections 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3470d0 0x55950b3494c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3370c0 0x55950b334a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5ceffff640 0 client.26659 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b334540 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/2793007611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3279d0 0x55950b334000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 >> 172.21.15.137:0/2793007611 conn(0x55950b324060 msgr2=0x55950b329dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 shutdown_connections 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 1 -- 172.21.15.137:0/2793007611 wait complete. 2024-09-12T12:28:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.017+0000 7f5d19dfb740 20 client.26659 trim_cache size 0 max 16384 2024-09-12T12:28:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 Processor -- start 2024-09-12T12:28:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 -- start start 2024-09-12T12:28:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b34edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b34f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b331240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101adf00 con 0x55950b3279d0 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101ab710 con 0x55950b3370c0 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101a9580 con 0x55950b3282f0 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b34edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b34edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57238/0 (socket says 172.21.15.137:57238) 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 -- 172.21.15.137:0/3154127783 learned_addr learned my addr 172.21.15.137:0/3154127783 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b34f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d155fb640 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b331240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 -- 172.21.15.137:0/3154127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 msgr2=0x55950b34f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b34f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 -- 172.21.15.137:0/3154127783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 msgr2=0x55950b331240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b331240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 -- 172.21.15.137:0/3154127783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950b323e90 con 0x55950b3279d0 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.032+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b34edd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5d0409b9b0 tx=0x7f5d040d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445a72c60ae1cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5cfa7fc640 1 -- 172.21.15.137:0/3154127783 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0409ffb0 con 0x55950b3279d0 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5cfa7fc640 1 -- 172.21.15.137:0/3154127783 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d040a4870 con 0x55950b3279d0 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5cfa7fc640 1 -- 172.21.15.137:0/3154127783 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d040cfb40 con 0x55950b3279d0 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5d19dfb740 1 -- 172.21.15.137:0/3154127783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 msgr2=0x55950b34edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b34edd0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f5d0409b9b0 tx=0x7f5d040d5550 comp rx=0 tx=0).stop 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5d19dfb740 1 -- 172.21.15.137:0/3154127783 shutdown_connections 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3370c0 0x55950b331240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.033+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3282f0 0x55950b34f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/3154127783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b34edd0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 -- 172.21.15.137:0/3154127783 >> 172.21.15.137:0/3154127783 conn(0x55950b324060 msgr2=0x55950b336080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 -- 172.21.15.137:0/3154127783 shutdown_connections 2024-09-12T12:28:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 -- 172.21.15.137:0/3154127783 wait complete. 2024-09-12T12:28:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 -- start start 2024-09-12T12:28:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 Processor -- start 2024-09-12T12:28:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b33af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3282f0 0x55950b33b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.034+0000 7f5d19dfb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3370c0 0x55950b33b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101a9bd0 con 0x55950b3279d0 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101ab040 con 0x55950b3282f0 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d19dfb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d101aa2a0 con 0x55950b3370c0 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d19dfb740 10 client.0 _ll_register_callbacks cb 0x7f5cfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b33af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3282f0 0x55950b33b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b33af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57242/0 (socket says 172.21.15.137:57242) 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d155fb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3370c0 0x55950b33b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1349340623 learned_addr learned my addr 172.21.15.137:0/1349340623 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1349340623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3370c0 msgr2=0x55950b33b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3370c0 0x55950b33b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3282f0 msgr2=0x55950b33b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3282f0 0x55950b33b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 -- 172.21.15.137:0/1349340623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950b32ed50 con 0x55950b3279d0 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3282f0 0x55950b33b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d14dfa640 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b33af30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009330 tx=0x7f5d04000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c0f6e5561f8426 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5ceffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5ceffff640 1 -- 172.21.15.137:0/1349340623 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0409ffb0 con 0x55950b3279d0 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5ceffff640 1 -- 172.21.15.137:0/1349340623 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8440+0+0 (secure 0 0 0) 0x7f5d04018b30 con 0x55950b3279d0 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d040a4870 con 0x55950b3279d0 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5ceffff640 1 -- 172.21.15.137:0/1349340623 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5d04018d30 con 0x55950b3279d0 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d19dfb740 10 client.17325 fetch_fsmap learned FSMap version 48 2024-09-12T12:28:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.035+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55950b324cf0 con 0x55950b3279d0 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5ceffff640 1 -- 172.21.15.137:0/1349340623 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d040b2070 con 0x55950b3279d0 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5ceffff640 1 -- 172.21.15.137:0/1349340623 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f5d0409b220 con 0x55950b3279d0 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d19dfb740 10 client.17325 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d19dfb740 10 client.17325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d19dfb740 20 client.17325 populate_metadata read hostname 'smithi137' 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3b24c0 0x55950b3b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d19dfb740 4 client.17325 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55950b293d70 con 0x55950b3b24c0 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3b24c0 0x55950b3b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5ceffff640 10 client.17325 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.036+0000 7f5d0ffff640 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3b24c0 0x55950b3b48e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a6b38cf01468fd server_cookie=695eb554ec2c9de6 in_seq=0 out_seq=0 2024-09-12T12:28:59.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:28:58 smithi137 ceph-mon[30950]: pgmap v499: 97 pgs: 97 active+clean; 749 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 583 op/s 2024-09-12T12:28:59.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:28:58 smithi189 ceph-mon[35475]: pgmap v499: 97 pgs: 97 active+clean; 749 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 583 op/s 2024-09-12T12:28:59.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:28:58 smithi171 ceph-mon[35098]: pgmap v499: 97 pgs: 97 active+clean; 749 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.1 MiB/s wr, 583 op/s 2024-09-12T12:28:59.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 52981 37.04 MB/sec execute 31 sec latency 125.324 ms 2024-09-12T12:28:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.538+0000 7f5ceffff640 1 -- 172.21.15.137:0/1349340623 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55950b293d70 con 0x55950b3b24c0 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.538+0000 7f5ceffff640 10 client.17325 handle_command_reply: tid=0 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.538+0000 7f5d19dfb740 1 client.17325 shutdown 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.538+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 msgr2=0x55950b33af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.538+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b33af30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009330 tx=0x7f5d04000bc0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.538+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 shutdown_connections 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.538+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55950b3370c0 0x55950b33b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55950b3282f0 0x55950b33b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55950b3b24c0 0x55950b3b48e0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d19dfb740 1 --2- 172.21.15.137:0/1349340623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55950b3279d0 0x55950b33af30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5ceffff640 0 client.17325 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 >> 172.21.15.137:0/1349340623 conn(0x55950b324060 msgr2=0x55950b335860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 shutdown_connections 2024-09-12T12:28:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d19dfb740 1 -- 172.21.15.137:0/1349340623 wait complete. 2024-09-12T12:28:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d19dfb740 20 client.17325 trim_cache size 0 max 16384 2024-09-12T12:28:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:28:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:28:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:28:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5d000a2da0 msgr2=0x7f5d000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5d000a2da0 0x7f5d000a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5d08020000 tx=0x7f5d08021000 comp rx=0 tx=0).stop 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 msgr2=0x7f5d10198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10198380 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e9f0 tx=0x7f5d0400eeb0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 shutdown_connections 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5d000a2da0 0x7f5d000a5260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d10107b70 0x7f5d1019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d10102470 0x7f5d101988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.539+0000 7f5d1705e640 1 --2- 172.21.15.137:0/1936029820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d10101ad0 0x7f5d10198380 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:28:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.540+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 >> 172.21.15.137:0/1936029820 conn(0x7f5d100fd7d0 msgr2=0x7f5d100fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:28:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.540+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 shutdown_connections 2024-09-12T12:28:59.543 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:28:59.540+0000 7f5d1705e640 1 -- 172.21.15.137:0/1936029820 wait complete. 2024-09-12T12:29:00.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 53756 37.46 MB/sec execute 32 sec latency 210.232 ms 2024-09-12T12:29:01.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:00 smithi137 ceph-mon[30950]: pgmap v500: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 813 op/s 2024-09-12T12:29:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:00 smithi189 ceph-mon[35475]: pgmap v500: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 813 op/s 2024-09-12T12:29:01.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:00 smithi171 ceph-mon[35098]: pgmap v500: 97 pgs: 97 active+clean; 770 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.0 MiB/s wr, 813 op/s 2024-09-12T12:29:01.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 54413 37.21 MB/sec execute 33 sec latency 144.716 ms 2024-09-12T12:29:02.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 54737 36.85 MB/sec execute 34 sec latency 125.764 ms 2024-09-12T12:29:03.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:02 smithi137 ceph-mon[30950]: pgmap v501: 97 pgs: 97 active+clean; 774 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.8 MiB/s wr, 561 op/s 2024-09-12T12:29:03.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:02 smithi189 ceph-mon[35475]: pgmap v501: 97 pgs: 97 active+clean; 774 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.8 MiB/s wr, 561 op/s 2024-09-12T12:29:03.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:02 smithi171 ceph-mon[35098]: pgmap v501: 97 pgs: 97 active+clean; 774 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.8 MiB/s wr, 561 op/s 2024-09-12T12:29:03.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 55087 36.30 MB/sec execute 35 sec latency 404.064 ms 2024-09-12T12:29:04.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:03 smithi137 ceph-mon[30950]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T12:29:04.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:03 smithi189 ceph-mon[35475]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T12:29:04.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:03 smithi171 ceph-mon[35098]: osdmap e110: 12 total, 12 up, 12 in 2024-09-12T12:29:04.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 55465 35.64 MB/sec execute 36 sec latency 248.181 ms 2024-09-12T12:29:05.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:04 smithi137 ceph-mon[30950]: pgmap v503: 97 pgs: 97 active+clean; 774 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-09-12T12:29:05.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:04 smithi189 ceph-mon[35475]: pgmap v503: 97 pgs: 97 active+clean; 774 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-09-12T12:29:05.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:04 smithi171 ceph-mon[35098]: pgmap v503: 97 pgs: 97 active+clean; 774 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.9 MiB/s wr, 655 op/s 2024-09-12T12:29:05.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 56132 35.73 MB/sec execute 37 sec latency 274.992 ms 2024-09-12T12:29:06.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 56990 35.97 MB/sec execute 38 sec latency 42.606 ms 2024-09-12T12:29:07.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:06 smithi189 ceph-mon[35475]: pgmap v504: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 784 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 840 op/s 2024-09-12T12:29:07.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:06 smithi171 ceph-mon[35098]: pgmap v504: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 784 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 840 op/s 2024-09-12T12:29:07.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 57734 36.19 MB/sec execute 39 sec latency 42.867 ms 2024-09-12T12:29:07.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:06 smithi137 ceph-mon[30950]: pgmap v504: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 784 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 840 op/s 2024-09-12T12:29:08.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 58419 36.16 MB/sec execute 40 sec latency 54.102 ms 2024-09-12T12:29:09.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:08 smithi189 ceph-mon[35475]: pgmap v505: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 784 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 840 op/s 2024-09-12T12:29:09.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:08 smithi171 ceph-mon[35098]: pgmap v505: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 784 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 840 op/s 2024-09-12T12:29:09.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 59170 36.23 MB/sec execute 41 sec latency 54.983 ms 2024-09-12T12:29:09.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:08 smithi137 ceph-mon[30950]: pgmap v505: 97 pgs: 10 active+clean+snaptrim_wait, 5 active+clean+snaptrim, 82 active+clean; 784 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 840 op/s 2024-09-12T12:29:10.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 59708 36.13 MB/sec execute 42 sec latency 244.897 ms 2024-09-12T12:29:11.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:10 smithi189 ceph-mon[35475]: pgmap v506: 97 pgs: 97 active+clean; 833 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2024-09-12T12:29:11.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:10 smithi171 ceph-mon[35098]: pgmap v506: 97 pgs: 97 active+clean; 833 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2024-09-12T12:29:11.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 60521 36.14 MB/sec execute 43 sec latency 114.249 ms 2024-09-12T12:29:11.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:10 smithi137 ceph-mon[30950]: pgmap v506: 97 pgs: 97 active+clean; 833 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 749 op/s 2024-09-12T12:29:12.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 61347 36.61 MB/sec execute 44 sec latency 48.073 ms 2024-09-12T12:29:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 -- 172.21.15.137:0/2215531624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024101ad0 msgr2=0x7fd024101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2215531624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024101ad0 0x7fd024101f30 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd014009930 tx=0x7fd01402a420 comp rx=0 tx=0).stop 2024-09-12T12:29:12.648 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 -- 172.21.15.137:0/2215531624 shutdown_connections 2024-09-12T12:29:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2215531624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd024102470 0x7fd02410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2215531624 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024101ad0 0x7fd024101f30 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2215531624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd024107b70 0x7fd024107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 -- 172.21.15.137:0/2215531624 >> 172.21.15.137:0/2215531624 conn(0x7fd0240fd7d0 msgr2=0x7fd0240ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 -- 172.21.15.137:0/2215531624 shutdown_connections 2024-09-12T12:29:12.649 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.647+0000 7fd02c08d640 1 -- 172.21.15.137:0/2215531624 wait complete. 2024-09-12T12:29:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.648+0000 7fd02c08d640 1 Processor -- start 2024-09-12T12:29:12.651 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.648+0000 7fd02c08d640 1 -- start start 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02c08d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd024101ad0 0x7fd024108a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02c08d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd024102470 0x7fd024108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02c08d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024107b70 0x7fd0241094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02c08d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd024111f20 con 0x7fd024107b70 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02c08d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd024111da0 con 0x7fd024101ad0 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02c08d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241120a0 con 0x7fd024102470 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024107b70 0x7fd0241094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024107b70 0x7fd0241094c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41782/0 (socket says 172.21.15.137:41782) 2024-09-12T12:29:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 -- 172.21.15.137:0/2832507834 learned_addr learned my addr 172.21.15.137:0/2832507834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd029e29640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd024101ad0 0x7fd024108a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd029628640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd024102470 0x7fd024108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 -- 172.21.15.137:0/2832507834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd024102470 msgr2=0x7fd024108f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd024102470 0x7fd024108f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 -- 172.21.15.137:0/2832507834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd024101ad0 msgr2=0x7fd024108a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd024101ad0 0x7fd024108a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 -- 172.21.15.137:0/2832507834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0241aabf0 con 0x7fd024107b70 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd029628640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd024102470 0x7fd024108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:29:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.649+0000 7fd02a62a640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024107b70 0x7fd0241094c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00eb40 tx=0x7fd01c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.650+0000 7fd01affd640 1 -- 172.21.15.137:0/2832507834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c004b20 con 0x7fd024107b70 2024-09-12T12:29:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.650+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0241aaee0 con 0x7fd024107b70 2024-09-12T12:29:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.650+0000 7fd01affd640 1 -- 172.21.15.137:0/2832507834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c004cc0 con 0x7fd024107b70 2024-09-12T12:29:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.650+0000 7fd01affd640 1 -- 172.21.15.137:0/2832507834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c010760 con 0x7fd024107b70 2024-09-12T12:29:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.650+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0241ab3b0 con 0x7fd024107b70 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.652+0000 7fd01affd640 1 -- 172.21.15.137:0/2832507834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd01c005020 con 0x7fd024107b70 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 Processor -- start 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 -- start start 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd01affd640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd0100a2da0 0x7fd0100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af011770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5563aef6b2f0 con 0x5563af011cb0 2024-09-12T12:29:12.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5563aef13270 con 0x5563af006af0 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5563aef672b0 con 0x5563af007410 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd029628640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd0100a2da0 0x7fd0100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd029e29640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd02a62a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd029e29640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af006ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39388/0 (socket says 172.21.15.137:39388) 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd029e29640 1 -- 172.21.15.137:0/3916124354 learned_addr learned my addr 172.21.15.137:0/3916124354 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd029628640 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af011770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.653+0000 7fd01affd640 1 -- 172.21.15.137:0/2832507834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8528+0+0 (secure 0 0 0) 0x7fd01c0cfa90 con 0x7fd024107b70 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd02a62a640 1 -- 172.21.15.137:0/3916124354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 msgr2=0x5563af011770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd02a62a640 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af011770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd02a62a640 1 -- 172.21.15.137:0/3916124354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 msgr2=0x5563af006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd02a62a640 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd02a62a640 1 -- 172.21.15.137:0/3916124354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563aef12340 con 0x5563af011cb0 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd029e29640 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af006ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd029628640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd0100a2da0 0x7fd0100a5260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd014000c00 tx=0x7fd014009680 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd02a62a640 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0140a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0cd1e0 tx=0x7fd01c09bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96a09781b82d5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd00ffff640 1 -- 172.21.15.137:0/3916124354 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c00d340 con 0x5563af011cb0 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd00ffff640 1 -- 172.21.15.137:0/3916124354 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c09c6b0 con 0x5563af011cb0 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.654+0000 7fd00ffff640 1 -- 172.21.15.137:0/3916124354 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c098620 con 0x5563af011cb0 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 -- 172.21.15.137:0/3916124354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 msgr2=0x5563af0140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0140a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0cd1e0 tx=0x7fd01c09bd80 comp rx=0 tx=0).stop 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 -- 172.21.15.137:0/3916124354 shutdown_connections 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0140a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af011770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/3916124354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af006ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 -- 172.21.15.137:0/3916124354 >> 172.21.15.137:0/3916124354 conn(0x5563af003060 msgr2=0x5563af003440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 -- 172.21.15.137:0/3916124354 shutdown_connections 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 -- 172.21.15.137:0/3916124354 wait complete. 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.655+0000 7fd02ee2c740 1 -- start start 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 1 Processor -- start 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af00eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af00f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af00f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5563af014880 con 0x5563af006af0 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5563aeef4290 con 0x5563af007410 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5563af014a00 con 0x5563af011cb0 2024-09-12T12:29:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02ee2c740 10 client.0 _ll_register_callbacks cb 0x5563aef25140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af00f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029e29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af00eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af00f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39394/0 (socket says 172.21.15.137:39394) 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 -- 172.21.15.137:0/2609501410 learned_addr learned my addr 172.21.15.137:0/2609501410 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd02a62a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af00f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 -- 172.21.15.137:0/2609501410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 msgr2=0x5563af00f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af00f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 -- 172.21.15.137:0/2609501410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 msgr2=0x5563af00eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af00eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.656+0000 7fd029628640 1 -- 172.21.15.137:0/2609501410 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563aec7e210 con 0x5563af007410 2024-09-12T12:29:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd029628640 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af00f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd014009930 tx=0x7fd014043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22a6693babf4848a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd00d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd00d7fa640 1 -- 172.21.15.137:0/2609501410 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01404f070 con 0x5563af007410 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd00d7fa640 1 -- 172.21.15.137:0/2609501410 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8528+0+0 (secure 0 0 0) 0x7fd0140035f0 con 0x5563af007410 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01c09c6b0 con 0x5563af007410 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd00d7fa640 1 -- 172.21.15.137:0/2609501410 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd01405b050 con 0x5563af007410 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd02ee2c740 10 client.26621 fetch_fsmap learned FSMap version 48 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5563aed4a620 con 0x5563af007410 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd00d7fa640 1 -- 172.21.15.137:0/2609501410 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd01c09c6b0 con 0x5563af007410 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.657+0000 7fd00d7fa640 1 -- 172.21.15.137:0/2609501410 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fd014048040 con 0x5563af007410 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd02ee2c740 10 client.26621 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd02ee2c740 10 client.26621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:12.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd02ee2c740 20 client.26621 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af021c10 0x5563af024040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd02ee2c740 4 client.26621 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:29:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563aef78a20 con 0x5563af021c10 2024-09-12T12:29:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd029e29640 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af021c10 0x5563af024040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd029e29640 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af021c10 0x5563af024040 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8326ff72ddffc0ab server_cookie=34493cdd9751a975 in_seq=0 out_seq=0 2024-09-12T12:29:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.658+0000 7fd00d7fa640 10 client.26621 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.663+0000 7fd00d7fa640 1 -- 172.21.15.137:0/2609501410 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563aef78a20 con 0x5563af021c10 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.663+0000 7fd00d7fa640 10 client.26621 handle_command_reply: tid=0 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.663+0000 7fd02ee2c740 1 client.26621 shutdown 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 msgr2=0x5563af00f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af00f0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd014009930 tx=0x7fd014043000 comp rx=0 tx=0).stop 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 shutdown_connections 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af00f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af00f0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af021c10 0x5563af024040 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd00d7fa640 0 client.26621 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2609501410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af00eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 >> 172.21.15.137:0/2609501410 conn(0x5563af003060 msgr2=0x5563af008d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 shutdown_connections 2024-09-12T12:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2609501410 wait complete. 2024-09-12T12:29:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.664+0000 7fd02ee2c740 20 client.26621 trim_cache size 0 max 16384 2024-09-12T12:29:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 Processor -- start 2024-09-12T12:29:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 -- start start 2024-09-12T12:29:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af00f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af010b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241b66e0 con 0x5563af011cb0 2024-09-12T12:29:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241b3ef0 con 0x5563af006af0 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.680+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241b1d40 con 0x5563af007410 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af00f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd02a62a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd02a62a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0299a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41796/0 (socket says 172.21.15.137:41796) 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af00f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39404/0 (socket says 172.21.15.137:39404) 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 -- 172.21.15.137:0/2920115962 learned_addr learned my addr 172.21.15.137:0/2920115962 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029628640 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af010b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 -- 172.21.15.137:0/2920115962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 msgr2=0x5563af010b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af010b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 -- 172.21.15.137:0/2920115962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 msgr2=0x5563af0299a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0299a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 -- 172.21.15.137:0/2920115962 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563af0246c0 con 0x5563af006af0 2024-09-12T12:29:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd029e29640 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af00f440 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5563af024940 tx=0x7fd020005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41437eb7e75bd115 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd00f7fe640 1 -- 172.21.15.137:0/2920115962 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020004230 con 0x5563af006af0 2024-09-12T12:29:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd00f7fe640 1 -- 172.21.15.137:0/2920115962 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0200043d0 con 0x5563af006af0 2024-09-12T12:29:12.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.681+0000 7fd00f7fe640 1 -- 172.21.15.137:0/2920115962 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0200035f0 con 0x5563af006af0 2024-09-12T12:29:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2920115962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 msgr2=0x5563af00f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af00f440 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5563af024940 tx=0x7fd020005f90 comp rx=0 tx=0).stop 2024-09-12T12:29:12.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2920115962 shutdown_connections 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af011cb0 0x5563af0299a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af007410 0x5563af010b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/2920115962 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af006af0 0x5563af00f440 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2920115962 >> 172.21.15.137:0/2920115962 conn(0x5563af003060 msgr2=0x5563af0103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2920115962 shutdown_connections 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.682+0000 7fd02ee2c740 1 -- 172.21.15.137:0/2920115962 wait complete. 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 -- start start 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 Processor -- start 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af01a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af0155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af015ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241b23b0 con 0x5563af006af0 2024-09-12T12:29:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241b3820 con 0x5563af007410 2024-09-12T12:29:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0241b2a80 con 0x5563af011cb0 2024-09-12T12:29:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02ee2c740 10 client.0 _ll_register_callbacks cb 0x7fd01c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd029e29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af01a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd029e29640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af01a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41810/0 (socket says 172.21.15.137:41810) 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd029628640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af0155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd029e29640 1 -- 172.21.15.137:0/101046488 learned_addr learned my addr 172.21.15.137:0/101046488 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.683+0000 7fd02a62a640 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af015ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd029628640 1 -- 172.21.15.137:0/101046488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 msgr2=0x5563af015ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd029628640 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af015ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd029628640 1 -- 172.21.15.137:0/101046488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 msgr2=0x5563af01a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd029628640 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af01a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd029628640 1 -- 172.21.15.137:0/101046488 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563af016180 con 0x5563af007410 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd029628640 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af0155a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd01403d7b0 tx=0x7fd01403d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd6c9c0295443d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd00d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:29:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd00d7fa640 1 -- 172.21.15.137:0/101046488 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01404f070 con 0x5563af007410 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0200043d0 con 0x5563af007410 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd00d7fa640 1 -- 172.21.15.137:0/101046488 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8528+0+0 (secure 0 0 0) 0x7fd01403dc30 con 0x5563af007410 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd00d7fa640 1 -- 172.21.15.137:0/101046488 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd014066050 con 0x5563af007410 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd02ee2c740 10 client.26627 fetch_fsmap learned FSMap version 48 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.684+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5563af08c810 con 0x5563af007410 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd00d7fa640 1 -- 172.21.15.137:0/101046488 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0200043d0 con 0x5563af007410 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd00d7fa640 1 -- 172.21.15.137:0/101046488 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fd014048040 con 0x5563af007410 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd02ee2c740 10 client.26627 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd02ee2c740 10 client.26627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd02ee2c740 20 client.26627 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af08cf90 0x5563af08f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd02ee2c740 4 client.26627 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:29:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563aef72d70 con 0x5563af08cf90 2024-09-12T12:29:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd029e29640 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af08cf90 0x5563af08f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd00d7fa640 10 client.26627 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.685+0000 7fd029e29640 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af08cf90 0x5563af08f3b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47483a5910bdb871 server_cookie=a1898b8c942def00 in_seq=0 out_seq=0 2024-09-12T12:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd00d7fa640 1 -- 172.21.15.137:0/101046488 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563aef72d70 con 0x5563af08cf90 2024-09-12T12:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd00d7fa640 10 client.26627 handle_command_reply: tid=0 2024-09-12T12:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 client.26627 shutdown 2024-09-12T12:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 msgr2=0x5563af0155a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af0155a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd01403d7b0 tx=0x7fd01403d6b0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 shutdown_connections 2024-09-12T12:29:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563af011cb0 0x5563af015ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5563af007410 0x5563af0155a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5563af08cf90 0x5563af08f3b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 --2- 172.21.15.137:0/101046488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5563af006af0 0x5563af01a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd00d7fa640 0 client.26627 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.966+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 >> 172.21.15.137:0/101046488 conn(0x5563af003060 msgr2=0x5563af008d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 shutdown_connections 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02ee2c740 1 -- 172.21.15.137:0/101046488 wait complete. 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02ee2c740 20 client.26627 trim_cache size 0 max 16384 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:29:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:29:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:29:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:29:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd0100a2da0 msgr2=0x7fd0100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd0100a2da0 0x7fd0100a5260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd014000c00 tx=0x7fd014009680 comp rx=0 tx=0).stop 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024107b70 msgr2=0x7fd0241094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024107b70 0x7fd0241094c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00eb40 tx=0x7fd01c00c2d0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 shutdown_connections 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd0100a2da0 0x7fd0100a5260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd024107b70 0x7fd0241094c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd024102470 0x7fd024108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 --2- 172.21.15.137:0/2832507834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd024101ad0 0x7fd024108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 >> 172.21.15.137:0/2832507834 conn(0x7fd0240fd7d0 msgr2=0x7fd02410d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 shutdown_connections 2024-09-12T12:29:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:12.967+0000 7fd02c08d640 1 -- 172.21.15.137:0/2832507834 wait complete. 2024-09-12T12:29:13.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:12 smithi189 ceph-mon[35475]: pgmap v507: 97 pgs: 97 active+clean; 845 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-12T12:29:13.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:12 smithi189 ceph-mon[35475]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T12:29:13.100 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:12 smithi137 ceph-mon[30950]: pgmap v507: 97 pgs: 97 active+clean; 845 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-12T12:29:13.100 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:12 smithi137 ceph-mon[30950]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T12:29:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:12 smithi171 ceph-mon[35098]: pgmap v507: 97 pgs: 97 active+clean; 845 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-12T12:29:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:12 smithi171 ceph-mon[35098]: osdmap e111: 12 total, 12 up, 12 in 2024-09-12T12:29:13.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 61858 36.26 MB/sec execute 45 sec latency 284.142 ms 2024-09-12T12:29:14.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 62390 36.12 MB/sec execute 46 sec latency 97.203 ms 2024-09-12T12:29:15.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:14 smithi189 ceph-mon[35475]: pgmap v509: 97 pgs: 97 active+clean; 845 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-12T12:29:15.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:14 smithi171 ceph-mon[35098]: pgmap v509: 97 pgs: 97 active+clean; 845 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-12T12:29:15.220 INFO:tasks.workunit.client.0.smithi137.stdout: 10 62994 36.02 MB/sec execute 47 sec latency 241.167 ms 2024-09-12T12:29:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:14 smithi137 ceph-mon[30950]: pgmap v509: 97 pgs: 97 active+clean; 845 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-12T12:29:16.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 63852 36.22 MB/sec execute 48 sec latency 63.870 ms 2024-09-12T12:29:17.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:16 smithi189 ceph-mon[35475]: pgmap v510: 97 pgs: 97 active+clean; 844 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-09-12T12:29:17.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:16 smithi171 ceph-mon[35098]: pgmap v510: 97 pgs: 97 active+clean; 844 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-09-12T12:29:17.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 64620 36.32 MB/sec execute 49 sec latency 141.722 ms 2024-09-12T12:29:17.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:16 smithi137 ceph-mon[30950]: pgmap v510: 97 pgs: 97 active+clean; 844 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-09-12T12:29:18.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 65200 36.33 MB/sec execute 50 sec latency 98.383 ms 2024-09-12T12:29:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:18 smithi189 ceph-mon[35475]: pgmap v511: 97 pgs: 97 active+clean; 844 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-09-12T12:29:19.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:18 smithi171 ceph-mon[35098]: pgmap v511: 97 pgs: 97 active+clean; 844 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-09-12T12:29:19.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 65853 36.24 MB/sec execute 51 sec latency 95.635 ms 2024-09-12T12:29:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:18 smithi137 ceph-mon[30950]: pgmap v511: 97 pgs: 97 active+clean; 844 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 754 op/s 2024-09-12T12:29:20.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 66457 36.06 MB/sec execute 52 sec latency 227.019 ms 2024-09-12T12:29:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:20 smithi189 ceph-mon[35475]: pgmap v512: 97 pgs: 97 active+clean; 857 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.8 MiB/s wr, 629 op/s 2024-09-12T12:29:21.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:20 smithi171 ceph-mon[35098]: pgmap v512: 97 pgs: 97 active+clean; 857 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.8 MiB/s wr, 629 op/s 2024-09-12T12:29:21.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 67383 36.32 MB/sec execute 53 sec latency 104.888 ms 2024-09-12T12:29:21.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:20 smithi137 ceph-mon[30950]: pgmap v512: 97 pgs: 97 active+clean; 857 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.8 MiB/s wr, 629 op/s 2024-09-12T12:29:22.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 68109 36.57 MB/sec execute 54 sec latency 76.910 ms 2024-09-12T12:29:23.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:22 smithi189 ceph-mon[35475]: pgmap v513: 97 pgs: 97 active+clean; 876 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 812 op/s 2024-09-12T12:29:23.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:22 smithi171 ceph-mon[35098]: pgmap v513: 97 pgs: 97 active+clean; 876 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 812 op/s 2024-09-12T12:29:23.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 68729 36.45 MB/sec execute 55 sec latency 87.071 ms 2024-09-12T12:29:23.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:22 smithi137 ceph-mon[30950]: pgmap v513: 97 pgs: 97 active+clean; 876 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 812 op/s 2024-09-12T12:29:24.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 69419 36.29 MB/sec execute 56 sec latency 106.228 ms 2024-09-12T12:29:25.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:24 smithi189 ceph-mon[35475]: pgmap v514: 97 pgs: 97 active+clean; 876 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 744 op/s 2024-09-12T12:29:25.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:24 smithi171 ceph-mon[35098]: pgmap v514: 97 pgs: 97 active+clean; 876 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 744 op/s 2024-09-12T12:29:25.221 INFO:tasks.workunit.client.0.smithi137.stdout: 10 70033 36.27 MB/sec execute 57 sec latency 299.511 ms 2024-09-12T12:29:25.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:24 smithi137 ceph-mon[30950]: pgmap v514: 97 pgs: 97 active+clean; 876 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 744 op/s 2024-09-12T12:29:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 -- 172.21.15.137:0/2989165325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 msgr2=0x7fb57c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 --2- 172.21.15.137:0/2989165325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c106dc0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb578009930 tx=0x7fb57802a420 comp rx=0 tx=0).stop 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 -- 172.21.15.137:0/2989165325 shutdown_connections 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 --2- 172.21.15.137:0/2989165325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb57c101370 0x7fb57c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 --2- 172.21.15.137:0/2989165325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb57c1009d0 0x7fb57c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 --2- 172.21.15.137:0/2989165325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c106dc0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 -- 172.21.15.137:0/2989165325 >> 172.21.15.137:0/2989165325 conn(0x7fb57c0fc6b0 msgr2=0x7fb57c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 -- 172.21.15.137:0/2989165325 shutdown_connections 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.090+0000 7fb584077640 1 -- 172.21.15.137:0/2989165325 wait complete. 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 Processor -- start 2024-09-12T12:29:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 -- start start 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb57c1009d0 0x7fb57c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb57c101370 0x7fb57c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c110e20 con 0x7fb57c1069e0 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c110ca0 con 0x7fb57c101370 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.091+0000 7fb584077640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c110fa0 con 0x7fb57c1009d0 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb581612640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb57c101370 0x7fb57c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41238/0 (socket says 172.21.15.137:41238) 2024-09-12T12:29:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 -- 172.21.15.137:0/861486814 learned_addr learned my addr 172.21.15.137:0/861486814 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb581e13640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb57c1009d0 0x7fb57c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 -- 172.21.15.137:0/861486814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb57c1009d0 msgr2=0x7fb57c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb57c1009d0 0x7fb57c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 -- 172.21.15.137:0/861486814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb57c101370 msgr2=0x7fb57c197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb57c101370 0x7fb57c197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 -- 172.21.15.137:0/861486814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c19c2e0 con 0x7fb57c1069e0 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb582614640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c19bbb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb57400cce0 tx=0x7fb57400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb572ffd640 1 -- 172.21.15.137:0/861486814 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57400f450 con 0x7fb57c1069e0 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c19c5d0 con 0x7fb57c1069e0 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb572ffd640 1 -- 172.21.15.137:0/861486814 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb57400fad0 con 0x7fb57c1069e0 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c1a3e60 con 0x7fb57c1069e0 2024-09-12T12:29:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.092+0000 7fb572ffd640 1 -- 172.21.15.137:0/861486814 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574018c10 con 0x7fb57c1069e0 2024-09-12T12:29:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.095+0000 7fb586e0c740 1 Processor -- start 2024-09-12T12:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.096+0000 7fb572ffd640 1 -- 172.21.15.137:0/861486814 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb57400f5f0 con 0x7fb57c1069e0 2024-09-12T12:29:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.096+0000 7fb586e0c740 1 -- start start 2024-09-12T12:29:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb586e0c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc422132f0 con 0x55dc422b23b0 2024-09-12T12:29:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb586e0c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55dc421bb270 con 0x55dc422bdd70 2024-09-12T12:29:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb586e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc4220f2b0 con 0x55dc422b11f0 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41250/0 (socket says 172.21.15.137:41250) 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 -- 172.21.15.137:0/2586310033 learned_addr learned my addr 172.21.15.137:0/2586310033 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581e13640 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb572ffd640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5680a2e20 0x7fb5680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 -- 172.21.15.137:0/2586310033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 msgr2=0x55dc422b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 -- 172.21.15.137:0/2586310033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 msgr2=0x55dc422b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.097+0000 7fb581612640 1 -- 172.21.15.137:0/2586310033 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc421ba340 con 0x55dc422bdd70 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.098+0000 7fb572ffd640 1 -- 172.21.15.137:0/861486814 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fb5740cfb10 con 0x7fb57c1069e0 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.098+0000 7fb582614640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5680a2e20 0x7fb5680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.098+0000 7fb581612640 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422c01a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00bfd0 tx=0x7fb56c00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca3ff29f2fc083c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.098+0000 7fb567fff640 1 -- 172.21.15.137:0/2586310033 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0096a0 con 0x55dc422bdd70 2024-09-12T12:29:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.098+0000 7fb567fff640 1 -- 172.21.15.137:0/2586310033 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56c009d20 con 0x55dc422bdd70 2024-09-12T12:29:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.098+0000 7fb567fff640 1 -- 172.21.15.137:0/2586310033 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c010c90 con 0x55dc422bdd70 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb582614640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5680a2e20 0x7fb5680a52e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb57400ab20 tx=0x7fb57400af30 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 -- 172.21.15.137:0/2586310033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 msgr2=0x55dc422c01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422c01a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00bfd0 tx=0x7fb56c00b390 comp rx=0 tx=0).stop 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 -- 172.21.15.137:0/2586310033 shutdown_connections 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422c01a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 --2- 172.21.15.137:0/2586310033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 -- 172.21.15.137:0/2586310033 >> 172.21.15.137:0/2586310033 conn(0x55dc422ad270 msgr2=0x55dc422ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 -- 172.21.15.137:0/2586310033 shutdown_connections 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.099+0000 7fb586e0c740 1 -- 172.21.15.137:0/2586310033 wait complete. 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 -- start start 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 Processor -- start 2024-09-12T12:29:26.103 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422bb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422bb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55dc422c0c60 con 0x55dc422b23b0 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55dc4219c290 con 0x55dc422bdd70 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dc422c0de0 con 0x55dc422b11f0 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.100+0000 7fb586e0c740 10 client.0 _ll_register_callbacks cb 0x55dc421cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422bb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422bb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41262/0 (socket says 172.21.15.137:41262) 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 -- 172.21.15.137:0/3107692710 learned_addr learned my addr 172.21.15.137:0/3107692710 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581612640 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422bb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 -- 172.21.15.137:0/3107692710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 msgr2=0x55dc422bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 -- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 msgr2=0x55dc422bb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.104 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422bb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 -- 172.21.15.137:0/3107692710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc41f26210 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581612640 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422bb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb581e13640 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422bb290 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb578009930 tx=0x7fb5780319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc84af42a40d650e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb5657fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb5657fa640 1 -- 172.21.15.137:0/3107692710 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578035040 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb5657fa640 1 -- 172.21.15.137:0/3107692710 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fb578004240 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb56c009d20 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb5657fa640 1 -- 172.21.15.137:0/3107692710 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb578004440 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb586e0c740 10 client.17355 fetch_fsmap learned FSMap version 48 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.101+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55dc41ff2620 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb5657fa640 1 -- 172.21.15.137:0/3107692710 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56c009d20 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb5657fa640 1 -- 172.21.15.137:0/3107692710 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fb578004bf0 con 0x55dc422b23b0 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb586e0c740 10 client.17355 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:29:26.105 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb586e0c740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb586e0c740 20 client.17355 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb586e0c740 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc422d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb586e0c740 4 client.17355 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc42220a20 con 0x55dc422ce170 2024-09-12T12:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb581612640 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc422d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb5657fa640 10 client.17355 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:26.106 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.102+0000 7fb581612640 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc422d0580 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc0cc486e47fad4 server_cookie=5b6d8f1b7de91d2 in_seq=0 out_seq=0 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.103+0000 7fb5657fa640 1 -- 172.21.15.137:0/3107692710 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc42220a20 con 0x55dc422ce170 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.103+0000 7fb5657fa640 10 client.17355 handle_command_reply: tid=0 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.103+0000 7fb586e0c740 1 client.17355 shutdown 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 msgr2=0x55dc422bb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422bb290 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb578009930 tx=0x7fb5780319b0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 shutdown_connections 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422bb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.107 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc422d0580 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb5657fa640 0 client.17355 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422bb290 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 --2- 172.21.15.137:0/3107692710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 >> 172.21.15.137:0/3107692710 conn(0x55dc422ad270 msgr2=0x55dc422b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 shutdown_connections 2024-09-12T12:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 1 -- 172.21.15.137:0/3107692710 wait complete. 2024-09-12T12:29:26.108 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.104+0000 7fb586e0c740 20 client.17355 trim_cache size 0 max 16384 2024-09-12T12:29:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.122+0000 7fb586e0c740 1 Processor -- start 2024-09-12T12:29:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.122+0000 7fb586e0c740 1 -- start start 2024-09-12T12:29:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422b11f0 0x55dc422ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b23b0 0x55dc422bb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422bdd70 0x55dc422d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb586e0c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1adf10 con 0x55dc422bdd70 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb586e0c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1ab720 con 0x55dc422b11f0 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb586e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1a9590 con 0x55dc422b23b0 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422bdd70 0x55dc422d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422bdd70 0x55dc422d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41264/0 (socket says 172.21.15.137:41264) 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 -- 172.21.15.137:0/221763964 learned_addr learned my addr 172.21.15.137:0/221763964 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581e13640 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b23b0 0x55dc422bb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.123 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422b11f0 0x55dc422ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 -- 172.21.15.137:0/221763964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b23b0 msgr2=0x55dc422bb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b23b0 0x55dc422bb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 -- 172.21.15.137:0/221763964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422b11f0 msgr2=0x55dc422ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422b11f0 0x55dc422ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 -- 172.21.15.137:0/221763964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc422acb10 con 0x55dc422bdd70 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.123+0000 7fb581612640 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422bdd70 0x55dc422d5e90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0127a0 tx=0x7fb56c010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9155ee647bd6b72c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.124+0000 7fb5677fe640 1 -- 172.21.15.137:0/221763964 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c02e070 con 0x55dc422bdd70 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.124+0000 7fb5677fe640 1 -- 172.21.15.137:0/221763964 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56c010e80 con 0x55dc422bdd70 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.124+0000 7fb5677fe640 1 -- 172.21.15.137:0/221763964 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c009c10 con 0x55dc422bdd70 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.124+0000 7fb586e0c740 1 -- 172.21.15.137:0/221763964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422bdd70 msgr2=0x55dc422d5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.124+0000 7fb586e0c740 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422bdd70 0x55dc422d5e90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0127a0 tx=0x7fb56c010890 comp rx=0 tx=0).stop 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 -- 172.21.15.137:0/221763964 shutdown_connections 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422bdd70 0x55dc422d5e90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b23b0 0x55dc422bb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 --2- 172.21.15.137:0/221763964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422b11f0 0x55dc422ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 -- 172.21.15.137:0/221763964 >> 172.21.15.137:0/221763964 conn(0x55dc422ad270 msgr2=0x55dc422abc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 -- 172.21.15.137:0/221763964 shutdown_connections 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 -- 172.21.15.137:0/221763964 wait complete. 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 -- start start 2024-09-12T12:29:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.125+0000 7fb586e0c740 1 Processor -- start 2024-09-12T12:29:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb586e0c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb586e0c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1a9be0 con 0x55dc422b23b0 2024-09-12T12:29:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb586e0c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1ab050 con 0x55dc422bdd70 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb586e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c1aa2b0 con 0x55dc422b11f0 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb586e0c740 10 client.0 _ll_register_callbacks cb 0x7fb5780314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41280/0 (socket says 172.21.15.137:41280) 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 -- 172.21.15.137:0/4157175770 learned_addr learned my addr 172.21.15.137:0/4157175770 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581612640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581612640 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 -- 172.21.15.137:0/4157175770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 msgr2=0x55dc422c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 -- 172.21.15.137:0/4157175770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 msgr2=0x55dc422d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.126+0000 7fb581e13640 1 -- 172.21.15.137:0/4157175770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc422d5ef0 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb581e13640 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422c6960 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55dc422bc390 tx=0x7fb5780045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3303cdd9b173573 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb5657fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb5657fa640 1 -- 172.21.15.137:0/4157175770 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb578035040 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb5657fa640 1 -- 172.21.15.137:0/4157175770 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fb578003fc0 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc4221ad70 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb5657fa640 1 -- 172.21.15.137:0/4157175770 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb578009520 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb586e0c740 10 client.17361 fetch_fsmap learned FSMap version 48 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55dc42338b80 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.127+0000 7fb5657fa640 1 -- 172.21.15.137:0/4157175770 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dc4221ad70 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb5657fa640 1 -- 172.21.15.137:0/4157175770 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fb57805f020 con 0x55dc422b23b0 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb586e0c740 10 client.17361 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb586e0c740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb586e0c740 20 client.17361 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb586e0c740 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc4233b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb586e0c740 4 client.17361 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc42197900 con 0x55dc422ce170 2024-09-12T12:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb581612640 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc4233b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb5657fa640 10 client.17361 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.128+0000 7fb581612640 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc4233b3d0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fb4582f9343870e server_cookie=b0fd2f8cb887254 in_seq=0 out_seq=0 2024-09-12T12:29:26.222 INFO:tasks.workunit.client.0.smithi137.stdout: 10 70816 36.50 MB/sec execute 58 sec latency 69.384 ms 2024-09-12T12:29:26.403 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb5657fa640 1 -- 172.21.15.137:0/4157175770 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc42197900 con 0x55dc422ce170 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb5657fa640 10 client.17361 handle_command_reply: tid=0 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 client.17361 shutdown 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 msgr2=0x55dc422c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422c6960 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55dc422bc390 tx=0x7fb5780045e0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 shutdown_connections 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55dc422bdd70 0x55dc422d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55dc422ce170 0x55dc4233b3d0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dc422b23b0 0x55dc422c6960 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 --2- 172.21.15.137:0/4157175770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dc422b11f0 0x55dc422c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb5657fa640 0 client.17361 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.403+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 >> 172.21.15.137:0/4157175770 conn(0x55dc422ad270 msgr2=0x55dc422b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:26.404 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.404+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 shutdown_connections 2024-09-12T12:29:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.404+0000 7fb586e0c740 1 -- 172.21.15.137:0/4157175770 wait complete. 2024-09-12T12:29:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.404+0000 7fb586e0c740 20 client.17361 trim_cache size 0 max 16384 2024-09-12T12:29:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:29:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:29:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:29:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:29:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5680a2e20 msgr2=0x7fb5680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5680a2e20 0x7fb5680a52e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb57400ab20 tx=0x7fb57400af30 comp rx=0 tx=0).stop 2024-09-12T12:29:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 msgr2=0x7fb57c19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c19bbb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb57400cce0 tx=0x7fb57400a620 comp rx=0 tx=0).stop 2024-09-12T12:29:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 shutdown_connections 2024-09-12T12:29:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb5680a2e20 0x7fb5680a52e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb57c1069e0 0x7fb57c19bbb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb57c101370 0x7fb57c197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 --2- 172.21.15.137:0/861486814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb57c1009d0 0x7fb57c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 >> 172.21.15.137:0/861486814 conn(0x7fb57c0fc6b0 msgr2=0x7fb57c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 shutdown_connections 2024-09-12T12:29:26.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:26.405+0000 7fb584077640 1 -- 172.21.15.137:0/861486814 wait complete. 2024-09-12T12:29:27.007 INFO:tasks.quiescer.fs.[cephfs]:Total duration (48.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T12:29:27.008 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-12T12:29:27.008 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:29:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:26 smithi189 ceph-mon[35475]: pgmap v515: 97 pgs: 97 active+clean; 897 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 17 MiB/s wr, 979 op/s 2024-09-12T12:29:27.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:26 smithi171 ceph-mon[35098]: pgmap v515: 97 pgs: 97 active+clean; 897 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 17 MiB/s wr, 979 op/s 2024-09-12T12:29:27.222 INFO:tasks.workunit.client.0.smithi137.stdout: 10 71481 36.56 MB/sec execute 59 sec latency 169.543 ms 2024-09-12T12:29:27.282 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:29:27.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:26 smithi137 ceph-mon[30950]: pgmap v515: 97 pgs: 97 active+clean; 897 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 17 MiB/s wr, 979 op/s 2024-09-12T12:29:27.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 -- 172.21.15.137:0/1108774751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102ef0 msgr2=0x7f93d410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:27.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 --2- 172.21.15.137:0/1108774751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102ef0 0x7f93d410da40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009ca0 tx=0x7f93cc02a950 comp rx=0 tx=0).stop 2024-09-12T12:29:27.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 -- 172.21.15.137:0/1108774751 shutdown_connections 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 --2- 172.21.15.137:0/1108774751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102ef0 0x7f93d410da40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 --2- 172.21.15.137:0/1108774751 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d4102570 0x7f93d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 --2- 172.21.15.137:0/1108774751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d4100e20 0x7f93d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 -- 172.21.15.137:0/1108774751 >> 172.21.15.137:0/1108774751 conn(0x7f93d40fc7d0 msgr2=0x7f93d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 -- 172.21.15.137:0/1108774751 shutdown_connections 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.816+0000 7f93dafe6640 1 -- 172.21.15.137:0/1108774751 wait complete. 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 Processor -- start 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 -- start start 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d4100e20 0x7f93d4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102570 0x7f93d4199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d4102ef0 0x7f93d41a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f93d4110fd0 con 0x7f93d4102570 2024-09-12T12:29:27.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f93d4110e50 con 0x7f93d4102ef0 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93dafe6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93d4111150 con 0x7f93d4100e20 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93d8d82640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d4100e20 0x7f93d4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93d8d82640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d4100e20 0x7f93d4199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:44598/0 (socket says 172.21.15.137:44598) 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93d3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102570 0x7f93d4199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93d8d82640 1 -- 172.21.15.137:0/2593581878 learned_addr learned my addr 172.21.15.137:0/2593581878 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.817+0000 7f93d9583640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d4102ef0 0x7f93d41a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d3fff640 1 -- 172.21.15.137:0/2593581878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d4100e20 msgr2=0x7f93d4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d3fff640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d4100e20 0x7f93d4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d3fff640 1 -- 172.21.15.137:0/2593581878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d4102ef0 msgr2=0x7f93d41a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d3fff640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d4102ef0 0x7f93d41a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d3fff640 1 -- 172.21.15.137:0/2593581878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d41a4900 con 0x7f93d4102570 2024-09-12T12:29:27.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d9583640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d4102ef0 0x7f93d41a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d3fff640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102570 0x7f93d4199880 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e9f0 tx=0x7f93c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d1ffb640 1 -- 172.21.15.137:0/2593581878 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c000ccc0 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d41a4ad0 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d1ffb640 1 -- 172.21.15.137:0/2593581878 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c0010910 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93d1ffb640 1 -- 172.21.15.137:0/2593581878 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0018960 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.818+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d41a5010 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.820+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d4101200 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.821+0000 7f93d1ffb640 1 -- 172.21.15.137:0/2593581878 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f93c0010ac0 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.821+0000 7f93d1ffb640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f93c40a2cf0 0x7f93c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.821+0000 7f93d8d82640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f93c40a2cf0 0x7f93c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.821+0000 7f93d1ffb640 1 -- 172.21.15.137:0/2593581878 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f93c00cf740 con 0x7f93d4102570 2024-09-12T12:29:27.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.822+0000 7f93d8d82640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f93c40a2cf0 0x7f93c40a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005fd0 tx=0x7f93c800b480 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:27.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:27.826+0000 7f93d1ffb640 1 -- 172.21.15.137:0/2593581878 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93c009bd30 con 0x7f93d4102570 2024-09-12T12:29:28.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.108+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93d41a51e0 con 0x7f93d4102570 2024-09-12T12:29:28.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.109+0000 7f93d1ffb640 1 -- 172.21.15.137:0/2593581878 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11943 (secure 0 0 0) 0x7f93c00a0030 con 0x7f93d4102570 2024-09-12T12:29:28.110 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:29:28.110 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":48,"btime":"2024-09-12T12:28:53:756310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:28:52.892638+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26647":{"gid":26647,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3747343886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3747343886},{"type":"v1","addr":"172.21.15.189:6835","nonce":3747343886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:29:28.110 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 48 2024-09-12T12:29:28.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f93c40a2cf0 msgr2=0x7f93c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:28.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f93c40a2cf0 0x7f93c40a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f93c8005fd0 tx=0x7f93c800b480 comp rx=0 tx=0).stop 2024-09-12T12:29:28.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102570 msgr2=0x7f93d4199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:28.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102570 0x7f93d4199880 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f93c000e9f0 tx=0x7f93c000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:29:28.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 shutdown_connections 2024-09-12T12:29:28.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f93c40a2cf0 0x7f93c40a51b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:28.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f93d4102ef0 0x7f93d41a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:28.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93d4102570 0x7f93d4199880 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:28.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 --2- 172.21.15.137:0/2593581878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93d4100e20 0x7f93d4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:28.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 >> 172.21.15.137:0/2593581878 conn(0x7f93d40fc7d0 msgr2=0x7f93d4106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:28.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 shutdown_connections 2024-09-12T12:29:28.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:28.111+0000 7f93dafe6640 1 -- 172.21.15.137:0/2593581878 wait complete. 2024-09-12T12:29:28.222 INFO:tasks.workunit.client.0.smithi137.stdout: 10 72034 36.38 MB/sec execute 60 sec latency 44.502 ms 2024-09-12T12:29:28.913 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-12T12:29:28.913 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-12T12:29:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:28 smithi189 ceph-mon[35475]: pgmap v516: 97 pgs: 97 active+clean; 897 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-09-12T12:29:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:28 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2593581878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:29.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:28 smithi171 ceph-mon[35098]: pgmap v516: 97 pgs: 97 active+clean; 897 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-09-12T12:29:29.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:28 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2593581878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:29.206 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:29:29.222 INFO:tasks.workunit.client.0.smithi137.stdout: 10 72820 36.37 MB/sec execute 61 sec latency 71.291 ms 2024-09-12T12:29:29.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:28 smithi137 ceph-mon[30950]: pgmap v516: 97 pgs: 97 active+clean; 897 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-09-12T12:29:29.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:28 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2593581878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:29.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/2536292562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 msgr2=0x7fe4b810c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:29.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/2536292562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b810c9f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009900 tx=0x7fe4a802a410 comp rx=0 tx=0).stop 2024-09-12T12:29:29.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/2536292562 shutdown_connections 2024-09-12T12:29:29.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/2536292562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b810cf30 0x7fe4b8119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/2536292562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b810c9f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/2536292562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b8112640 0x7fe4b8112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/2536292562 >> 172.21.15.137:0/2536292562 conn(0x7fe4b806cdc0 msgr2=0x7fe4b806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:29.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/2536292562 shutdown_connections 2024-09-12T12:29:29.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.728+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/2536292562 wait complete. 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 Processor -- start 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 -- start start 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b81a2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b810cf30 0x7fe4b81a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b8112640 0x7fe4b81a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811c9a0 con 0x7fe4b810c590 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811c820 con 0x7fe4b810cf30 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bfc6a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b811cb20 con 0x7fe4b8112640 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b81a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b81a2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43800/0 (socket says 172.21.15.137:43800) 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 -- 172.21.15.137:0/433016534 learned_addr learned my addr 172.21.15.137:0/433016534 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bd205640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b810cf30 0x7fe4b81a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4be207640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b8112640 0x7fe4b81a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 -- 172.21.15.137:0/433016534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b8112640 msgr2=0x7fe4b81a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b8112640 0x7fe4b81a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 -- 172.21.15.137:0/433016534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b810cf30 msgr2=0x7fe4b81a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b810cf30 0x7fe4b81a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4bda06640 1 -- 172.21.15.137:0/433016534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81a7eb0 con 0x7fe4b810c590 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.729+0000 7fe4be207640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b8112640 0x7fe4b81a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.730+0000 7fe4bda06640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b81a2f00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400ebc0 tx=0x7fe4b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.730+0000 7fe4aeffd640 1 -- 172.21.15.137:0/433016534 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4010630 con 0x7fe4b810c590 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.730+0000 7fe4aeffd640 1 -- 172.21.15.137:0/433016534 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b4010cb0 con 0x7fe4b810c590 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.730+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81a81a0 con 0x7fe4b810c590 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.730+0000 7fe4aeffd640 1 -- 172.21.15.137:0/433016534 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4018bd0 con 0x7fe4b810c590 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.730+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81afa80 con 0x7fe4b810c590 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.732+0000 7fe4aeffd640 1 -- 172.21.15.137:0/433016534 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe4b40107d0 con 0x7fe4b810c590 2024-09-12T12:29:29.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 Processor -- start 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 -- start start 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef4850 0x5566bbef4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbeff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbeffa30 0x5566bbf01e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5566bbe59160 con 0x5566bbeffa30 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5566bbe00720 con 0x5566bbef5170 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4c2043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566bbe55120 con 0x5566bbef4850 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4aeffd640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4a40a2d70 0x7fe4a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbeff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbeff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:45832/0 (socket says 172.21.15.137:45832) 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4bda06640 1 -- 172.21.15.137:0/3988114058 learned_addr learned my addr 172.21.15.137:0/3988114058 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.733+0000 7fe4bd205640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbeffa30 0x5566bbf01e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4be207640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4a40a2d70 0x7fe4a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4aeffd640 1 -- 172.21.15.137:0/433016534 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fe4b40cf9f0 con 0x7fe4b810c590 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4be207640 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef4850 0x5566bbef4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4bda06640 1 -- 172.21.15.137:0/3988114058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef4850 msgr2=0x5566bbef4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4bda06640 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef4850 0x5566bbef4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4bda06640 1 -- 172.21.15.137:0/3988114058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbeffa30 msgr2=0x5566bbf01e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4bda06640 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbeffa30 0x5566bbf01e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.734+0000 7fe4bda06640 1 -- 172.21.15.137:0/3988114058 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bbbcaba0 con 0x5566bbef5170 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.735+0000 7fe4be207640 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef4850 0x5566bbef4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.735+0000 7fe4bd205640 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbeffa30 0x5566bbf01e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.735+0000 7fe4bda06640 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbeff4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4b409bec0 tx=0x7fe4b406efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd0e877b95a13fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.735+0000 7fe4be207640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4a40a2d70 0x7fe4a40a5230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00146f0 tx=0x7fe4b000c2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.735+0000 7fe4a3fff640 1 -- 172.21.15.137:0/3988114058 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4010e60 con 0x5566bbef5170 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.735+0000 7fe4a3fff640 1 -- 172.21.15.137:0/3988114058 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b409c630 con 0x5566bbef5170 2024-09-12T12:29:29.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.735+0000 7fe4a3fff640 1 -- 172.21.15.137:0/3988114058 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b409a230 con 0x5566bbef5170 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 -- 172.21.15.137:0/3988114058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 msgr2=0x5566bbeff4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbeff4f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4b409bec0 tx=0x7fe4b406efa0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 -- 172.21.15.137:0/3988114058 shutdown_connections 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbeffa30 0x5566bbf01e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbeff4f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3988114058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef4850 0x5566bbef4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 -- 172.21.15.137:0/3988114058 >> 172.21.15.137:0/3988114058 conn(0x5566bbef0de0 msgr2=0x5566bbef11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 -- 172.21.15.137:0/3988114058 shutdown_connections 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 -- 172.21.15.137:0/3988114058 wait complete. 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 -- start start 2024-09-12T12:29:29.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.736+0000 7fe4c2043740 1 Processor -- start 2024-09-12T12:29:29.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbefc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbefce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbeffa30 0x5566bbefd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4c2043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5566bbf02600 con 0x5566bbef4850 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4c2043740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5566bbde1900 con 0x5566bbef5170 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4c2043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5566bbf02780 con 0x5566bbeffa30 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbefc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4c2043740 10 client.0 _ll_register_callbacks cb 0x5566bbddf780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbefc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43820/0 (socket says 172.21.15.137:43820) 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bd205640 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbefce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 -- 172.21.15.137:0/1408633904 learned_addr learned my addr 172.21.15.137:0/1408633904 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4be207640 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbeffa30 0x5566bbefd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 -- 172.21.15.137:0/1408633904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbeffa30 msgr2=0x5566bbefd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbeffa30 0x5566bbefd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 -- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 msgr2=0x5566bbefce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbefce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.737+0000 7fe4bda06640 1 -- 172.21.15.137:0/1408633904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bbe21c50 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4bda06640 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbefc940 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe4b409bec0 tx=0x7fe4b401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2485331337a42540 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4a17fa640 1 -- 172.21.15.137:0/1408633904 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b409bf30 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4a17fa640 1 -- 172.21.15.137:0/1408633904 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fe4b40985a0 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b409c630 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4a17fa640 1 -- 172.21.15.137:0/1408633904 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe4b40a02e0 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 10 client.17379 fetch_fsmap learned FSMap version 48 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5566bbf0e2e0 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4a17fa640 1 -- 172.21.15.137:0/1408633904 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b40b7030 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4a17fa640 1 -- 172.21.15.137:0/1408633904 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fe4b409c190 con 0x5566bbef4850 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 10 client.17379 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 10 client.17379 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:29:29.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 20 client.17379 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbf0f630 0x5566bbf11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 4 client.17379 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.738+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566bbe66890 con 0x5566bbf0f630 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.739+0000 7fe4bd205640 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbf0f630 0x5566bbf11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.739+0000 7fe4bd205640 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbf0f630 0x5566bbf11a20 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06f7181074f9b0d server_cookie=fa83ded3e528b862 in_seq=0 out_seq=0 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.739+0000 7fe4a17fa640 10 client.17379 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.741+0000 7fe4a17fa640 1 -- 172.21.15.137:0/1408633904 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566bbe66890 con 0x5566bbf0f630 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.741+0000 7fe4a17fa640 10 client.17379 handle_command_reply: tid=0 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.741+0000 7fe4c2043740 1 client.17379 shutdown 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 msgr2=0x5566bbefc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbefc940 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe4b409bec0 tx=0x7fe4b401f000 comp rx=0 tx=0).stop 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 shutdown_connections 2024-09-12T12:29:29.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbeffa30 0x5566bbefd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbefce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbf0f630 0x5566bbf11a20 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1408633904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbefc940 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 >> 172.21.15.137:0/1408633904 conn(0x5566bbef0de0 msgr2=0x5566bbef6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4a17fa640 0 client.17379 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 shutdown_connections 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 1 -- 172.21.15.137:0/1408633904 wait complete. 2024-09-12T12:29:29.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.742+0000 7fe4c2043740 20 client.17379 trim_cache size 0 max 16384 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 Processor -- start 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 -- start start 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef0de0 0x5566bbf0b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbef9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef5170 0x5566bbefa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81babd0 con 0x5566bbef4850 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b83e0 con 0x5566bbef0de0 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4c2043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b6250 con 0x5566bbef5170 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4bd205640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbef9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4bd205640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbef9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43824/0 (socket says 172.21.15.137:43824) 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4bd205640 1 -- 172.21.15.137:0/1331335804 learned_addr learned my addr 172.21.15.137:0/1331335804 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:29.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.761+0000 7fe4bda06640 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef0de0 0x5566bbf0b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4be207640 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef5170 0x5566bbefa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4bd205640 1 -- 172.21.15.137:0/1331335804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef5170 msgr2=0x5566bbefa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4bd205640 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef5170 0x5566bbefa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4bd205640 1 -- 172.21.15.137:0/1331335804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef0de0 msgr2=0x5566bbf0b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4bd205640 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef0de0 0x5566bbf0b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4bd205640 1 -- 172.21.15.137:0/1331335804 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bbeff2b0 con 0x5566bbef4850 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4bd205640 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbef9ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe4a802a930 tx=0x7fe4a8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5919f85e0b9b472a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4a37fe640 1 -- 172.21.15.137:0/1331335804 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8035040 con 0x5566bbef4850 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4a37fe640 1 -- 172.21.15.137:0/1331335804 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a80047a0 con 0x5566bbef4850 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.762+0000 7fe4a37fe640 1 -- 172.21.15.137:0/1331335804 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80319e0 con 0x5566bbef4850 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 -- 172.21.15.137:0/1331335804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 msgr2=0x5566bbef9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbef9ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe4a802a930 tx=0x7fe4a8031400 comp rx=0 tx=0).stop 2024-09-12T12:29:29.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 -- 172.21.15.137:0/1331335804 shutdown_connections 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef5170 0x5566bbefa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbef9ba0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 --2- 172.21.15.137:0/1331335804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef0de0 0x5566bbf0b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 -- 172.21.15.137:0/1331335804 >> 172.21.15.137:0/1331335804 conn(0x5566bbefb580 msgr2=0x5566bbf0a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 -- 172.21.15.137:0/1331335804 shutdown_connections 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 -- 172.21.15.137:0/1331335804 wait complete. 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 -- start start 2024-09-12T12:29:29.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.763+0000 7fe4c2043740 1 Processor -- start 2024-09-12T12:29:29.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef0de0 0x5566bbf127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbf12d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4c2043740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbf78f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4c2043740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b68a0 con 0x5566bbef4850 2024-09-12T12:29:29.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4c2043740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b7d10 con 0x5566bbef5170 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4c2043740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81b6f70 con 0x5566bbef0de0 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4c2043740 10 client.0 _ll_register_callbacks cb 0x7fe4a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef0de0 0x5566bbf127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbf12d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4be207640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbf78f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bda06640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef0de0 0x5566bbf127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:44814/0 (socket says 172.21.15.137:44814) 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bda06640 1 -- 172.21.15.137:0/3859293926 learned_addr learned my addr 172.21.15.137:0/3859293926 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4be207640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbf78f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:45848/0 (socket says 172.21.15.137:45848) 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbf12d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43830/0 (socket says 172.21.15.137:43830) 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 -- 172.21.15.137:0/3859293926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef0de0 msgr2=0x5566bbf127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef0de0 0x5566bbf127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 -- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 msgr2=0x5566bbf78f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbf78f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 -- 172.21.15.137:0/3859293926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566bbf135e0 con 0x5566bbef4850 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4be207640 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbf78f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4bd205640 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbf12d00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x5566bbef0db0 tx=0x7fe4a80043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8e3ed76946d2d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.764+0000 7fe4a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4a17fa640 1 -- 172.21.15.137:0/3859293926 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8054060 con 0x5566bbef4850 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4a17fa640 1 -- 172.21.15.137:0/3859293926 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fe4a8031880 con 0x5566bbef4850 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a80047a0 con 0x5566bbef4850 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4a17fa640 1 -- 172.21.15.137:0/3859293926 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe4a8035030 con 0x5566bbef4850 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 10 client.17385 fetch_fsmap learned FSMap version 48 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5566bbf7a5a0 con 0x5566bbef4850 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4a17fa640 1 -- 172.21.15.137:0/3859293926 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a80047a0 con 0x5566bbef4850 2024-09-12T12:29:29.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4a17fa640 1 -- 172.21.15.137:0/3859293926 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fe4b4070ff0 con 0x5566bbef4850 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 10 client.17385 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 10 client.17385 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 20 client.17385 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbeffa30 0x5566bbf7cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 4 client.17385 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x5566bbe60be0 con 0x5566bbeffa30 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.765+0000 7fe4bda06640 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbeffa30 0x5566bbf7cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.766+0000 7fe4a17fa640 10 client.17385 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:29.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:29.766+0000 7fe4bda06640 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbeffa30 0x5566bbf7cd20 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d007eae22f7ef134 server_cookie=6e5d80f6c0072434 in_seq=0 out_seq=0 2024-09-12T12:29:30.222 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73299 36.20 MB/sec execute 62 sec latency 452.375 ms 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.629+0000 7fe4a17fa640 1 -- 172.21.15.137:0/3859293926 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fe4b40b7260 con 0x5566bbeffa30 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4a17fa640 10 client.17385 handle_command_reply: tid=0 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 client.17385 shutdown 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 msgr2=0x5566bbf12d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbf12d00 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x5566bbef0db0 tx=0x7fe4a80043d0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 shutdown_connections 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5566bbef5170 0x5566bbf78f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5566bbeffa30 0x5566bbf7cd20 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5566bbef4850 0x5566bbf12d00 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 --2- 172.21.15.137:0/3859293926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5566bbef0de0 0x5566bbf127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4a17fa640 0 client.17385 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 >> 172.21.15.137:0/3859293926 conn(0x5566bbefb580 msgr2=0x5566bbf0b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 shutdown_connections 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 1 -- 172.21.15.137:0/3859293926 wait complete. 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.630+0000 7fe4c2043740 20 client.17385 trim_cache size 0 max 16384 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 48, 2024-09-12T12:29:30.632 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 58, 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "5b1aeeed": { 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 58, 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.2, 2024-09-12T12:29:30.633 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:30.634 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:29:30.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4a40a2d70 msgr2=0x7fe4a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:30.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4a40a2d70 0x7fe4a40a5230 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00146f0 tx=0x7fe4b000c2d0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 msgr2=0x7fe4b81a2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:30.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b81a2f00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400ebc0 tx=0x7fe4b400c6a0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 shutdown_connections 2024-09-12T12:29:30.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe4a40a2d70 0x7fe4a40a5230 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b8112640 0x7fe4b81a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe4b810cf30 0x7fe4b81a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 --2- 172.21.15.137:0/433016534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4b810c590 0x7fe4b81a2f00 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:30.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 >> 172.21.15.137:0/433016534 conn(0x7fe4b806cdc0 msgr2=0x7fe4b810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:30.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 shutdown_connections 2024-09-12T12:29:30.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:30.631+0000 7fe4bfc6a640 1 -- 172.21.15.137:0/433016534 wait complete. 2024-09-12T12:29:30.902 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:30 smithi137 ceph-mon[30950]: pgmap v517: 97 pgs: 97 active+clean; 900 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-09-12T12:29:30.903 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:30 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:29:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:30 smithi189 ceph-mon[35475]: pgmap v517: 97 pgs: 97 active+clean; 900 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-09-12T12:29:31.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:30 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:29:31.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:30 smithi171 ceph-mon[35098]: pgmap v517: 97 pgs: 97 active+clean; 900 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 722 op/s 2024-09-12T12:29:31.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:30 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:29:31.222 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 35.63 MB/sec execute 63 sec latency 1452.464 ms 2024-09-12T12:29:31.933 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5b1aeeed', quiesce took 4.9 seconds. Will release after: 19.35 2024-09-12T12:29:32.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 35.08 MB/sec execute 64 sec latency 2452.577 ms 2024-09-12T12:29:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:32 smithi189 ceph-mon[35475]: pgmap v518: 97 pgs: 97 active+clean; 903 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 921 op/s 2024-09-12T12:29:33.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:32 smithi171 ceph-mon[35098]: pgmap v518: 97 pgs: 97 active+clean; 903 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 921 op/s 2024-09-12T12:29:33.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 34.54 MB/sec execute 65 sec latency 3452.684 ms 2024-09-12T12:29:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:32 smithi137 ceph-mon[30950]: pgmap v518: 97 pgs: 97 active+clean; 903 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 15 MiB/s wr, 921 op/s 2024-09-12T12:29:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:33 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:29:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:33 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:29:34.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:33 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:29:34.222 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 34.01 MB/sec execute 66 sec latency 4452.788 ms 2024-09-12T12:29:34.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:33 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:29:34.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:33 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:29:34.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:33 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:29:34.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:33 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:29:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:33 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:29:34.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:33 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:29:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:34 smithi171 ceph-mon[35098]: pgmap v519: 97 pgs: 97 active+clean; 903 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-09-12T12:29:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:34 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:29:35.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 33.51 MB/sec execute 67 sec latency 5452.910 ms 2024-09-12T12:29:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:34 smithi137 ceph-mon[30950]: pgmap v519: 97 pgs: 97 active+clean; 903 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-09-12T12:29:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:34 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:29:35.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:34 smithi189 ceph-mon[35475]: pgmap v519: 97 pgs: 97 active+clean; 903 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-09-12T12:29:35.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:34 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:29:36.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 33.01 MB/sec execute 68 sec latency 6452.991 ms 2024-09-12T12:29:37.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:36 smithi171 ceph-mon[35098]: pgmap v520: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 840 op/s 2024-09-12T12:29:37.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 32.54 MB/sec execute 69 sec latency 7453.100 ms 2024-09-12T12:29:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:36 smithi137 ceph-mon[30950]: pgmap v520: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 840 op/s 2024-09-12T12:29:37.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:36 smithi189 ceph-mon[35475]: pgmap v520: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 840 op/s 2024-09-12T12:29:38.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 32.07 MB/sec execute 70 sec latency 8453.235 ms 2024-09-12T12:29:39.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:38 smithi171 ceph-mon[35098]: pgmap v521: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-09-12T12:29:39.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 31.62 MB/sec execute 71 sec latency 9453.340 ms 2024-09-12T12:29:39.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:38 smithi137 ceph-mon[30950]: pgmap v521: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-09-12T12:29:39.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:38 smithi189 ceph-mon[35475]: pgmap v521: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.8 MiB/s wr, 537 op/s 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 -- 172.21.15.137:0/705844251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 msgr2=0x7f2d40107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/705844251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d40107f60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009930 tx=0x7f2d3002a420 comp rx=0 tx=0).stop 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 -- 172.21.15.137:0/705844251 shutdown_connections 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/705844251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d40102480 0x7f2d4010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/705844251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d40101ae0 0x7f2d40101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/705844251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d40107f60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 -- 172.21.15.137:0/705844251 >> 172.21.15.137:0/705844251 conn(0x7f2d400fd7e0 msgr2=0x7f2d400ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:39.505 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 -- 172.21.15.137:0/705844251 shutdown_connections 2024-09-12T12:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 -- 172.21.15.137:0/705844251 wait complete. 2024-09-12T12:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.504+0000 7f2d45b8d640 1 Processor -- start 2024-09-12T12:29:39.506 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d45b8d640 1 -- start start 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d45b8d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d40101ae0 0x7f2d40198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d45b8d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d40102480 0x7f2d40198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d45b8d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d4019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d45b8d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d40111f30 con 0x7f2d40107b80 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d45b8d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2d40111db0 con 0x7f2d40102480 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d45b8d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d401120b0 con 0x7f2d40101ae0 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d4019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d4019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51552/0 (socket says 172.21.15.137:51552) 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 -- 172.21.15.137:0/3069937271 learned_addr learned my addr 172.21.15.137:0/3069937271 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d40101ae0 0x7f2d40198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.508 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d40102480 0x7f2d40198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 -- 172.21.15.137:0/3069937271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d40101ae0 msgr2=0x7f2d40198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d40101ae0 0x7f2d40198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 -- 172.21.15.137:0/3069937271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d40102480 msgr2=0x7f2d40198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d40102480 0x7f2d40198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 -- 172.21.15.137:0/3069937271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4019d330 con 0x7f2d40107b80 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.505+0000 7f2d3ffff640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d4019cc00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800cce0 tx=0x7f2d3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.506+0000 7f2d3cff9640 1 -- 172.21.15.137:0/3069937271 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800f450 con 0x7f2d40107b80 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.506+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4019d620 con 0x7f2d40107b80 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.506+0000 7f2d3cff9640 1 -- 172.21.15.137:0/3069937271 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3800fad0 con 0x7f2d40107b80 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.506+0000 7f2d3cff9640 1 -- 172.21.15.137:0/3069937271 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38018c10 con 0x7f2d40107b80 2024-09-12T12:29:39.509 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.506+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d401a4eb0 con 0x7f2d40107b80 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.508+0000 7f2d3cff9640 1 -- 172.21.15.137:0/3069937271 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2d3800f5f0 con 0x7f2d40107b80 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.508+0000 7f2d48a02740 1 Processor -- start 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.508+0000 7f2d48a02740 1 -- start start 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d48a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd2b84e2f0 con 0x55fd2b8e9af0 2024-09-12T12:29:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d48a02740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fd2b7f6270 con 0x55fd2b8f4cb0 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d48a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd2b84a2b0 con 0x55fd2b8ea410 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3cff9640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2d280a2da0 0x7f2d280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8e9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51568/0 (socket says 172.21.15.137:51568) 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 -- 172.21.15.137:0/1643357955 learned_addr learned my addr 172.21.15.137:0/1643357955 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3ffff640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2d280a2da0 0x7f2d280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3cff9640 1 -- 172.21.15.137:0/3069937271 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f2d380cfab0 con 0x7f2d40107b80 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 -- 172.21.15.137:0/1643357955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8ea410 msgr2=0x55fd2b8f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.517 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 -- 172.21.15.137:0/1643357955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8f4cb0 msgr2=0x55fd2b8f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 -- 172.21.15.137:0/1643357955 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2b7f5340 con 0x55fd2b8e9af0 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3ffff640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2d280a2da0 0x7f2d280a5260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2d380cd2d0 tx=0x7f2d38091480 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.509+0000 7f2d3effd640 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8e9ed0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2d34014980 tx=0x7f2d3400b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229dbdcf39ba71a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d25ffb640 1 -- 172.21.15.137:0/1643357955 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34014e40 con 0x55fd2b8e9af0 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d25ffb640 1 -- 172.21.15.137:0/1643357955 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3400dbc0 con 0x55fd2b8e9af0 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d25ffb640 1 -- 172.21.15.137:0/1643357955 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34011a70 con 0x55fd2b8e9af0 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 -- 172.21.15.137:0/1643357955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e9af0 msgr2=0x55fd2b8e9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8e9ed0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2d34014980 tx=0x7f2d3400b470 comp rx=0 tx=0).stop 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 -- 172.21.15.137:0/1643357955 shutdown_connections 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.518 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 --2- 172.21.15.137:0/1643357955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8e9ed0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 -- 172.21.15.137:0/1643357955 >> 172.21.15.137:0/1643357955 conn(0x55fd2b8e6060 msgr2=0x55fd2b8e6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 -- 172.21.15.137:0/1643357955 shutdown_connections 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.510+0000 7f2d48a02740 1 -- 172.21.15.137:0/1643357955 wait complete. 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 -- start start 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 Processor -- start 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fd2b8f7880 con 0x55fd2b8ea410 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fd2b7d7290 con 0x55fd2b8e9af0 2024-09-12T12:29:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fd2b8f7a00 con 0x55fd2b8f4cb0 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d48a02740 10 client.0 _ll_register_callbacks cb 0x55fd2b808140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d3f7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51574/0 (socket says 172.21.15.137:51574) 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d3f7fe640 1 -- 172.21.15.137:0/3775034386 learned_addr learned my addr 172.21.15.137:0/3775034386 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.511+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d3f7fe640 1 -- 172.21.15.137:0/3775034386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8f4cb0 msgr2=0x55fd2b8f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d3f7fe640 1 -- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e9af0 msgr2=0x55fd2b8f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d3f7fe640 1 -- 172.21.15.137:0/3775034386 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd2b561210 con 0x55fd2b8ea410 2024-09-12T12:29:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f1ff0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2d30000c00 tx=0x7f2d300319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efc1052d1398f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d1effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d1effd640 1 -- 172.21.15.137:0/3775034386 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30035040 con 0x55fd2b8ea410 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d1effd640 1 -- 172.21.15.137:0/3775034386 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f2d30004300 con 0x55fd2b8ea410 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3400dbc0 con 0x55fd2b8ea410 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d1effd640 1 -- 172.21.15.137:0/3775034386 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2d30005020 con 0x55fd2b8ea410 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d48a02740 10 client.17397 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fd2b62d620 con 0x55fd2b8ea410 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.512+0000 7f2d1effd640 1 -- 172.21.15.137:0/3775034386 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3400dbc0 con 0x55fd2b8ea410 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d1effd640 1 -- 172.21.15.137:0/3775034386 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f2d3004d020 con 0x55fd2b8ea410 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d48a02740 10 client.17397 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d48a02740 10 client.17397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d48a02740 20 client.17397 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b904b40 0x55fd2b906f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d48a02740 4 client.17397 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd2b85ba20 con 0x55fd2b904b40 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b904b40 0x55fd2b906f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d1effd640 10 client.17397 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.513+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b904b40 0x55fd2b906f70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2382b289a5249ff8 server_cookie=54d1e82fee821f5 in_seq=0 out_seq=0 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d1effd640 1 -- 172.21.15.137:0/3775034386 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd2b85ba20 con 0x55fd2b904b40 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d1effd640 10 client.17397 handle_command_reply: tid=0 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 client.17397 shutdown 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 msgr2=0x55fd2b8f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f1ff0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2d30000c00 tx=0x7f2d300319b0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 shutdown_connections 2024-09-12T12:29:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8f4cb0 0x55fd2b8f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b904b40 0x55fd2b906f70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8f1ff0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3775034386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d1effd640 0 client.17397 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 >> 172.21.15.137:0/3775034386 conn(0x55fd2b8e6060 msgr2=0x55fd2b8ebd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 shutdown_connections 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.514+0000 7f2d48a02740 1 -- 172.21.15.137:0/3775034386 wait complete. 2024-09-12T12:29:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.515+0000 7f2d48a02740 20 client.17397 trim_cache size 0 max 16384 2024-09-12T12:29:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 Processor -- start 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 -- start start 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e6060 0x55fd2b8ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8f2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8ea410 0x55fd2b90c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d401b0070 con 0x55fd2b8e6060 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2d401ad880 con 0x55fd2b8ea410 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d48a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d401ab6f0 con 0x55fd2b8e9af0 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e6060 0x55fd2b8ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e6060 0x55fd2b8ec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51588/0 (socket says 172.21.15.137:51588) 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 -- 172.21.15.137:0/787831767 learned_addr learned my addr 172.21.15.137:0/787831767 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 -- 172.21.15.137:0/787831767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 msgr2=0x55fd2b8f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 --2- 172.21.15.137:0/787831767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 -- 172.21.15.137:0/787831767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8ea410 msgr2=0x55fd2b90c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 --2- 172.21.15.137:0/787831767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8ea410 0x55fd2b90c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.529+0000 7f2d3effd640 1 -- 172.21.15.137:0/787831767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd2b8e5e90 con 0x55fd2b8e6060 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d3effd640 1 --2- 172.21.15.137:0/787831767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e6060 0x55fd2b8ec560 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400b440 tx=0x7f2d34005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f3de722156640a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d257fa640 1 -- 172.21.15.137:0/787831767 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400de50 con 0x55fd2b8e6060 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d257fa640 1 -- 172.21.15.137:0/787831767 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d34005040 con 0x55fd2b8e6060 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d257fa640 1 -- 172.21.15.137:0/787831767 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3400d8a0 con 0x55fd2b8e6060 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 -- 172.21.15.137:0/787831767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e6060 msgr2=0x55fd2b8ec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 --2- 172.21.15.137:0/787831767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e6060 0x55fd2b8ec560 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400b440 tx=0x7f2d34005f30 comp rx=0 tx=0).stop 2024-09-12T12:29:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 -- 172.21.15.137:0/787831767 shutdown_connections 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 --2- 172.21.15.137:0/787831767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8ea410 0x55fd2b90c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 --2- 172.21.15.137:0/787831767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 --2- 172.21.15.137:0/787831767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8e6060 0x55fd2b8ec560 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 -- 172.21.15.137:0/787831767 >> 172.21.15.137:0/787831767 conn(0x55fd2b8e6ae0 msgr2=0x55fd2b8f3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 -- 172.21.15.137:0/787831767 shutdown_connections 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.530+0000 7f2d48a02740 1 -- 172.21.15.137:0/787831767 wait complete. 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 -- start start 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 Processor -- start 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e6060 0x55fd2b8fd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d401abd40 con 0x55fd2b8ea410 2024-09-12T12:29:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2d401ad1b0 con 0x55fd2b8e6060 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2d401ac410 con 0x55fd2b8e9af0 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d48a02740 10 client.0 _ll_register_callbacks cb 0x7f2d30009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8eef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51596/0 (socket says 172.21.15.137:51596) 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 -- 172.21.15.137:0/3400463327 learned_addr learned my addr 172.21.15.137:0/3400463327 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e6060 0x55fd2b8fd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 -- 172.21.15.137:0/3400463327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 msgr2=0x55fd2b8ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 -- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e6060 msgr2=0x55fd2b8fd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e6060 0x55fd2b8fd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.531+0000 7f2d3effd640 1 -- 172.21.15.137:0/3400463327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd2b8e7110 con 0x55fd2b8ea410 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d3effd640 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8eef40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2d34014950 tx=0x7f2d34005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c66298efd0890e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d1effd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d1effd640 1 -- 172.21.15.137:0/3400463327 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34032020 con 0x55fd2b8ea410 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d1effd640 1 -- 172.21.15.137:0/3400463327 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f2d340053f0 con 0x55fd2b8ea410 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d34005040 con 0x55fd2b8ea410 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d1effd640 1 -- 172.21.15.137:0/3400463327 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2d340055f0 con 0x55fd2b8ea410 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 10 client.17403 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55fd2b96f760 con 0x55fd2b8ea410 2024-09-12T12:29:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d1effd640 1 -- 172.21.15.137:0/3400463327 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d34005040 con 0x55fd2b8ea410 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d1effd640 1 -- 172.21.15.137:0/3400463327 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f2d34004b30 con 0x55fd2b8ea410 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 10 client.17403 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 20 client.17403 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b8f4cb0 0x55fd2b971e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 4 client.17403 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.532+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd2b855d70 con 0x55fd2b8f4cb0 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.533+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b8f4cb0 0x55fd2b971e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.533+0000 7f2d3f7fe640 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b8f4cb0 0x55fd2b971e00 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9883b103f9a89f24 server_cookie=f2e129bced7592e7 in_seq=0 out_seq=0 2024-09-12T12:29:39.540 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.533+0000 7f2d1effd640 10 client.17403 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.664+0000 7f2d1effd640 1 -- 172.21.15.137:0/3400463327 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fd2b855d70 con 0x55fd2b8f4cb0 2024-09-12T12:29:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.664+0000 7f2d1effd640 10 client.17403 handle_command_reply: tid=0 2024-09-12T12:29:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.664+0000 7f2d48a02740 1 client.17403 shutdown 2024-09-12T12:29:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.664+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 msgr2=0x55fd2b8eef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.664+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8eef40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2d34014950 tx=0x7f2d34005fa0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 shutdown_connections 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fd2b8f4cb0 0x55fd2b971e00 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fd2b8ea410 0x55fd2b8eef40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd2b8e9af0 0x55fd2b8ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d1effd640 0 client.17403 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 --2- 172.21.15.137:0/3400463327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fd2b8e6060 0x55fd2b8fd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 >> 172.21.15.137:0/3400463327 conn(0x55fd2b8e6ae0 msgr2=0x55fd2b8e49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 shutdown_connections 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 1 -- 172.21.15.137:0/3400463327 wait complete. 2024-09-12T12:29:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d48a02740 20 client.17403 trim_cache size 0 max 16384 2024-09-12T12:29:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:29:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:29:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:29:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2d280a2da0 msgr2=0x7f2d280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2d280a2da0 0x7f2d280a5260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2d380cd2d0 tx=0x7f2d38091480 comp rx=0 tx=0).stop 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 msgr2=0x7f2d4019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d4019cc00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800cce0 tx=0x7f2d3800a620 comp rx=0 tx=0).stop 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.665+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 shutdown_connections 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.666+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f2d280a2da0 0x7f2d280a5260 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.666+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d40107b80 0x7f2d4019cc00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.666+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d40102480 0x7f2d40198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.666+0000 7f2d45b8d640 1 --2- 172.21.15.137:0/3069937271 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d40101ae0 0x7f2d40198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.666+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 >> 172.21.15.137:0/3069937271 conn(0x7f2d400fd7e0 msgr2=0x7f2d400fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.666+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 shutdown_connections 2024-09-12T12:29:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:39.666+0000 7f2d45b8d640 1 -- 172.21.15.137:0/3069937271 wait complete. 2024-09-12T12:29:40.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 31.18 MB/sec execute 72 sec latency 10453.393 ms 2024-09-12T12:29:41.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:40 smithi171 ceph-mon[35098]: pgmap v522: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.2 MiB/s wr, 553 op/s 2024-09-12T12:29:41.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 30.75 MB/sec execute 73 sec latency 11453.489 ms 2024-09-12T12:29:41.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:40 smithi137 ceph-mon[30950]: pgmap v522: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.2 MiB/s wr, 553 op/s 2024-09-12T12:29:41.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:40 smithi189 ceph-mon[35475]: pgmap v522: 97 pgs: 97 active+clean; 897 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.2 MiB/s wr, 553 op/s 2024-09-12T12:29:42.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 30.34 MB/sec execute 74 sec latency 12453.591 ms 2024-09-12T12:29:43.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:42 smithi171 ceph-mon[35098]: pgmap v523: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 517 op/s 2024-09-12T12:29:43.223 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 29.93 MB/sec execute 75 sec latency 13453.638 ms 2024-09-12T12:29:43.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:42 smithi137 ceph-mon[30950]: pgmap v523: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 517 op/s 2024-09-12T12:29:43.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:42 smithi189 ceph-mon[35475]: pgmap v523: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.6 MiB/s wr, 517 op/s 2024-09-12T12:29:44.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 29.54 MB/sec execute 76 sec latency 14453.693 ms 2024-09-12T12:29:45.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:44 smithi171 ceph-mon[35098]: pgmap v524: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 214 op/s 2024-09-12T12:29:45.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 29.15 MB/sec execute 77 sec latency 15453.730 ms 2024-09-12T12:29:45.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:44 smithi137 ceph-mon[30950]: pgmap v524: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 214 op/s 2024-09-12T12:29:45.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:44 smithi189 ceph-mon[35475]: pgmap v524: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 214 op/s 2024-09-12T12:29:46.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 28.78 MB/sec execute 78 sec latency 16453.850 ms 2024-09-12T12:29:47.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:46 smithi171 ceph-mon[35098]: pgmap v525: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 324 op/s 2024-09-12T12:29:47.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 28.42 MB/sec execute 79 sec latency 17453.952 ms 2024-09-12T12:29:47.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:46 smithi137 ceph-mon[30950]: pgmap v525: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 324 op/s 2024-09-12T12:29:47.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:46 smithi189 ceph-mon[35475]: pgmap v525: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 324 op/s 2024-09-12T12:29:48.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 28.06 MB/sec execute 80 sec latency 18454.025 ms 2024-09-12T12:29:49.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:48 smithi171 ceph-mon[35098]: pgmap v526: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 4.3 MiB/s wr, 207 op/s 2024-09-12T12:29:49.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 27.71 MB/sec execute 81 sec latency 19454.126 ms 2024-09-12T12:29:49.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:48 smithi137 ceph-mon[30950]: pgmap v526: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 4.3 MiB/s wr, 207 op/s 2024-09-12T12:29:49.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:48 smithi189 ceph-mon[35475]: pgmap v526: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 4.3 MiB/s wr, 207 op/s 2024-09-12T12:29:50.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 27.38 MB/sec execute 82 sec latency 20454.243 ms 2024-09-12T12:29:51.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:50 smithi171 ceph-mon[35098]: pgmap v527: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.9 MiB/s wr, 234 op/s 2024-09-12T12:29:51.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 27.05 MB/sec execute 83 sec latency 21454.327 ms 2024-09-12T12:29:51.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:29:51.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:50 smithi137 ceph-mon[30950]: pgmap v527: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.9 MiB/s wr, 234 op/s 2024-09-12T12:29:51.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:50 smithi189 ceph-mon[35475]: pgmap v527: 97 pgs: 97 active+clean; 888 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 4.9 MiB/s wr, 234 op/s 2024-09-12T12:29:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:29:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 -- 172.21.15.137:0/907280274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 msgr2=0x7f47b810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 --2- 172.21.15.137:0/907280274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b810ec70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009cd0 tx=0x7f47b002a8d0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 -- 172.21.15.137:0/907280274 shutdown_connections 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 --2- 172.21.15.137:0/907280274 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b810ec70 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 --2- 172.21.15.137:0/907280274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b8103780 0x7f47b8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 --2- 172.21.15.137:0/907280274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b8102dd0 0x7f47b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 -- 172.21.15.137:0/907280274 >> 172.21.15.137:0/907280274 conn(0x7f47b80fc7d0 msgr2=0x7f47b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 -- 172.21.15.137:0/907280274 shutdown_connections 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.066+0000 7f47bdd74640 1 -- 172.21.15.137:0/907280274 wait complete. 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 Processor -- start 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 -- start start 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b8102dd0 0x7f47b81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b8103780 0x7f47b8198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8112110 con 0x7f47b8108300 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8111f90 con 0x7f47b8102dd0 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47bdd74640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f47b8112290 con 0x7f47b8103780 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47b7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38644/0 (socket says 172.21.15.137:38644) 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.067+0000 7f47b7fff640 1 -- 172.21.15.137:0/4153445641 learned_addr learned my addr 172.21.15.137:0/4153445641 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b6ffd640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b8103780 0x7f47b8198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b77fe640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b8102dd0 0x7f47b81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b7fff640 1 -- 172.21.15.137:0/4153445641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b8103780 msgr2=0x7f47b8198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b7fff640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b8103780 0x7f47b8198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b7fff640 1 -- 172.21.15.137:0/4153445641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b8102dd0 msgr2=0x7f47b81985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b7fff640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b8102dd0 0x7f47b81985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b7fff640 1 -- 172.21.15.137:0/4153445641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b819d580 con 0x7f47b8108300 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b77fe640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b8102dd0 0x7f47b81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b7fff640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b819cea0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009ca0 tx=0x7f47b002f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b4ff9640 1 -- 172.21.15.137:0/4153445641 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b0031860 con 0x7f47b8108300 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b4ff9640 1 -- 172.21.15.137:0/4153445641 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b0003e50 con 0x7f47b8108300 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b819d810 con 0x7f47b8108300 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47b4ff9640 1 -- 172.21.15.137:0/4153445641 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b00307f0 con 0x7f47b8108300 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.068+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b8100960 con 0x7f47b8108300 2024-09-12T12:29:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.069+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b806a7b0 con 0x7f47b8108300 2024-09-12T12:29:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.071+0000 7f47b4ff9640 1 -- 172.21.15.137:0/4153445641 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f47b00041d0 con 0x7f47b8108300 2024-09-12T12:29:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.072+0000 7f47b4ff9640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f47a00a2cf0 0x7f47a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.072+0000 7f47b4ff9640 1 -- 172.21.15.137:0/4153445641 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f47b00f0490 con 0x7f47b8108300 2024-09-12T12:29:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.072+0000 7f47b77fe640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f47a00a2cf0 0x7f47a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.072+0000 7f47b77fe640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f47a00a2cf0 0x7f47a00a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47a8006fd0 tx=0x7f47a8008040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.075+0000 7f47b4ff9640 1 -- 172.21.15.137:0/4153445641 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47b00bd1e0 con 0x7f47b8108300 2024-09-12T12:29:52.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 26.73 MB/sec execute 84 sec latency 22454.356 ms 2024-09-12T12:29:52.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.274+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47b81031b0 con 0x7f47b8108300 2024-09-12T12:29:52.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.275+0000 7f47b4ff9640 1 -- 172.21.15.137:0/4153445641 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11942 (secure 0 0 0) 0x7f47b81031b0 con 0x7f47b8108300 2024-09-12T12:29:52.276 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:29:52.276 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":49,"btime":"2024-09-12T12:29:33:838508+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:29:32.898454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26647":{"gid":26647,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3747343886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3747343886},{"type":"v1","addr":"172.21.15.189:6835","nonce":3747343886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:29:52.276 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 49 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f47a00a2cf0 msgr2=0x7f47a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f47a00a2cf0 0x7f47a00a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47a8006fd0 tx=0x7f47a8008040 comp rx=0 tx=0).stop 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 msgr2=0x7f47b819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b819cea0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009ca0 tx=0x7f47b002f420 comp rx=0 tx=0).stop 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 shutdown_connections 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f47a00a2cf0 0x7f47a00a51b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f47b8108300 0x7f47b819cea0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47b8103780 0x7f47b8198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 --2- 172.21.15.137:0/4153445641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47b8102dd0 0x7f47b81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 >> 172.21.15.137:0/4153445641 conn(0x7f47b80fc7d0 msgr2=0x7f47b8105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 shutdown_connections 2024-09-12T12:29:52.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:52.277+0000 7f47bdd74640 1 -- 172.21.15.137:0/4153445641 wait complete. 2024-09-12T12:29:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 -- 172.21.15.137:0/828408990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9024102050 msgr2=0x7f902410d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 --2- 172.21.15.137:0/828408990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9024102050 0x7f902410d7d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f901c009ca0 tx=0x7f901c02a950 comp rx=0 tx=0).stop 2024-09-12T12:29:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 -- 172.21.15.137:0/828408990 shutdown_connections 2024-09-12T12:29:52.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 --2- 172.21.15.137:0/828408990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9024102050 0x7f902410d7d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 --2- 172.21.15.137:0/828408990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90241016d0 0x7f9024101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 --2- 172.21.15.137:0/828408990 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9024105af0 0x7f9024105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 -- 172.21.15.137:0/828408990 >> 172.21.15.137:0/828408990 conn(0x7f9024075ec0 msgr2=0x7f90240762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 -- 172.21.15.137:0/828408990 shutdown_connections 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 -- 172.21.15.137:0/828408990 wait complete. 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 Processor -- start 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.757+0000 7f902a853640 1 -- start start 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90241016d0 0x7f9024102890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024102050 0x7f9024102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9024105af0 0x7f9024103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110de0 con 0x7f90241016d0 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110c60 con 0x7f9024102050 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9024110f60 con 0x7f9024105af0 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90241016d0 0x7f9024102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90241016d0 0x7f9024102890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38676/0 (socket says 172.21.15.137:38676) 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 -- 172.21.15.137:0/2715665150 learned_addr learned my addr 172.21.15.137:0/2715665150 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9028df0640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9024105af0 0x7f9024103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f90237fe640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024102050 0x7f9024102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 -- 172.21.15.137:0/2715665150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9024105af0 msgr2=0x7f9024103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9024105af0 0x7f9024103310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 -- 172.21.15.137:0/2715665150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024102050 msgr2=0x7f9024102dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024102050 0x7f9024102dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 -- 172.21.15.137:0/2715665150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90241a6780 con 0x7f90241016d0 2024-09-12T12:29:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f9023fff640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90241016d0 0x7f9024102890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f901400d770 tx=0x7f901400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f90217fa640 1 -- 172.21.15.137:0/2715665150 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014017070 con 0x7f90241016d0 2024-09-12T12:29:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f90217fa640 1 -- 172.21.15.137:0/2715665150 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9014005500 con 0x7f90241016d0 2024-09-12T12:29:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90241a6a70 con 0x7f90241016d0 2024-09-12T12:29:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f90217fa640 1 -- 172.21.15.137:0/2715665150 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014004e00 con 0x7f90241016d0 2024-09-12T12:29:52.762 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.758+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90241a6fc0 con 0x7f90241016d0 2024-09-12T12:29:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.760+0000 7f90217fa640 1 -- 172.21.15.137:0/2715665150 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9014004fa0 con 0x7f90241016d0 2024-09-12T12:29:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 Processor -- start 2024-09-12T12:29:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 -- start start 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829731ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829732410 0x55c82973c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c82973ccb0 0x55c82973f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f90217fa640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f900c0a2e20 0x7f900c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c8296962f0 con 0x55c82973ccb0 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c82963e270 con 0x55c829732410 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f902d5eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c8296922b0 con 0x55c829731af0 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f9028df0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c82973ccb0 0x55c82973f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f90237fe640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f900c0a2e20 0x7f900c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f9028df0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c82973ccb0 0x55c82973f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38678/0 (socket says 172.21.15.137:38678) 2024-09-12T12:29:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f9028df0640 1 -- 172.21.15.137:0/2191515994 learned_addr learned my addr 172.21.15.137:0/2191515994 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f90217fa640 1 -- 172.21.15.137:0/2715665150 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f90140cee30 con 0x7f90241016d0 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f9028df0640 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829731ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.761+0000 7f9023fff640 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829732410 0x55c82973c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f9028df0640 1 -- 172.21.15.137:0/2191515994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 msgr2=0x55c829731ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f9028df0640 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829731ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f9028df0640 1 -- 172.21.15.137:0/2191515994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829732410 msgr2=0x55c82973c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f9028df0640 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829732410 0x55c82973c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f9028df0640 1 -- 172.21.15.137:0/2191515994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82963d340 con 0x55c82973ccb0 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f90237fe640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f900c0a2e20 0x7f900c0a52e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9024104320 tx=0x7f901800b590 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f9028df0640 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c82973ccb0 0x55c82973f0a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f901c009430 tx=0x7f901c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cb4a9dc9f42bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f900a7fc640 1 -- 172.21.15.137:0/2191515994 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c034040 con 0x55c82973ccb0 2024-09-12T12:29:52.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f900a7fc640 1 -- 172.21.15.137:0/2191515994 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901c003970 con 0x55c82973ccb0 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.762+0000 7f900a7fc640 1 -- 172.21.15.137:0/2191515994 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901c004310 con 0x55c82973ccb0 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 -- 172.21.15.137:0/2191515994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c82973ccb0 msgr2=0x55c82973f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c82973ccb0 0x55c82973f0a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f901c009430 tx=0x7f901c009360 comp rx=0 tx=0).stop 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 -- 172.21.15.137:0/2191515994 shutdown_connections 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c82973ccb0 0x55c82973f0a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829732410 0x55c82973c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 --2- 172.21.15.137:0/2191515994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829731ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 -- 172.21.15.137:0/2191515994 >> 172.21.15.137:0/2191515994 conn(0x55c82972e060 msgr2=0x55c82972e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 -- 172.21.15.137:0/2191515994 shutdown_connections 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 -- 172.21.15.137:0/2191515994 wait complete. 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 -- start start 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.763+0000 7f902d5eb740 1 Processor -- start 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c829739ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c82973a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55c82973f880 con 0x55c829732410 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c82961f290 con 0x55c829731af0 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c82973fa00 con 0x55c82973ccb0 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 10 client.0 _ll_register_callbacks cb 0x55c829650140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38684/0 (socket says 172.21.15.137:38684) 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 -- 172.21.15.137:0/1723204334 learned_addr learned my addr 172.21.15.137:0/1723204334 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f9023fff640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c829739ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f9028df0640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c82973a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 -- 172.21.15.137:0/1723204334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 msgr2=0x55c82973a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c82973a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 -- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 msgr2=0x55c829739ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c829739ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 -- 172.21.15.137:0/1723204334 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8293a9210 con 0x55c829732410 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f9028df0640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c82973a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90237fe640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a0e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f901800ffc0 tx=0x7f90180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4c7ea2b0ed79a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90037fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90037fe640 1 -- 172.21.15.137:0/1723204334 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018022070 con 0x55c829732410 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90037fe640 1 -- 172.21.15.137:0/1723204334 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f901801e860 con 0x55c829732410 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f901c003970 con 0x55c829732410 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f90037fe640 1 -- 172.21.15.137:0/1723204334 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f901802e050 con 0x55c829732410 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 10 client.17421 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:52.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.764+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c829475620 con 0x55c829732410 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f90037fe640 1 -- 172.21.15.137:0/1723204334 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f901c003970 con 0x55c829732410 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f90037fe640 1 -- 172.21.15.137:0/1723204334 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f9018010b40 con 0x55c829732410 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f902d5eb740 10 client.17421 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f902d5eb740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f902d5eb740 20 client.17421 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c82974ccd0 0x55c82974f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f902d5eb740 4 client.17421 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:29:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8296a3a20 con 0x55c82974ccd0 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f9023fff640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c82974ccd0 0x55c82974f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f9023fff640 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c82974ccd0 0x55c82974f130 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7315e58b78cc7b1e server_cookie=aa79f910b1b82c53 in_seq=0 out_seq=0 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.765+0000 7f90037fe640 10 client.17421 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f90037fe640 1 -- 172.21.15.137:0/1723204334 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8296a3a20 con 0x55c82974ccd0 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f90037fe640 10 client.17421 handle_command_reply: tid=0 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 client.17421 shutdown 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 msgr2=0x55c82973a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a0e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f901800ffc0 tx=0x7f90180046c0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 shutdown_connections 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c82973a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c82974ccd0 0x55c82974f130 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a0e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1723204334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c829739ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f90037fe640 0 client.17421 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 >> 172.21.15.137:0/1723204334 conn(0x55c82972e060 msgr2=0x55c829733d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 shutdown_connections 2024-09-12T12:29:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 1 -- 172.21.15.137:0/1723204334 wait complete. 2024-09-12T12:29:52.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.766+0000 7f902d5eb740 20 client.17421 trim_cache size 0 max 16384 2024-09-12T12:29:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.780+0000 7f902d5eb740 1 Processor -- start 2024-09-12T12:29:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.780+0000 7f902d5eb740 1 -- start start 2024-09-12T12:29:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829738450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c82973ccb0 0x55c829754a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f902d5eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90241b2250 con 0x55c829732410 2024-09-12T12:29:52.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f902d5eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90241afa60 con 0x55c82973ccb0 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f902d5eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90241ad8d0 con 0x55c829731af0 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38696/0 (socket says 172.21.15.137:38696) 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 -- 172.21.15.137:0/3052590333 learned_addr learned my addr 172.21.15.137:0/3052590333 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f9023fff640 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829738450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 -- 172.21.15.137:0/3052590333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 msgr2=0x55c829738450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829738450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 -- 172.21.15.137:0/3052590333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c82973ccb0 msgr2=0x55c829754a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c82973ccb0 0x55c829754a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 -- 172.21.15.137:0/3052590333 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c829732240 con 0x55c829732410 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f9023fff640 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829738450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f90237fe640 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9018026450 tx=0x7f9018011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e072b42f135fdaf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f9009ffb640 1 -- 172.21.15.137:0/3052590333 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018036030 con 0x55c829732410 2024-09-12T12:29:52.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f9009ffb640 1 -- 172.21.15.137:0/3052590333 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9018026e30 con 0x55c829732410 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.781+0000 7f9009ffb640 1 -- 172.21.15.137:0/3052590333 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018022050 con 0x55c829732410 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 -- 172.21.15.137:0/3052590333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 msgr2=0x55c82973a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a450 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9018026450 tx=0x7f9018011ca0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 -- 172.21.15.137:0/3052590333 shutdown_connections 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c82973ccb0 0x55c829754a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c82973a450 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 --2- 172.21.15.137:0/3052590333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c829731af0 0x55c829738450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 -- 172.21.15.137:0/3052590333 >> 172.21.15.137:0/3052590333 conn(0x55c82972e060 msgr2=0x55c82973bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 -- 172.21.15.137:0/3052590333 shutdown_connections 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 -- 172.21.15.137:0/3052590333 wait complete. 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 -- start start 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.782+0000 7f902d5eb740 1 Processor -- start 2024-09-12T12:29:52.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c8297451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c8297404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c829740a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f90241adf20 con 0x55c829732410 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90241af390 con 0x55c829731af0 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90241ae5f0 con 0x55c82973ccb0 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 10 client.0 _ll_register_callbacks cb 0x7f9018015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c8297404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c8297404d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38706/0 (socket says 172.21.15.137:38706) 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 -- 172.21.15.137:0/1684164545 learned_addr learned my addr 172.21.15.137:0/1684164545 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 -- 172.21.15.137:0/1684164545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 msgr2=0x55c829740a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f9028df0640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c829740a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c829740a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 -- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 msgr2=0x55c8297451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f9023fff640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c8297451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c8297451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 -- 172.21.15.137:0/1684164545 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c829740fd0 con 0x55c829732410 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f9028df0640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c829740a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f9023fff640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c8297451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90237fe640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c8297404d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f901800f8e0 tx=0x7f9018026780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af719a6cd8df2ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90037fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90037fe640 1 -- 172.21.15.137:0/1684164545 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901803f020 con 0x55c829732410 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f90037fe640 1 -- 172.21.15.137:0/1684164545 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f9018005120 con 0x55c829732410 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.783+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9018026e30 con 0x55c829732410 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f90037fe640 1 -- 172.21.15.137:0/1684164545 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f901804a050 con 0x55c829732410 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 10 client.17427 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55c8297b7720 con 0x55c829732410 2024-09-12T12:29:52.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f90037fe640 1 -- 172.21.15.137:0/1684164545 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9018026e30 con 0x55c829732410 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f90037fe640 1 -- 172.21.15.137:0/1684164545 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7f90180046c0 con 0x55c829732410 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 10 client.17427 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 10 client.17427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 20 client.17427 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c8297b7de0 0x55c8297ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 4 client.17427 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c82969dd70 con 0x55c8297b7de0 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.784+0000 7f9023fff640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c8297b7de0 0x55c8297ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.785+0000 7f90037fe640 10 client.17427 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.785+0000 7f9023fff640 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c8297b7de0 0x55c8297ba200 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3428e3a6007d17a server_cookie=1f7f26faabb41784 in_seq=0 out_seq=0 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.863+0000 7f90037fe640 1 -- 172.21.15.137:0/1684164545 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c82969dd70 con 0x55c8297b7de0 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.863+0000 7f90037fe640 10 client.17427 handle_command_reply: tid=0 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.863+0000 7f902d5eb740 1 client.17427 shutdown 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 msgr2=0x55c8297404d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c8297404d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f901800f8e0 tx=0x7f9018026780 comp rx=0 tx=0).stop 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 shutdown_connections 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c82973ccb0 0x55c829740a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55c8297b7de0 0x55c8297ba200 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c829732410 0x55c8297404d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f90037fe640 0 client.17427 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 --2- 172.21.15.137:0/1684164545 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c829731af0 0x55c8297451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 >> 172.21.15.137:0/1684164545 conn(0x55c82972e060 msgr2=0x55c82972c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 shutdown_connections 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 1 -- 172.21.15.137:0/1684164545 wait complete. 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902d5eb740 20 client.17427 trim_cache size 0 max 16384 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:29:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:29:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:29:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f900c0a2e20 msgr2=0x7f900c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902a853640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f900c0a2e20 0x7f900c0a52e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9024104320 tx=0x7f901800b590 comp rx=0 tx=0).stop 2024-09-12T12:29:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90241016d0 msgr2=0x7f9024102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.864+0000 7f902a853640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90241016d0 0x7f9024102890 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f901400d770 tx=0x7f901400dc30 comp rx=0 tx=0).stop 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 shutdown_connections 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f900c0a2e20 0x7f900c0a52e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9024105af0 0x7f9024103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024102050 0x7f9024102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 --2- 172.21.15.137:0/2715665150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90241016d0 0x7f9024102890 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 >> 172.21.15.137:0/2715665150 conn(0x7f9024075ec0 msgr2=0x7f90240ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 shutdown_connections 2024-09-12T12:29:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:29:52.865+0000 7f902a853640 1 -- 172.21.15.137:0/2715665150 wait complete. 2024-09-12T12:29:52.975 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:52 smithi137 ceph-mon[30950]: pgmap v528: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 6.6 MiB/s wr, 287 op/s 2024-09-12T12:29:52.975 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:52 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4153445641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:53.028 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 5b1aeeed --release --await' 2024-09-12T12:29:53.029 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 5b1aeeed --release --await 2024-09-12T12:29:53.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:52 smithi171 ceph-mon[35098]: pgmap v528: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 6.6 MiB/s wr, 287 op/s 2024-09-12T12:29:53.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:52 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4153445641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:53.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73315 26.41 MB/sec execute 85 sec latency 23454.412 ms 2024-09-12T12:29:53.317 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:29:53.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:52 smithi189 ceph-mon[35475]: pgmap v528: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 6.6 MiB/s wr, 287 op/s 2024-09-12T12:29:53.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:52 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4153445641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 -- 172.21.15.137:0/1844467100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 msgr2=0x7efe541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 --2- 172.21.15.137:0/1844467100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe541021c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7efe50009900 tx=0x7efe5002a410 comp rx=0 tx=0).stop 2024-09-12T12:29:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 -- 172.21.15.137:0/1844467100 shutdown_connections 2024-09-12T12:29:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 --2- 172.21.15.137:0/1844467100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe54106f10 0x7efe5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 --2- 172.21.15.137:0/1844467100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54102790 0x7efe541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 --2- 172.21.15.137:0/1844467100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe541021c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 -- 172.21.15.137:0/1844467100 >> 172.21.15.137:0/1844467100 conn(0x7efe540fd8c0 msgr2=0x7efe540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 -- 172.21.15.137:0/1844467100 shutdown_connections 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.840+0000 7efe5be14640 1 -- 172.21.15.137:0/1844467100 wait complete. 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 Processor -- start 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 -- start start 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe54198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54102790 0x7efe54198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe54106f10 0x7efe5419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111f70 con 0x7efe54101de0 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111df0 con 0x7efe54102790 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5be14640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe541120f0 con 0x7efe54106f10 2024-09-12T12:29:53.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe54198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe54198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38730/0 (socket says 172.21.15.137:38730) 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 -- 172.21.15.137:0/1230917598 learned_addr learned my addr 172.21.15.137:0/1230917598 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe593af640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54102790 0x7efe54198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe54106f10 0x7efe5419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 -- 172.21.15.137:0/1230917598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe54106f10 msgr2=0x7efe5419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe54106f10 0x7efe5419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 -- 172.21.15.137:0/1230917598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54102790 msgr2=0x7efe54198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54102790 0x7efe54198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.841+0000 7efe59bb0640 1 -- 172.21.15.137:0/1230917598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5419d400 con 0x7efe54101de0 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.842+0000 7efe59bb0640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe54198450 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efe50006010 tx=0x7efe500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.842+0000 7efe4affd640 1 -- 172.21.15.137:0/1230917598 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5003f030 con 0x7efe54101de0 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.842+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5419d690 con 0x7efe54101de0 2024-09-12T12:29:53.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.842+0000 7efe4affd640 1 -- 172.21.15.137:0/1230917598 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe50003ce0 con 0x7efe54101de0 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.842+0000 7efe4affd640 1 -- 172.21.15.137:0/1230917598 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50035070 con 0x7efe54101de0 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.842+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a4fd0 con 0x7efe54101de0 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.844+0000 7efe4affd640 1 -- 172.21.15.137:0/1230917598 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efe50042030 con 0x7efe54101de0 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.844+0000 7efe5e1ed740 1 Processor -- start 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5e1ed740 1 -- start start 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe4affd640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2cf0 0x7efe400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6060 0x5583ac9f6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583aca00d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583ac95b620 con 0x5583aca01240 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5583ac902e50 con 0x5583ac9f6060 2024-09-12T12:29:53.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583ac9575e0 con 0x5583ac9f6980 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2cf0 0x7efe400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe4affd640 1 -- 172.21.15.137:0/1230917598 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7efe500ef9c0 con 0x7efe54101de0 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5a3b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5a3b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca03630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38732/0 (socket says 172.21.15.137:38732) 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5a3b1640 1 -- 172.21.15.137:0/3076568431 learned_addr learned my addr 172.21.15.137:0/3076568431 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe59bb0640 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6060 0x5583ac9f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe593af640 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583aca00d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe59bb0640 1 -- 172.21.15.137:0/3076568431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 msgr2=0x5583aca00d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe59bb0640 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583aca00d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe59bb0640 1 -- 172.21.15.137:0/3076568431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 msgr2=0x5583aca03630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe59bb0640 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe59bb0640 1 -- 172.21.15.137:0/3076568431 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ac8f3d40 con 0x5583ac9f6060 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.845+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2cf0 0x7efe400a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe4c012780 tx=0x7efe4c012b80 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.846+0000 7efe59bb0640 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6060 0x5583ac9f6440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe500ed0d0 tx=0x7efe500b8330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca850fefa1a189f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.846+0000 7efe3ffff640 1 -- 172.21.15.137:0/3076568431 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50049040 con 0x5583ac9f6060 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.846+0000 7efe3ffff640 1 -- 172.21.15.137:0/3076568431 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe500bc010 con 0x5583ac9f6060 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.846+0000 7efe3ffff640 1 -- 172.21.15.137:0/3076568431 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe500b8680 con 0x5583ac9f6060 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 -- 172.21.15.137:0/3076568431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6060 msgr2=0x5583ac9f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6060 0x5583ac9f6440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe500ed0d0 tx=0x7efe500b8330 comp rx=0 tx=0).stop 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 -- 172.21.15.137:0/3076568431 shutdown_connections 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583aca00d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/3076568431 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6060 0x5583ac9f6440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 -- 172.21.15.137:0/3076568431 >> 172.21.15.137:0/3076568431 conn(0x5583ac9f25f0 msgr2=0x5583ac9f29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 -- 172.21.15.137:0/3076568431 shutdown_connections 2024-09-12T12:29:53.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 -- 172.21.15.137:0/3076568431 wait complete. 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 -- start start 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.847+0000 7efe5e1ed740 1 Processor -- start 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9fe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583ac9fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583aca01240 0x5583ac9feb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5583aca03bd0 con 0x5583ac9f6060 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5583ac8e3f10 con 0x5583ac9f6980 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5583aca03d50 con 0x5583aca01240 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 10 client.0 _ll_register_callbacks cb 0x5583ac907db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9fe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9fe0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38744/0 (socket says 172.21.15.137:38744) 2024-09-12T12:29:53.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 -- 172.21.15.137:0/2792061841 learned_addr learned my addr 172.21.15.137:0/2792061841 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583aca01240 0x5583ac9feb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe593af640 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583ac9fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 -- 172.21.15.137:0/2792061841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583aca01240 msgr2=0x5583ac9feb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583aca01240 0x5583ac9feb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 -- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 msgr2=0x5583ac9fe610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583ac9fe610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 -- 172.21.15.137:0/2792061841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ac66a580 con 0x5583ac9f6060 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe59bb0640 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9fe0d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7efe500ed130 tx=0x7efe500c4c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919d9fe0c062baf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe3d7fa640 1 -- 172.21.15.137:0/2792061841 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50075020 con 0x5583ac9f6060 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe3d7fa640 1 -- 172.21.15.137:0/2792061841 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7efe500ed350 con 0x5583ac9f6060 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe500bc010 con 0x5583ac9f6060 2024-09-12T12:29:53.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe3d7fa640 1 -- 172.21.15.137:0/2792061841 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efe50049070 con 0x5583ac9f6060 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 10 client.17439 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.848+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5583ac734960 con 0x5583ac9f6060 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe3d7fa640 1 -- 172.21.15.137:0/2792061841 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe500bc010 con 0x5583ac9f6060 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe3d7fa640 1 -- 172.21.15.137:0/2792061841 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7efe500bf070 con 0x5583ac9f6060 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe5e1ed740 10 client.17439 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe5e1ed740 10 client.17439 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe5e1ed740 20 client.17439 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca10900 0x5583aca12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe5e1ed740 4 client.17439 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583ac968b90 con 0x5583aca10900 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe593af640 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca10900 0x5583aca12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe3d7fa640 10 client.17439 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.849+0000 7efe593af640 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca10900 0x5583aca12cf0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250f2ac7f9475f20 server_cookie=268e14bd59af607a in_seq=0 out_seq=0 2024-09-12T12:29:53.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe3d7fa640 1 -- 172.21.15.137:0/2792061841 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583ac968b90 con 0x5583aca10900 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe3d7fa640 10 client.17439 handle_command_reply: tid=0 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 client.17439 shutdown 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 msgr2=0x5583ac9fe0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9fe0d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7efe500ed130 tx=0x7efe500c4c40 comp rx=0 tx=0).stop 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 shutdown_connections 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583aca01240 0x5583ac9feb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583ac9fe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca10900 0x5583aca12cf0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/2792061841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9fe0d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 >> 172.21.15.137:0/2792061841 conn(0x5583ac9f25f0 msgr2=0x5583ac9f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe3d7fa640 0 client.17439 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 shutdown_connections 2024-09-12T12:29:53.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 1 -- 172.21.15.137:0/2792061841 wait complete. 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.850+0000 7efe5e1ed740 20 client.17439 trim_cache size 0 max 16384 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 Processor -- start 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 -- start start 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583ac9f1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583aca01240 0x5583aca182f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe541b01b0 con 0x5583ac9f6060 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ad9c0 con 0x5583aca01240 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ab830 con 0x5583ac9f6980 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe59bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe59bb0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38758/0 (socket says 172.21.15.137:38758) 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.867+0000 7efe59bb0640 1 -- 172.21.15.137:0/1482722937 learned_addr learned my addr 172.21.15.137:0/1482722937 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe593af640 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583ac9f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583aca01240 0x5583aca182f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe59bb0640 1 -- 172.21.15.137:0/1482722937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 msgr2=0x5583ac9f1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe59bb0640 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583ac9f1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe59bb0640 1 -- 172.21.15.137:0/1482722937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583aca01240 msgr2=0x5583aca182f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe59bb0640 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583aca01240 0x5583aca182f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe59bb0640 1 -- 172.21.15.137:0/1482722937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583aca13060 con 0x5583ac9f6060 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe59bb0640 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9f8ac0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efe50005ab0 tx=0x7efe500ba520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7244a40027c52574 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe3f7fe640 1 -- 172.21.15.137:0/1482722937 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50075040 con 0x5583ac9f6060 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe3f7fe640 1 -- 172.21.15.137:0/1482722937 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe500bc010 con 0x5583ac9f6060 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.868+0000 7efe3f7fe640 1 -- 172.21.15.137:0/1482722937 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe500bc270 con 0x5583ac9f6060 2024-09-12T12:29:53.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 -- 172.21.15.137:0/1482722937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 msgr2=0x5583ac9f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9f8ac0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7efe50005ab0 tx=0x7efe500ba520 comp rx=0 tx=0).stop 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 -- 172.21.15.137:0/1482722937 shutdown_connections 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583aca01240 0x5583aca182f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6980 0x5583ac9f1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/1482722937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583ac9f6060 0x5583ac9f8ac0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 -- 172.21.15.137:0/1482722937 >> 172.21.15.137:0/1482722937 conn(0x5583ac9f25f0 msgr2=0x5583ac9f0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 -- 172.21.15.137:0/1482722937 shutdown_connections 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 -- 172.21.15.137:0/1482722937 wait complete. 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 -- start start 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 Processor -- start 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.869+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6060 0x5583aca04dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583aca05310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5e1ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca05850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efe541abe80 con 0x5583aca01240 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ad2f0 con 0x5583ac9f6980 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5e1ed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ac550 con 0x5583ac9f6060 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5e1ed740 10 client.0 _ll_register_callbacks cb 0x7efe50003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca05850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca05850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38772/0 (socket says 172.21.15.137:38772) 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 -- 172.21.15.137:0/485744509 learned_addr learned my addr 172.21.15.137:0/485744509 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe59bb0640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6060 0x5583aca04dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe593af640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583aca05310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 -- 172.21.15.137:0/485744509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6060 msgr2=0x5583aca04dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6060 0x5583aca04dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 -- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 msgr2=0x5583aca05310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583aca05310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 -- 172.21.15.137:0/485744509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583aca7a4c0 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe59bb0640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6060 0x5583aca04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5a3b1640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca05850 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00ed80 tx=0x7efe4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d787a30635a4e0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe3d7fa640 1 -- 172.21.15.137:0/485744509 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c02e060 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe3d7fa640 1 -- 172.21.15.137:0/485744509 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7efe4c003b70 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.870+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe500bc010 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe3d7fa640 1 -- 172.21.15.137:0/485744509 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efe4c003d70 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 10 client.17445 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5583aca7b7e0 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe3d7fa640 1 -- 172.21.15.137:0/485744509 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe500bc010 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe3d7fa640 1 -- 172.21.15.137:0/485744509 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7efe4c028030 con 0x5583aca01240 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 10 client.17445 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:53.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 10 client.17445 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:29:53.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 20 client.17445 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:53.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca7bf90 0x5583aca7e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:53.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 4 client.17445 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5b1aeeed", "release": true, "await": true}] 2024-09-12T12:29:53.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5b1aeeed", "release": true, "await": true}) -- 0x5583ac9630a0 con 0x5583aca7bf90 2024-09-12T12:29:53.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.871+0000 7efe59bb0640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca7bf90 0x5583aca7e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:53.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.872+0000 7efe3d7fa640 10 client.17445 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:53.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.872+0000 7efe59bb0640 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca7bf90 0x5583aca7e3b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05e65564927509b server_cookie=29e2cf2a2b930e67 in_seq=0 out_seq=0 2024-09-12T12:29:53.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe3d7fa640 1 -- 172.21.15.137:0/485744509 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5583ac9630a0 con 0x5583aca7bf90 2024-09-12T12:29:53.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe3d7fa640 10 client.17445 handle_command_reply: tid=0 2024-09-12T12:29:53.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 client.17445 shutdown 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 msgr2=0x5583aca05850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca05850 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00ed80 tx=0x7efe4c004ab0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 shutdown_connections 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5583aca7bf90 0x5583aca7e3b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5583aca01240 0x5583aca05850 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5583ac9f6980 0x5583aca05310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe3d7fa640 0 client.17445 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 --2- 172.21.15.137:0/485744509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5583ac9f6060 0x5583aca04dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.908+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 >> 172.21.15.137:0/485744509 conn(0x5583ac9f25f0 msgr2=0x5583ac9f1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:53.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.909+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 shutdown_connections 2024-09-12T12:29:53.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.909+0000 7efe5e1ed740 1 -- 172.21.15.137:0/485744509 wait complete. 2024-09-12T12:29:53.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.909+0000 7efe5e1ed740 20 client.17445 trim_cache size 0 max 16384 2024-09-12T12:29:53.911 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 49, 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 60, 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "5b1aeeed": { 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 60, 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:29:53.912 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.2, 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:53.913 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:29:53.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.909+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2cf0 msgr2=0x7efe400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.909+0000 7efe5be14640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2cf0 0x7efe400a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe4c012780 tx=0x7efe4c012b80 comp rx=0 tx=0).stop 2024-09-12T12:29:53.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.909+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 msgr2=0x7efe54198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:53.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.909+0000 7efe5be14640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe54198450 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7efe50006010 tx=0x7efe500042f0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 shutdown_connections 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7efe400a2cf0 0x7efe400a51b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe54106f10 0x7efe5419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe54102790 0x7efe54198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 --2- 172.21.15.137:0/1230917598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efe54101de0 0x7efe54198450 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 >> 172.21.15.137:0/1230917598 conn(0x7efe540fd8c0 msgr2=0x7efe540fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 shutdown_connections 2024-09-12T12:29:53.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:53.910+0000 7efe5be14640 1 -- 172.21.15.137:0/1230917598 wait complete. 2024-09-12T12:29:54.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 73572 26.29 MB/sec execute 86 sec latency 24144.097 ms 2024-09-12T12:29:54.721 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5b1aeeed', total seconds elapsed: 27.7 2024-09-12T12:29:54.721 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-12T12:29:54.721 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:29:55.022 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:29:55.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:54 smithi137 ceph-mon[30950]: pgmap v529: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 5.7 MiB/s wr, 205 op/s 2024-09-12T12:29:55.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:54 smithi171 ceph-mon[35098]: pgmap v529: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 5.7 MiB/s wr, 205 op/s 2024-09-12T12:29:55.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 74447 26.57 MB/sec execute 87 sec latency 40.464 ms 2024-09-12T12:29:55.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:54 smithi189 ceph-mon[35475]: pgmap v529: 97 pgs: 97 active+clean; 887 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 5.7 MiB/s wr, 205 op/s 2024-09-12T12:29:55.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.625+0000 7f15771a0640 1 -- 172.21.15.137:0/408654500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 msgr2=0x7f157010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:55.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.625+0000 7f15771a0640 1 --2- 172.21.15.137:0/408654500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 0x7f157010da40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1564009860 tx=0x7f156402a410 comp rx=0 tx=0).stop 2024-09-12T12:29:55.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.628+0000 7f15771a0640 1 -- 172.21.15.137:0/408654500 shutdown_connections 2024-09-12T12:29:55.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.628+0000 7f15771a0640 1 --2- 172.21.15.137:0/408654500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 0x7f157010da40 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.628+0000 7f15771a0640 1 --2- 172.21.15.137:0/408654500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570102570 0x7f15701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.628+0000 7f15771a0640 1 --2- 172.21.15.137:0/408654500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1570100e20 0x7f1570101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.628+0000 7f15771a0640 1 -- 172.21.15.137:0/408654500 >> 172.21.15.137:0/408654500 conn(0x7f15700fc7d0 msgr2=0x7f15700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:55.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.631+0000 7f15771a0640 1 -- 172.21.15.137:0/408654500 shutdown_connections 2024-09-12T12:29:55.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.632+0000 7f15771a0640 1 -- 172.21.15.137:0/408654500 wait complete. 2024-09-12T12:29:55.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 Processor -- start 2024-09-12T12:29:55.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 -- start start 2024-09-12T12:29:55.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570100e20 0x7f157010d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:55.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1570102570 0x7f15701087b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:55.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 0x7f1570108cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:55.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f157006a9c0 con 0x7f1570102ef0 2024-09-12T12:29:55.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f157006a840 con 0x7f1570100e20 2024-09-12T12:29:55.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.633+0000 7f15771a0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f157006ab40 con 0x7f1570102570 2024-09-12T12:29:55.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.634+0000 7f1574f3c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570100e20 0x7f157010d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:55.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.634+0000 7f1574f3c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570100e20 0x7f157010d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:46140/0 (socket says 172.21.15.137:46140) 2024-09-12T12:29:55.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.634+0000 7f1574f3c640 1 -- 172.21.15.137:0/1251761744 learned_addr learned my addr 172.21.15.137:0/1251761744 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:55.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.634+0000 7f156ffff640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1570102570 0x7f15701087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:55.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.634+0000 7f157573d640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 0x7f1570108cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:55.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.635+0000 7f1574f3c640 1 -- 172.21.15.137:0/1251761744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1570102570 msgr2=0x7f15701087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:55.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.635+0000 7f1574f3c640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1570102570 0x7f15701087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.635+0000 7f1574f3c640 1 -- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 msgr2=0x7f1570108cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:55.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.635+0000 7f1574f3c640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 0x7f1570108cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.635+0000 7f1574f3c640 1 -- 172.21.15.137:0/1251761744 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1570109420 con 0x7f1570100e20 2024-09-12T12:29:55.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.635+0000 7f1574f3c640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570100e20 0x7f157010d720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f156800ed80 tx=0x7f156800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:55.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.636+0000 7f156dffb640 1 -- 172.21.15.137:0/1251761744 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156801c070 con 0x7f1570100e20 2024-09-12T12:29:55.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.637+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15701ab140 con 0x7f1570100e20 2024-09-12T12:29:55.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.637+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15701ab670 con 0x7f1570100e20 2024-09-12T12:29:55.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.637+0000 7f156dffb640 1 -- 172.21.15.137:0/1251761744 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1568017b20 con 0x7f1570100e20 2024-09-12T12:29:55.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.637+0000 7f156dffb640 1 -- 172.21.15.137:0/1251761744 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156800fd30 con 0x7f1570100e20 2024-09-12T12:29:55.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.642+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155c005290 con 0x7f1570100e20 2024-09-12T12:29:55.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.642+0000 7f156dffb640 1 -- 172.21.15.137:0/1251761744 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f156801f070 con 0x7f1570100e20 2024-09-12T12:29:55.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.644+0000 7f156dffb640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15580a2ec0 0x7f15580a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:55.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.644+0000 7f156dffb640 1 -- 172.21.15.137:0/1251761744 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f15680cf220 con 0x7f1570100e20 2024-09-12T12:29:55.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.645+0000 7f156ffff640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15580a2ec0 0x7f15580a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:55.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.645+0000 7f156ffff640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15580a2ec0 0x7f15580a5380 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1570109f70 tx=0x7f15600090e0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.651+0000 7f156dffb640 1 -- 172.21.15.137:0/1251761744 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f156809b8c0 con 0x7f1570100e20 2024-09-12T12:29:55.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.906+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f155c005850 con 0x7f1570100e20 2024-09-12T12:29:55.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.907+0000 7f156dffb640 1 -- 172.21.15.137:0/1251761744 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11942 (secure 0 0 0) 0x7f156809fbc0 con 0x7f1570100e20 2024-09-12T12:29:55.908 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:29:55.909 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":49,"btime":"2024-09-12T12:29:33:838508+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:29:32.898454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26647":{"gid":26647,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3747343886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3747343886},{"type":"v1","addr":"172.21.15.189:6835","nonce":3747343886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:29:55.909 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 49 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15580a2ec0 msgr2=0x7f15580a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15580a2ec0 0x7f15580a5380 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1570109f70 tx=0x7f15600090e0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570100e20 msgr2=0x7f157010d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570100e20 0x7f157010d720 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f156800ed80 tx=0x7f156800c6e0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 shutdown_connections 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f15580a2ec0 0x7f15580a5380 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1570102ef0 0x7f1570108cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1570102570 0x7f15701087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 --2- 172.21.15.137:0/1251761744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1570100e20 0x7f157010d720 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 >> 172.21.15.137:0/1251761744 conn(0x7f15700fc7d0 msgr2=0x7f1570106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:55.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 shutdown_connections 2024-09-12T12:29:55.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:55.910+0000 7f15771a0640 1 -- 172.21.15.137:0/1251761744 wait complete. 2024-09-12T12:29:56.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 74792 26.53 MB/sec execute 88 sec latency 279.374 ms 2024-09-12T12:29:56.652 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-12T12:29:56.652 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-12T12:29:56.979 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:29:57.172 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:56 smithi137 ceph-mon[30950]: pgmap v530: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.9 MiB/s wr, 360 op/s 2024-09-12T12:29:57.173 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:56 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1251761744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:57.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:56 smithi171 ceph-mon[35098]: pgmap v530: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.9 MiB/s wr, 360 op/s 2024-09-12T12:29:57.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:56 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1251761744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:57.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75425 26.57 MB/sec execute 89 sec latency 49.016 ms 2024-09-12T12:29:57.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:56 smithi189 ceph-mon[35475]: pgmap v530: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.9 MiB/s wr, 360 op/s 2024-09-12T12:29:57.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:56 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1251761744' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.581+0000 7ff5de3b7640 1 -- 172.21.15.137:0/588211020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 msgr2=0x7ff5d8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.581+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/588211020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 0x7ff5d8106e50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8009900 tx=0x7ff5c802a410 comp rx=0 tx=0).stop 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.582+0000 7ff5de3b7640 1 -- 172.21.15.137:0/588211020 shutdown_connections 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.582+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/588211020 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5d8101410 0x7ff5d810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.582+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/588211020 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 0x7ff5d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.582+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/588211020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 0x7ff5d8106e50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.582+0000 7ff5de3b7640 1 -- 172.21.15.137:0/588211020 >> 172.21.15.137:0/588211020 conn(0x7ff5d80fc7d0 msgr2=0x7ff5d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.582+0000 7ff5de3b7640 1 -- 172.21.15.137:0/588211020 shutdown_connections 2024-09-12T12:29:57.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.582+0000 7ff5de3b7640 1 -- 172.21.15.137:0/588211020 wait complete. 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.583+0000 7ff5de3b7640 1 Processor -- start 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.583+0000 7ff5de3b7640 1 -- start start 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5de3b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 0x7ff5d810e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5de3b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5d8101410 0x7ff5d810e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5de3b7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 0x7ff5d810eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5de3b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8111f50 con 0x7ff5d8100a70 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5de3b7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d8111dd0 con 0x7ff5d8101410 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5de3b7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d81120d0 con 0x7ff5d8106a70 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 0x7ff5d810e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 0x7ff5d810e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38830/0 (socket says 172.21.15.137:38830) 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 -- 172.21.15.137:0/4287519563 learned_addr learned my addr 172.21.15.137:0/4287519563 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5cffff640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5d8101410 0x7ff5d810e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5dc954640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 0x7ff5d810eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 -- 172.21.15.137:0/4287519563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 msgr2=0x7ff5d810eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:57.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 0x7ff5d810eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 -- 172.21.15.137:0/4287519563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5d8101410 msgr2=0x7ff5d810e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5d8101410 0x7ff5d810e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5d7fff640 1 -- 172.21.15.137:0/4287519563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d81aae00 con 0x7ff5d8100a70 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.584+0000 7ff5cffff640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5d8101410 0x7ff5d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.585+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 0x7ff5d810e0d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.585+0000 7ff5d5ffb640 1 -- 172.21.15.137:0/4287519563 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c803f030 con 0x7ff5d8100a70 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.585+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d81aafd0 con 0x7ff5d8100a70 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.585+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d81ab480 con 0x7ff5d8100a70 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.585+0000 7ff5d5ffb640 1 -- 172.21.15.137:0/4287519563 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c8004440 con 0x7ff5d8100a70 2024-09-12T12:29:57.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.585+0000 7ff5d5ffb640 1 -- 172.21.15.137:0/4287519563 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8035070 con 0x7ff5d8100a70 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.588+0000 7ff5d5ffb640 1 -- 172.21.15.137:0/4287519563 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff5c8042030 con 0x7ff5d8100a70 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.589+0000 7ff5d5ffb640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff5c40a2d70 0x7ff5c40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.589+0000 7ff5e0790740 1 Processor -- start 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.589+0000 7ff5d5ffb640 1 -- 172.21.15.137:0/4287519563 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7ff5c80ef610 con 0x7ff5d8100a70 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.589+0000 7ff5e0790740 1 -- start start 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.589+0000 7ff5cffff640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff5c40a2d70 0x7ff5c40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d74850 0x55e0b1d74c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d75170 0x55e0b1d7f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d81e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5e0790740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1cd9160 con 0x55e0b1d75170 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5e0790740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1c80720 con 0x55e0b1d7fa30 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5e0790740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1cd5120 con 0x55e0b1d74850 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d75170 0x55e0b1d7f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5dc954640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d81e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d75170 0x55e0b1d7f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38844/0 (socket says 172.21.15.137:38844) 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 -- 172.21.15.137:0/1511602306 learned_addr learned my addr 172.21.15.137:0/1511602306 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d74850 0x55e0b1d74c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 -- 172.21.15.137:0/1511602306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d74850 msgr2=0x55e0b1d74c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d74850 0x55e0b1d74c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 -- 172.21.15.137:0/1511602306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 msgr2=0x55e0b1d81e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d81e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.590+0000 7ff5cffff640 1 -- 172.21.15.137:0/1511602306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b1a4aba0 con 0x55e0b1d75170 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.591+0000 7ff5cffff640 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d75170 0x55e0b1d7f4f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0013ae0 tx=0x7ff5c000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e59399cae47c83b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:57.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.591+0000 7ff5ce7fc640 1 -- 172.21.15.137:0/1511602306 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000fb90 con 0x55e0b1d75170 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.591+0000 7ff5ce7fc640 1 -- 172.21.15.137:0/1511602306 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c000fdf0 con 0x55e0b1d75170 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.591+0000 7ff5cffff640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff5c40a2d70 0x7ff5c40a5230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0020000 tx=0x7ff5c0016000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.591+0000 7ff5ce7fc640 1 -- 172.21.15.137:0/1511602306 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0004ab0 con 0x55e0b1d75170 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.593+0000 7ff5e0790740 1 -- 172.21.15.137:0/1511602306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d75170 msgr2=0x55e0b1d7f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.593+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d75170 0x55e0b1d7f4f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0013ae0 tx=0x7ff5c000f3c0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.593+0000 7ff5e0790740 1 -- 172.21.15.137:0/1511602306 shutdown_connections 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.593+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d81e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.593+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d75170 0x55e0b1d7f4f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.593+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1511602306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d74850 0x55e0b1d74c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.593+0000 7ff5e0790740 1 -- 172.21.15.137:0/1511602306 >> 172.21.15.137:0/1511602306 conn(0x55e0b1d70de0 msgr2=0x55e0b1d711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:57.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.596+0000 7ff5e0790740 1 -- 172.21.15.137:0/1511602306 shutdown_connections 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.596+0000 7ff5e0790740 1 -- 172.21.15.137:0/1511602306 wait complete. 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.596+0000 7ff5e0790740 1 -- start start 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.596+0000 7ff5e0790740 1 Processor -- start 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d7c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d75170 0x55e0b1d7ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d7d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5e0790740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1d82650 con 0x55e0b1d74850 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5cffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d75170 0x55e0b1d7ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5cffff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d75170 0x55e0b1d7ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:46182/0 (socket says 172.21.15.137:46182) 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5cffff640 1 -- 172.21.15.137:0/1407540907 learned_addr learned my addr 172.21.15.137:0/1407540907 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1c61900 con 0x55e0b1d75170 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d7c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e0b1d827d0 con 0x55e0b1d7fa30 2024-09-12T12:29:57.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.597+0000 7ff5dc954640 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d7d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.598+0000 7ff5e0790740 10 client.0 _ll_register_callbacks cb 0x55e0b1c5f780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.598+0000 7ff5dc954640 1 -- 172.21.15.137:0/1407540907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d75170 msgr2=0x55e0b1d7ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.598+0000 7ff5dc954640 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d75170 0x55e0b1d7ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.598+0000 7ff5dc954640 1 -- 172.21.15.137:0/1407540907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 msgr2=0x55e0b1d7c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.598+0000 7ff5dc954640 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d7c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.598+0000 7ff5dc954640 1 -- 172.21.15.137:0/1407540907 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b1ca1c50 con 0x55e0b1d7fa30 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5dc954640 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d7d3d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0011940 tx=0x7ff5d0011e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984665340a2544c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5b77fe640 1 -- 172.21.15.137:0/1407540907 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0010ca0 con 0x55e0b1d7fa30 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c000fdf0 con 0x55e0b1d7fa30 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5b77fe640 1 -- 172.21.15.137:0/1407540907 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7ff5d0010e40 con 0x55e0b1d7fa30 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5b77fe640 1 -- 172.21.15.137:0/1407540907 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff5d0004b20 con 0x55e0b1d7fa30 2024-09-12T12:29:57.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5e0790740 10 client.26779 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.599+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e0b1d9f430 con 0x55e0b1d7fa30 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.601+0000 7ff5b77fe640 1 -- 172.21.15.137:0/1407540907 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c000fdf0 con 0x55e0b1d7fa30 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.602+0000 7ff5b77fe640 1 -- 172.21.15.137:0/1407540907 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff5d0021030 con 0x55e0b1d7fa30 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.602+0000 7ff5e0790740 10 client.26779 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.602+0000 7ff5e0790740 10 client.26779 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.602+0000 7ff5e0790740 20 client.26779 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.602+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1da0dc0 0x55e0b1da3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.602+0000 7ff5e0790740 4 client.26779 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.602+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0b1ce6890 con 0x55e0b1da0dc0 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.603+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1da0dc0 0x55e0b1da3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.604+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1da0dc0 0x55e0b1da3220 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bd06b24f113a859 server_cookie=10d0b3c4a851d733 in_seq=0 out_seq=0 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.604+0000 7ff5b77fe640 10 client.26779 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5b77fe640 1 -- 172.21.15.137:0/1407540907 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0b1ce6890 con 0x55e0b1da0dc0 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5b77fe640 10 client.26779 handle_command_reply: tid=0 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 client.26779 shutdown 2024-09-12T12:29:57.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d7fa30 msgr2=0x55e0b1d7d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d7d3d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0011940 tx=0x7ff5d0011e00 comp rx=0 tx=0).stop 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 shutdown_connections 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d7fa30 0x55e0b1d7d3d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d75170 0x55e0b1d7ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1da0dc0 0x55e0b1da3220 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 --2- 172.21.15.137:0/1407540907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d7c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 >> 172.21.15.137:0/1407540907 conn(0x55e0b1d70de0 msgr2=0x55e0b1d76ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.606+0000 7ff5b77fe640 0 client.26779 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.607+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 shutdown_connections 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.607+0000 7ff5e0790740 1 -- 172.21.15.137:0/1407540907 wait complete. 2024-09-12T12:29:57.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.607+0000 7ff5e0790740 20 client.26779 trim_cache size 0 max 16384 2024-09-12T12:29:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 Processor -- start 2024-09-12T12:29:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 -- start start 2024-09-12T12:29:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d74850 0x55e0b1d769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1da8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d7fa30 0x55e0b1da9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b8011050 con 0x55e0b1d7fa30 2024-09-12T12:29:57.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b800e860 con 0x55e0b1d74850 2024-09-12T12:29:57.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5e0790740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b800c6d0 con 0x55e0b1d75170 2024-09-12T12:29:57.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.632+0000 7ff5dc954640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d7fa30 0x55e0b1da9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5dc954640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d7fa30 0x55e0b1da9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38862/0 (socket says 172.21.15.137:38862) 2024-09-12T12:29:57.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5dc954640 1 -- 172.21.15.137:0/5996571 learned_addr learned my addr 172.21.15.137:0/5996571 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:57.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d74850 0x55e0b1d769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5cffff640 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1da8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5dc954640 1 -- 172.21.15.137:0/5996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 msgr2=0x55e0b1da8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5dc954640 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1da8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5dc954640 1 -- 172.21.15.137:0/5996571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d74850 msgr2=0x55e0b1d769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5dc954640 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d74850 0x55e0b1d769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5dc954640 1 -- 172.21.15.137:0/5996571 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b1d782b0 con 0x55e0b1d7fa30 2024-09-12T12:29:57.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.633+0000 7ff5cffff640 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1da8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:29:57.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.634+0000 7ff5dc954640 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d7fa30 0x55e0b1da9070 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00114c0 tx=0x7ff5d0005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666ca664348ef636 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:57.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.634+0000 7ff5cdffb640 1 -- 172.21.15.137:0/5996571 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0010ca0 con 0x55e0b1d7fa30 2024-09-12T12:29:57.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.634+0000 7ff5cdffb640 1 -- 172.21.15.137:0/5996571 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d0010e40 con 0x55e0b1d7fa30 2024-09-12T12:29:57.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.635+0000 7ff5cdffb640 1 -- 172.21.15.137:0/5996571 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0022790 con 0x55e0b1d7fa30 2024-09-12T12:29:57.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 -- 172.21.15.137:0/5996571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d7fa30 msgr2=0x55e0b1da9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:57.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d7fa30 0x55e0b1da9070 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00114c0 tx=0x7ff5d0005fb0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 -- 172.21.15.137:0/5996571 shutdown_connections 2024-09-12T12:29:57.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d7fa30 0x55e0b1da9070 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1da8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 --2- 172.21.15.137:0/5996571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d74850 0x55e0b1d769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 -- 172.21.15.137:0/5996571 >> 172.21.15.137:0/5996571 conn(0x55e0b1d70de0 msgr2=0x55e0b1d762a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.636+0000 7ff5e0790740 1 -- 172.21.15.137:0/5996571 shutdown_connections 2024-09-12T12:29:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.637+0000 7ff5e0790740 1 -- 172.21.15.137:0/5996571 wait complete. 2024-09-12T12:29:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.637+0000 7ff5e0790740 1 -- start start 2024-09-12T12:29:57.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.638+0000 7ff5e0790740 1 Processor -- start 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.638+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.638+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1d845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.638+0000 7ff5e0790740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1e04ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.638+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d840a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38868/0 (socket says 172.21.15.137:38868) 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 -- 172.21.15.137:0/872881609 learned_addr learned my addr 172.21.15.137:0/872881609 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b800cd20 con 0x55e0b1d74850 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b800e190 con 0x55e0b1d7fa30 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b800d3f0 con 0x55e0b1d75170 2024-09-12T12:29:57.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5e0790740 10 client.0 _ll_register_callbacks cb 0x7ff5d0009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5dc954640 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1e04ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 -- 172.21.15.137:0/872881609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 msgr2=0x55e0b1d845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1d845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 -- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 msgr2=0x55e0b1e04ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1e04ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 -- 172.21.15.137:0/872881609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b1e05110 con 0x55e0b1d74850 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5d7fff640 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d840a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80b8070 tx=0x7ff5c80bb690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccebb0aa578bb379 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.639+0000 7ff5b77fe640 1 -- 172.21.15.137:0/872881609 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8049040 con 0x55e0b1d74850 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5b77fe640 1 -- 172.21.15.137:0/872881609 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7ff5c80b8280 con 0x55e0b1d74850 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5d0010e40 con 0x55e0b1d74850 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5b77fe640 1 -- 172.21.15.137:0/872881609 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff5c80c4880 con 0x55e0b1d74850 2024-09-12T12:29:57.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 10 client.17469 fetch_fsmap learned FSMap version 49 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55e0b1e0c4e0 con 0x55e0b1d74850 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5b77fe640 1 -- 172.21.15.137:0/872881609 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5d0010e40 con 0x55e0b1d74850 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5b77fe640 1 -- 172.21.15.137:0/872881609 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 49) ==== 8362+0+0 (secure 0 0 0) 0x7ff5c80c43f0 con 0x55e0b1d74850 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 10 client.17469 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 10 client.17469 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 20 client.17469 populate_metadata read hostname 'smithi137' 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1e0cd20 0x55e0b1e0f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 4 client.17469 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.640+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x55e0b1d6c410 con 0x55e0b1e0cd20 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.641+0000 7ff5cffff640 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1e0cd20 0x55e0b1e0f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:29:57.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.641+0000 7ff5b77fe640 10 client.17469 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:57.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:57.641+0000 7ff5cffff640 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1e0cd20 0x55e0b1e0f160 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eada60b2229ecc7d server_cookie=40278488c43f715c in_seq=0 out_seq=0 2024-09-12T12:29:58.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5b77fe640 1 -- 172.21.15.137:0/872881609 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e0b1d6c410 con 0x55e0b1e0cd20 2024-09-12T12:29:58.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5b77fe640 10 client.17469 handle_command_reply: tid=0 2024-09-12T12:29:58.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 client.17469 shutdown 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 msgr2=0x55e0b1d840a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d840a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80b8070 tx=0x7ff5c80bb690 comp rx=0 tx=0).stop 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 shutdown_connections 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e0b1d7fa30 0x55e0b1e04ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0b1d75170 0x55e0b1d845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e0b1e0cd20 0x55e0b1e0f160 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 --2- 172.21.15.137:0/872881609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e0b1d74850 0x55e0b1d840a0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5b77fe640 0 client.17469 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 >> 172.21.15.137:0/872881609 conn(0x55e0b1d70de0 msgr2=0x55e0b1d76ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 shutdown_connections 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.109+0000 7ff5e0790740 1 -- 172.21.15.137:0/872881609 wait complete. 2024-09-12T12:29:58.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.110+0000 7ff5e0790740 20 client.17469 trim_cache size 0 max 16384 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 49, 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 62, 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout: "c66e0300": { 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 62, 2024-09-12T12:29:58.112 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.2, 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:58.113 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:58.114 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:58.114 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:58.114 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:29:58.114 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:29:58.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.110+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff5c40a2d70 msgr2=0x7ff5c40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.110+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff5c40a2d70 0x7ff5c40a5230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0020000 tx=0x7ff5c0016000 comp rx=0 tx=0).stop 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.110+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 msgr2=0x7ff5d810e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.110+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 0x7ff5d810e0d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8006010 tx=0x7ff5c8004040 comp rx=0 tx=0).stop 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 shutdown_connections 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff5c40a2d70 0x7ff5c40a5230 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5d8106a70 0x7ff5d810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5d8101410 0x7ff5d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 --2- 172.21.15.137:0/4287519563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5d8100a70 0x7ff5d810e0d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 >> 172.21.15.137:0/4287519563 conn(0x7ff5d80fc7d0 msgr2=0x7ff5d80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 shutdown_connections 2024-09-12T12:29:58.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:29:58.111+0000 7ff5de3b7640 1 -- 172.21.15.137:0/4287519563 wait complete. 2024-09-12T12:29:58.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 26.38 MB/sec execute 90 sec latency 585.732 ms 2024-09-12T12:29:58.871 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c66e0300', quiesce took 4.1 seconds. Will release after: 20.15 2024-09-12T12:29:59.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:29:58 smithi171 ceph-mon[35098]: pgmap v531: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 250 op/s 2024-09-12T12:29:59.224 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 26.09 MB/sec execute 91 sec latency 1585.825 ms 2024-09-12T12:29:59.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:29:58 smithi137 ceph-mon[30950]: pgmap v531: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 250 op/s 2024-09-12T12:29:59.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:29:58 smithi189 ceph-mon[35475]: pgmap v531: 97 pgs: 97 active+clean; 890 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.9 MiB/s wr, 250 op/s 2024-09-12T12:30:00.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 25.81 MB/sec execute 92 sec latency 2585.866 ms 2024-09-12T12:30:01.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:00 smithi171 ceph-mon[35098]: pgmap v532: 97 pgs: 97 active+clean; 898 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 331 op/s 2024-09-12T12:30:01.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:00 smithi171 ceph-mon[35098]: overall HEALTH_OK 2024-09-12T12:30:01.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 25.53 MB/sec execute 93 sec latency 3585.970 ms 2024-09-12T12:30:01.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:00 smithi137 ceph-mon[30950]: pgmap v532: 97 pgs: 97 active+clean; 898 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 331 op/s 2024-09-12T12:30:01.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:00 smithi137 ceph-mon[30950]: overall HEALTH_OK 2024-09-12T12:30:01.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:00 smithi189 ceph-mon[35475]: pgmap v532: 97 pgs: 97 active+clean; 898 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 331 op/s 2024-09-12T12:30:01.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:00 smithi189 ceph-mon[35475]: overall HEALTH_OK 2024-09-12T12:30:02.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 25.26 MB/sec execute 94 sec latency 4586.061 ms 2024-09-12T12:30:03.143 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:02 smithi189 ceph-mon[35475]: pgmap v533: 97 pgs: 97 active+clean; 905 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.6 MiB/s wr, 460 op/s 2024-09-12T12:30:03.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:02 smithi171 ceph-mon[35098]: pgmap v533: 97 pgs: 97 active+clean; 905 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.6 MiB/s wr, 460 op/s 2024-09-12T12:30:03.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 24.99 MB/sec execute 95 sec latency 5586.110 ms 2024-09-12T12:30:03.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:02 smithi137 ceph-mon[30950]: pgmap v533: 97 pgs: 97 active+clean; 905 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.6 MiB/s wr, 460 op/s 2024-09-12T12:30:04.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 24.73 MB/sec execute 96 sec latency 6586.206 ms 2024-09-12T12:30:05.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:04 smithi171 ceph-mon[35098]: pgmap v534: 97 pgs: 97 active+clean; 905 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.6 MiB/s wr, 391 op/s 2024-09-12T12:30:05.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:04 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:05.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 24.48 MB/sec execute 97 sec latency 7586.309 ms 2024-09-12T12:30:05.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:04 smithi137 ceph-mon[30950]: pgmap v534: 97 pgs: 97 active+clean; 905 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.6 MiB/s wr, 391 op/s 2024-09-12T12:30:05.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:04 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:05.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:04 smithi189 ceph-mon[35475]: pgmap v534: 97 pgs: 97 active+clean; 905 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.6 MiB/s wr, 391 op/s 2024-09-12T12:30:05.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:04 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 -- 172.21.15.137:0/881532661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 msgr2=0x7f111810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 --2- 172.21.15.137:0/881532661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f111810d880 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1110009930 tx=0x7f111002a420 comp rx=0 tx=0).stop 2024-09-12T12:30:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 -- 172.21.15.137:0/881532661 shutdown_connections 2024-09-12T12:30:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 --2- 172.21.15.137:0/881532661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f111810d880 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 --2- 172.21.15.137:0/881532661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11181009b0 0x7f1118100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 --2- 172.21.15.137:0/881532661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11181069c0 0x7f1118106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 -- 172.21.15.137:0/881532661 >> 172.21.15.137:0/881532661 conn(0x7f1118077990 msgr2=0x7f11180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 -- 172.21.15.137:0/881532661 shutdown_connections 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.962+0000 7f111ef13640 1 -- 172.21.15.137:0/881532661 wait complete. 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 Processor -- start 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 -- start start 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11181009b0 0x7f1118198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f11181987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11181069c0 0x7f111819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110e00 con 0x7f1118101350 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110c80 con 0x7f11181009b0 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ef13640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1118110f80 con 0x7f11181069c0 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f1117fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f11181987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f1117fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f11181987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58478/0 (socket says 172.21.15.137:58478) 2024-09-12T12:30:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f1117fff640 1 -- 172.21.15.137:0/125960814 learned_addr learned my addr 172.21.15.137:0/125960814 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111ccaf640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11181009b0 0x7f1118198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f111d4b0640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11181069c0 0x7f111819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f1117fff640 1 -- 172.21.15.137:0/125960814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11181069c0 msgr2=0x7f111819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f1117fff640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11181069c0 0x7f111819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f1117fff640 1 -- 172.21.15.137:0/125960814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11181009b0 msgr2=0x7f1118198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.963+0000 7f1117fff640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11181009b0 0x7f1118198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f1117fff640 1 -- 172.21.15.137:0/125960814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111819d250 con 0x7f1118101350 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f111d4b0640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11181069c0 0x7f111819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f1117fff640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f11181987a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f110c00ea80 tx=0x7f110c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f1115ffb640 1 -- 172.21.15.137:0/125960814 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c00ce40 con 0x7f1118101350 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f1115ffb640 1 -- 172.21.15.137:0/125960814 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f110c005020 con 0x7f1118101350 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111819d540 con 0x7f1118101350 2024-09-12T12:30:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f1115ffb640 1 -- 172.21.15.137:0/125960814 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c011760 con 0x7f1118101350 2024-09-12T12:30:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.964+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11181a4dd0 con 0x7f1118101350 2024-09-12T12:30:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.966+0000 7f1115ffb640 1 -- 172.21.15.137:0/125960814 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f110c0051d0 con 0x7f1118101350 2024-09-12T12:30:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 Processor -- start 2024-09-12T12:30:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 -- start start 2024-09-12T12:30:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1115ffb640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f11000a2da0 0x7f11000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563d9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559b563332f0 con 0x559b563ceaf0 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559b562db270 con 0x559b563cf410 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1121ca3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559b5632f2b0 con 0x559b563d9cb0 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111d4b0640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f11000a2da0 0x7f11000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1115ffb640 1 -- 172.21.15.137:0/125960814 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f110c0cfb60 con 0x7f1118101350 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58486/0 (socket says 172.21.15.137:58486) 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 -- 172.21.15.137:0/3030676235 learned_addr learned my addr 172.21.15.137:0/3030676235 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f1117fff640 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563d9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111d4b0640 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 -- 172.21.15.137:0/3030676235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 msgr2=0x559b563dc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563dc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 -- 172.21.15.137:0/3030676235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 msgr2=0x559b563d9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563d9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.967+0000 7f111ccaf640 1 -- 172.21.15.137:0/3030676235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b562da340 con 0x559b563ceaf0 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.968+0000 7f111ccaf640 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ceed0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f110800cbe0 tx=0x7f110800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c41df0d0040834 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.968+0000 7f111d4b0640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f11000a2da0 0x7f11000a5260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1110005ab0 tx=0x7f1110032f80 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.968+0000 7f10feffd640 1 -- 172.21.15.137:0/3030676235 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108005220 con 0x559b563ceaf0 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.968+0000 7f10feffd640 1 -- 172.21.15.137:0/3030676235 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1108004c80 con 0x559b563ceaf0 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.968+0000 7f10feffd640 1 -- 172.21.15.137:0/3030676235 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080107b0 con 0x559b563ceaf0 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 -- 172.21.15.137:0/3030676235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 msgr2=0x559b563ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ceed0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f110800cbe0 tx=0x7f110800e720 comp rx=0 tx=0).stop 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 -- 172.21.15.137:0/3030676235 shutdown_connections 2024-09-12T12:30:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563d9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 --2- 172.21.15.137:0/3030676235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ceed0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 -- 172.21.15.137:0/3030676235 >> 172.21.15.137:0/3030676235 conn(0x559b563cb060 msgr2=0x559b563cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 -- 172.21.15.137:0/3030676235 shutdown_connections 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 -- 172.21.15.137:0/3030676235 wait complete. 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 -- start start 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.969+0000 7f1121ca3740 1 Processor -- start 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563ceaf0 0x559b563d6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563d9cb0 0x559b563d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559b563dc880 con 0x559b563cf410 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559b562bc290 con 0x559b563d9cb0 2024-09-12T12:30:05.976 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559b563dca00 con 0x559b563ceaf0 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 10 client.0 _ll_register_callbacks cb 0x559b562ed140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563d7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58488/0 (socket says 172.21.15.137:58488) 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 -- 172.21.15.137:0/754248973 learned_addr learned my addr 172.21.15.137:0/754248973 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f111d4b0640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563d9cb0 0x559b563d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f111ccaf640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563ceaf0 0x559b563d6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 -- 172.21.15.137:0/754248973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563ceaf0 msgr2=0x559b563d6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563ceaf0 0x559b563d6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 -- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563d9cb0 msgr2=0x559b563d7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563d9cb0 0x559b563d7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 -- 172.21.15.137:0/754248973 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b56046210 con 0x559b563cf410 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f111ccaf640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563ceaf0 0x559b563d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1117fff640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563d7040 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f110c0984a0 tx=0x7f110c0a04b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684b08728b04bbd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f10f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f10f7fff640 1 -- 172.21.15.137:0/754248973 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c01f040 con 0x559b563cf410 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f10f7fff640 1 -- 172.21.15.137:0/754248973 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f110c005540 con 0x559b563cf410 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1108004c80 con 0x559b563cf410 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f10f7fff640 1 -- 172.21.15.137:0/754248973 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f110c0a00a0 con 0x559b563cf410 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 10 client.17481 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.970+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x559b56112620 con 0x559b563cf410 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f10f7fff640 1 -- 172.21.15.137:0/754248973 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1108004c80 con 0x559b563cf410 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f10f7fff640 1 -- 172.21.15.137:0/754248973 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f110c00f040 con 0x559b563cf410 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f1121ca3740 10 client.17481 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f1121ca3740 10 client.17481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f1121ca3740 20 client.17481 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f1121ca3740 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b563e9bd0 0x559b563ec060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f1121ca3740 4 client.17481 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:30:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b56340a20 con 0x559b563e9bd0 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.971+0000 7f111ccaf640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b563e9bd0 0x559b563ec060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f10f7fff640 10 client.17481 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f111ccaf640 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b563e9bd0 0x559b563ec060 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e72f65e41aef4 server_cookie=4e0799ed71f85dca in_seq=0 out_seq=0 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f10f7fff640 1 -- 172.21.15.137:0/754248973 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b56340a20 con 0x559b563e9bd0 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f10f7fff640 10 client.17481 handle_command_reply: tid=0 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 client.17481 shutdown 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 msgr2=0x559b563d7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563d7040 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f110c0984a0 tx=0x7f110c0a04b0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 shutdown_connections 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563d9cb0 0x559b563d7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b563e9bd0 0x559b563ec060 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563d7040 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 --2- 172.21.15.137:0/754248973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563ceaf0 0x559b563d6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f10f7fff640 0 client.17481 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 >> 172.21.15.137:0/754248973 conn(0x559b563cb060 msgr2=0x559b563d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.972+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 shutdown_connections 2024-09-12T12:30:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.973+0000 7f1121ca3740 1 -- 172.21.15.137:0/754248973 wait complete. 2024-09-12T12:30:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.973+0000 7f1121ca3740 20 client.17481 trim_cache size 0 max 16384 2024-09-12T12:30:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 Processor -- start 2024-09-12T12:30:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 -- start start 2024-09-12T12:30:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563ceaf0 0x559b563d73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11181b0050 con 0x559b563cf410 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f11181ad890 con 0x559b563ceaf0 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1121ca3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11181ab700 con 0x559b563d9cb0 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58494/0 (socket says 172.21.15.137:58494) 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 -- 172.21.15.137:0/1557582746 learned_addr learned my addr 172.21.15.137:0/1557582746 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f111d4b0640 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 -- 172.21.15.137:0/1557582746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 msgr2=0x559b563f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 -- 172.21.15.137:0/1557582746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563ceaf0 msgr2=0x559b563d73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f111ccaf640 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563ceaf0 0x559b563d73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563ceaf0 0x559b563d73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f1117fff640 1 -- 172.21.15.137:0/1557582746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b563cae90 con 0x559b563cf410 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.987+0000 7f111d4b0640 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1117fff640 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563f1920 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f110c098a10 tx=0x7f110c0a4e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35663d0740c934be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f10fe7fc640 1 -- 172.21.15.137:0/1557582746 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c01f040 con 0x559b563cf410 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f10fe7fc640 1 -- 172.21.15.137:0/1557582746 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f110c09c040 con 0x559b563cf410 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f10fe7fc640 1 -- 172.21.15.137:0/1557582746 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c096bb0 con 0x559b563cf410 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 -- 172.21.15.137:0/1557582746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 msgr2=0x559b563f1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563f1920 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f110c098a10 tx=0x7f110c0a4e30 comp rx=0 tx=0).stop 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 -- 172.21.15.137:0/1557582746 shutdown_connections 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b563f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563cf410 0x559b563f1920 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 --2- 172.21.15.137:0/1557582746 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563ceaf0 0x559b563d73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 -- 172.21.15.137:0/1557582746 >> 172.21.15.137:0/1557582746 conn(0x559b563cb060 msgr2=0x559b563d8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 -- 172.21.15.137:0/1557582746 shutdown_connections 2024-09-12T12:30:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.988+0000 7f1121ca3740 1 -- 172.21.15.137:0/1557582746 wait complete. 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 -- start start 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 Processor -- start 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ddd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563de2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b564538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f11181abd50 con 0x559b563ceaf0 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f11181ad1c0 con 0x559b563cf410 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11181ac420 con 0x559b563d9cb0 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1121ca3740 10 client.0 _ll_register_callbacks cb 0x7f110c0983a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f111ccaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ddd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f111ccaf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ddd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58496/0 (socket says 172.21.15.137:58496) 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f111ccaf640 1 -- 172.21.15.137:0/924871923 learned_addr learned my addr 172.21.15.137:0/924871923 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f111ccaf640 1 -- 172.21.15.137:0/924871923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 msgr2=0x559b564538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f111ccaf640 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b564538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f1117fff640 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563de2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.989+0000 7f111ccaf640 1 -- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 msgr2=0x559b563de2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f111ccaf640 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563de2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f111ccaf640 1 -- 172.21.15.137:0/924871923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b563debc0 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f111ccaf640 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ddd80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1108000c00 tx=0x7f110800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1512879c9350bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f10f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f10f7fff640 1 -- 172.21.15.137:0/924871923 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11080052b0 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f10f7fff640 1 -- 172.21.15.137:0/924871923 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f11080035f0 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c09c040 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f10f7fff640 1 -- 172.21.15.137:0/924871923 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1108035050 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 10 client.17487 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x559b56454f40 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f10f7fff640 1 -- 172.21.15.137:0/924871923 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f110c09c040 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f10f7fff640 1 -- 172.21.15.137:0/924871923 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f11080107d0 con 0x559b563ceaf0 2024-09-12T12:30:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 10 client.17487 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 20 client.17487 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b56455490 0x559b564578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 4 client.17487 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.990+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b5633ad70 con 0x559b56455490 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.991+0000 7f1117fff640 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b56455490 0x559b564578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.991+0000 7f1117fff640 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b56455490 0x559b564578b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1840413be4885e server_cookie=950891af77094fa5 in_seq=0 out_seq=0 2024-09-12T12:30:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:05.991+0000 7f10f7fff640 10 client.17487 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f10f7fff640 1 -- 172.21.15.137:0/924871923 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b5633ad70 con 0x559b56455490 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f10f7fff640 10 client.17487 handle_command_reply: tid=0 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 client.17487 shutdown 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 msgr2=0x559b563ddd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ddd80 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f1108000c00 tx=0x7f110800c330 comp rx=0 tx=0).stop 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 shutdown_connections 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b563d9cb0 0x559b564538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559b563cf410 0x559b563de2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559b56455490 0x559b564578b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 --2- 172.21.15.137:0/924871923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559b563ceaf0 0x559b563ddd80 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f10f7fff640 0 client.17487 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 >> 172.21.15.137:0/924871923 conn(0x559b563cb060 msgr2=0x559b563c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 shutdown_connections 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 1 -- 172.21.15.137:0/924871923 wait complete. 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.114+0000 7f1121ca3740 20 client.17487 trim_cache size 0 max 16384 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:30:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:30:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f11000a2da0 msgr2=0x7f11000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f11000a2da0 0x7f11000a5260 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1110005ab0 tx=0x7f1110032f80 comp rx=0 tx=0).stop 2024-09-12T12:30:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 msgr2=0x7f11181987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f11181987a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f110c00ea80 tx=0x7f110c00ef40 comp rx=0 tx=0).stop 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 shutdown_connections 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f11000a2da0 0x7f11000a5260 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11181069c0 0x7f111819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1118101350 0x7f11181987a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 --2- 172.21.15.137:0/125960814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11181009b0 0x7f1118198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 >> 172.21.15.137:0/125960814 conn(0x7f1118077990 msgr2=0x7f11180ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 shutdown_connections 2024-09-12T12:30:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:06.115+0000 7f111ef13640 1 -- 172.21.15.137:0/125960814 wait complete. 2024-09-12T12:30:06.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 24.23 MB/sec execute 98 sec latency 8586.397 ms 2024-09-12T12:30:07.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:06 smithi171 ceph-mon[35098]: pgmap v535: 97 pgs: 97 active+clean; 903 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 489 op/s 2024-09-12T12:30:07.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 23.98 MB/sec execute 99 sec latency 9586.512 ms 2024-09-12T12:30:07.281 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:06 smithi189 ceph-mon[35475]: pgmap v535: 97 pgs: 97 active+clean; 903 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 489 op/s 2024-09-12T12:30:07.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:06 smithi137 ceph-mon[30950]: pgmap v535: 97 pgs: 97 active+clean; 903 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.8 MiB/s wr, 489 op/s 2024-09-12T12:30:08.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 23.74 MB/sec execute 100 sec latency 10586.552 ms 2024-09-12T12:30:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:08 smithi171 ceph-mon[35098]: pgmap v536: 97 pgs: 97 active+clean; 903 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.6 MiB/s wr, 334 op/s 2024-09-12T12:30:09.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:08 smithi171 ceph-mon[35098]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T12:30:09.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 23.51 MB/sec execute 101 sec latency 11586.597 ms 2024-09-12T12:30:09.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:08 smithi137 ceph-mon[30950]: pgmap v536: 97 pgs: 97 active+clean; 903 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.6 MiB/s wr, 334 op/s 2024-09-12T12:30:09.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:08 smithi137 ceph-mon[30950]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T12:30:09.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:08 smithi189 ceph-mon[35475]: pgmap v536: 97 pgs: 97 active+clean; 903 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.6 MiB/s wr, 334 op/s 2024-09-12T12:30:09.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:08 smithi189 ceph-mon[35475]: osdmap e112: 12 total, 12 up, 12 in 2024-09-12T12:30:10.225 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 23.28 MB/sec execute 102 sec latency 12586.645 ms 2024-09-12T12:30:11.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:10 smithi171 ceph-mon[35098]: pgmap v538: 97 pgs: 97 active+clean; 899 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 319 op/s 2024-09-12T12:30:11.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 23.05 MB/sec execute 103 sec latency 13586.735 ms 2024-09-12T12:30:11.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:10 smithi137 ceph-mon[30950]: pgmap v538: 97 pgs: 97 active+clean; 899 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 319 op/s 2024-09-12T12:30:11.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:10 smithi189 ceph-mon[35475]: pgmap v538: 97 pgs: 97 active+clean; 899 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.5 MiB/s wr, 319 op/s 2024-09-12T12:30:12.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 22.83 MB/sec execute 104 sec latency 14586.830 ms 2024-09-12T12:30:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:12 smithi171 ceph-mon[35098]: pgmap v539: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-12T12:30:13.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:12 smithi171 ceph-mon[35098]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T12:30:13.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 22.61 MB/sec execute 105 sec latency 15586.954 ms 2024-09-12T12:30:13.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:12 smithi137 ceph-mon[30950]: pgmap v539: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-12T12:30:13.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:12 smithi137 ceph-mon[30950]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T12:30:13.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:12 smithi189 ceph-mon[35475]: pgmap v539: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 2.7 MiB/s wr, 178 op/s 2024-09-12T12:30:13.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:12 smithi189 ceph-mon[35475]: osdmap e113: 12 total, 12 up, 12 in 2024-09-12T12:30:14.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 22.40 MB/sec execute 106 sec latency 16587.027 ms 2024-09-12T12:30:15.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:14 smithi171 ceph-mon[35098]: pgmap v541: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 75 op/s 2024-09-12T12:30:15.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 22.19 MB/sec execute 107 sec latency 17587.083 ms 2024-09-12T12:30:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:14 smithi137 ceph-mon[30950]: pgmap v541: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 75 op/s 2024-09-12T12:30:15.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:14 smithi189 ceph-mon[35475]: pgmap v541: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 90 KiB/s wr, 75 op/s 2024-09-12T12:30:16.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 21.99 MB/sec execute 108 sec latency 18587.170 ms 2024-09-12T12:30:17.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:16 smithi171 ceph-mon[35098]: pgmap v542: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 128 KiB/s wr, 82 op/s 2024-09-12T12:30:17.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 21.78 MB/sec execute 109 sec latency 19587.304 ms 2024-09-12T12:30:17.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:16 smithi137 ceph-mon[30950]: pgmap v542: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 128 KiB/s wr, 82 op/s 2024-09-12T12:30:17.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:16 smithi189 ceph-mon[35475]: pgmap v542: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 128 KiB/s wr, 82 op/s 2024-09-12T12:30:18.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 21.59 MB/sec execute 110 sec latency 20587.408 ms 2024-09-12T12:30:19.023 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:30:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:18 smithi171 ceph-mon[35098]: pgmap v543: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 106 KiB/s wr, 68 op/s 2024-09-12T12:30:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.206+0000 7fdecbb5f640 1 -- 172.21.15.137:0/1560514939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41009b0 msgr2=0x7fdec4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.206+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/1560514939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41009b0 0x7fdec4100e10 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009930 tx=0x7fdeb402a420 comp rx=0 tx=0).stop 2024-09-12T12:30:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 -- 172.21.15.137:0/1560514939 shutdown_connections 2024-09-12T12:30:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/1560514939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdec4101350 0x7fdec410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/1560514939 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41009b0 0x7fdec4100e10 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/1560514939 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec41069c0 0x7fdec4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 -- 172.21.15.137:0/1560514939 >> 172.21.15.137:0/1560514939 conn(0x7fdec4077990 msgr2=0x7fdec40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 -- 172.21.15.137:0/1560514939 shutdown_connections 2024-09-12T12:30:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 -- 172.21.15.137:0/1560514939 wait complete. 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 Processor -- start 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 -- start start 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdec41009b0 0x7fdec41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.207+0000 7fdecbb5f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec4101350 0x7fdec4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdecbb5f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41069c0 0x7fdec41a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdecbb5f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4110e00 con 0x7fdec41069c0 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdecbb5f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4110c80 con 0x7fdec4101350 2024-09-12T12:30:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdecbb5f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4110f80 con 0x7fdec41009b0 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41069c0 0x7fdec41a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41069c0 0x7fdec41a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59374/0 (socket says 172.21.15.137:59374) 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 -- 172.21.15.137:0/3191561888 learned_addr learned my addr 172.21.15.137:0/3191561888 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec4101350 0x7fdec4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdec41009b0 0x7fdec41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 -- 172.21.15.137:0/3191561888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdec41009b0 msgr2=0x7fdec41991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdec41009b0 0x7fdec41991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 -- 172.21.15.137:0/3191561888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec4101350 msgr2=0x7fdec4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec4101350 0x7fdec4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 -- 172.21.15.137:0/3191561888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec41a47b0 con 0x7fdec41069c0 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec4101350 0x7fdec4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdec41009b0 0x7fdec41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:30:19.210 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdeca0fc640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41069c0 0x7fdec41a4270 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009fd0 tx=0x7fdebc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdebaffd640 1 -- 172.21.15.137:0/3191561888 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc017070 con 0x7fdec41069c0 2024-09-12T12:30:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec41a4980 con 0x7fdec41069c0 2024-09-12T12:30:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdebaffd640 1 -- 172.21.15.137:0/3191561888 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdebc003e50 con 0x7fdec41069c0 2024-09-12T12:30:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdebaffd640 1 -- 172.21.15.137:0/3191561888 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0106c0 con 0x7fdec41069c0 2024-09-12T12:30:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.208+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec41a4ea0 con 0x7fdec41069c0 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.211+0000 7fdece9d4740 1 Processor -- start 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdebaffd640 1 -- 172.21.15.137:0/3191561888 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdebc003970 con 0x7fdec41069c0 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdece9d4740 1 -- start start 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938718ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938723770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x5649387260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdece9d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56493867d2f0 con 0x564938718af0 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdece9d4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564938625270 con 0x564938723cb0 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdece9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5649386792b0 con 0x564938719410 2024-09-12T12:30:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdec90fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938718ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdec90fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938718ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59390/0 (socket says 172.21.15.137:59390) 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdec90fa640 1 -- 172.21.15.137:0/2402775823 learned_addr learned my addr 172.21.15.137:0/2402775823 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdebaffd640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fdeb00a2da0 0x7fdeb00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x5649387260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.212+0000 7fdec98fb640 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938723770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdeca0fc640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fdeb00a2da0 0x7fdeb00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdebaffd640 1 -- 172.21.15.137:0/3191561888 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7fdebc0cf280 con 0x7fdec41069c0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdec90fa640 1 -- 172.21.15.137:0/2402775823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 msgr2=0x564938723770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938723770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdec90fa640 1 -- 172.21.15.137:0/2402775823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 msgr2=0x5649387260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x5649387260a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdec90fa640 1 -- 172.21.15.137:0/2402775823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564938624340 con 0x564938718af0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938718ed0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdeb402a940 tx=0x7fdeb4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b970fc986d3af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdeca0fc640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fdeb00a2da0 0x7fdeb00a5260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdebc097c20 tx=0x7fdebc06e830 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.215 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdeaffff640 1 -- 172.21.15.137:0/2402775823 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4035070 con 0x564938718af0 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdeaffff640 1 -- 172.21.15.137:0/2402775823 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb40317b0 con 0x564938718af0 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.213+0000 7fdeaffff640 1 -- 172.21.15.137:0/2402775823 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40046c0 con 0x564938718af0 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 -- 172.21.15.137:0/2402775823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 msgr2=0x564938718ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938718ed0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdeb402a940 tx=0x7fdeb4005b50 comp rx=0 tx=0).stop 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 -- 172.21.15.137:0/2402775823 shutdown_connections 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x5649387260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938723770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2402775823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938718ed0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 -- 172.21.15.137:0/2402775823 >> 172.21.15.137:0/2402775823 conn(0x564938715060 msgr2=0x564938715440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 -- 172.21.15.137:0/2402775823 shutdown_connections 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.214+0000 7fdece9d4740 1 -- 172.21.15.137:0/2402775823 wait complete. 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 -- start start 2024-09-12T12:30:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 Processor -- start 2024-09-12T12:30:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938720ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938721000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x564938721540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x564938726880 con 0x564938718af0 2024-09-12T12:30:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564938606290 con 0x564938723cb0 2024-09-12T12:30:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x564938726a00 con 0x564938719410 2024-09-12T12:30:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdece9d4740 10 client.0 _ll_register_callbacks cb 0x564938637140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdec90fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938720ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdec90fa640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938720ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59392/0 (socket says 172.21.15.137:59392) 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdec90fa640 1 -- 172.21.15.137:0/3411691352 learned_addr learned my addr 172.21.15.137:0/3411691352 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x564938721540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.215+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938721000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdec90fa640 1 -- 172.21.15.137:0/3411691352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 msgr2=0x564938721000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938721000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdec90fa640 1 -- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 msgr2=0x564938721540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x564938721540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdec90fa640 1 -- 172.21.15.137:0/3411691352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564938390210 con 0x564938718af0 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938720ac0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40096c0 tx=0x7fdeb40053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e275f734d39d0d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdead7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdead7fa640 1 -- 172.21.15.137:0/3411691352 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4035030 con 0x564938718af0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdead7fa640 1 -- 172.21.15.137:0/3411691352 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7fdeb4004260 con 0x564938718af0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb40317b0 con 0x564938718af0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdead7fa640 1 -- 172.21.15.137:0/3411691352 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdeb4059050 con 0x564938718af0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdece9d4740 10 client.17499 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56493845c620 con 0x564938718af0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdead7fa640 1 -- 172.21.15.137:0/3411691352 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb40317b0 con 0x564938718af0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdead7fa640 1 -- 172.21.15.137:0/3411691352 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7fdeb40035f0 con 0x564938718af0 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdece9d4740 10 client.17499 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdece9d4740 10 client.17499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdece9d4740 20 client.17499 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.216+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564938733a10 0x564938735e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.217+0000 7fdece9d4740 4 client.17499 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:30:19.222 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.217+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56493868aa20 con 0x564938733a10 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.217+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564938733a10 0x564938735e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.217+0000 7fdead7fa640 10 client.17499 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.217+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564938733a10 0x564938735e00 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=285f562d74a8de5d server_cookie=4db90d802967bd52 in_seq=0 out_seq=0 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdead7fa640 1 -- 172.21.15.137:0/3411691352 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56493868aa20 con 0x564938733a10 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdead7fa640 10 client.17499 handle_command_reply: tid=0 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 client.17499 shutdown 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 msgr2=0x564938720ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938720ac0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40096c0 tx=0x7fdeb40053e0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 shutdown_connections 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x564938721540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x564938721000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x564938733a10 0x564938735e00 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3411691352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938718af0 0x564938720ac0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdead7fa640 0 client.17499 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:19.223 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 >> 172.21.15.137:0/3411691352 conn(0x564938715060 msgr2=0x56493871adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 shutdown_connections 2024-09-12T12:30:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 1 -- 172.21.15.137:0/3411691352 wait complete. 2024-09-12T12:30:19.224 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.218+0000 7fdece9d4740 20 client.17499 trim_cache size 0 max 16384 2024-09-12T12:30:19.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 21.39 MB/sec execute 111 sec latency 21587.470 ms 2024-09-12T12:30:19.239 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 Processor -- start 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 -- start start 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938718af0 0x564938721370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x56493873b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938723cb0 0x56493873b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41afff0 con 0x564938723cb0 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41ad830 con 0x564938718af0 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdece9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41ab6a0 con 0x564938719410 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdec90fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938718af0 0x564938721370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdec90fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938718af0 0x564938721370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:40400/0 (socket says 172.21.15.137:40400) 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdec90fa640 1 -- 172.21.15.137:0/2816264108 learned_addr learned my addr 172.21.15.137:0/2816264108 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938723cb0 0x56493873b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.239+0000 7fdec98fb640 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x56493873b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdec90fa640 1 -- 172.21.15.137:0/2816264108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 msgr2=0x56493873b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x56493873b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdec90fa640 1 -- 172.21.15.137:0/2816264108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938723cb0 msgr2=0x56493873b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938723cb0 0x56493873b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdec90fa640 1 -- 172.21.15.137:0/2816264108 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564938714e90 con 0x564938718af0 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdec90fa640 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938718af0 0x564938721370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4032d90 tx=0x7fdeb4031b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f26dbd803bcceb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdeaf7fe640 1 -- 172.21.15.137:0/2816264108 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4035040 con 0x564938718af0 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdeaf7fe640 1 -- 172.21.15.137:0/2816264108 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4032e50 con 0x564938718af0 2024-09-12T12:30:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.240+0000 7fdeaf7fe640 1 -- 172.21.15.137:0/2816264108 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4004ba0 con 0x564938718af0 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 -- 172.21.15.137:0/2816264108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938718af0 msgr2=0x564938721370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938718af0 0x564938721370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4032d90 tx=0x7fdeb4031b20 comp rx=0 tx=0).stop 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 -- 172.21.15.137:0/2816264108 shutdown_connections 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938723cb0 0x56493873b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938719410 0x56493873b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 --2- 172.21.15.137:0/2816264108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938718af0 0x564938721370 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 -- 172.21.15.137:0/2816264108 >> 172.21.15.137:0/2816264108 conn(0x564938715060 msgr2=0x564938722b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 -- 172.21.15.137:0/2816264108 shutdown_connections 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 -- 172.21.15.137:0/2816264108 wait complete. 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 -- start start 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.241+0000 7fdece9d4740 1 Processor -- start 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938718af0 0x56493872c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938719410 0x56493871d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdece9d4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x56493871dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdece9d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41abcf0 con 0x564938719410 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdece9d4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41ad160 con 0x564938723cb0 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdece9d4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41ac3c0 con 0x564938718af0 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdece9d4740 10 client.0 _ll_register_callbacks cb 0x7fdec000b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938719410 0x56493871d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938719410 0x56493871d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59408/0 (socket says 172.21.15.137:59408) 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 -- 172.21.15.137:0/3531287737 learned_addr learned my addr 172.21.15.137:0/3531287737 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec90fa640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x56493871dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938718af0 0x56493872c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 -- 172.21.15.137:0/3531287737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938718af0 msgr2=0x56493872c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938718af0 0x56493872c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 -- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 msgr2=0x56493871dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x56493871dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec98fb640 1 -- 172.21.15.137:0/3531287737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56493871e300 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.242+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x56493871dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdec98fb640 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938719410 0x56493871d6d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b840 tx=0x7fdec0004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c1671859ac1a1b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdead7fa640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdead7fa640 1 -- 172.21.15.137:0/3531287737 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec001c510 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdead7fa640 1 -- 172.21.15.137:0/3531287737 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7fdec001c6b0 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb4032e50 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdead7fa640 1 -- 172.21.15.137:0/3531287737 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdec001c8b0 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 10 client.17505 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56493879e370 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdead7fa640 1 -- 172.21.15.137:0/3531287737 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb4032e50 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdead7fa640 1 -- 172.21.15.137:0/3531287737 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7fdec002c020 con 0x564938719410 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 10 client.17505 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 20 client.17505 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56493879eb30 0x5649387a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 4 client.17505 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:30:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.243+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564938684d70 con 0x56493879eb30 2024-09-12T12:30:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.244+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56493879eb30 0x5649387a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.244+0000 7fdead7fa640 10 client.17505 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.244+0000 7fdec90fa640 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56493879eb30 0x5649387a0f50 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c717035c5367070 server_cookie=59981cd6bcbbbda6 in_seq=0 out_seq=0 2024-09-12T12:30:19.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.280+0000 7fdead7fa640 1 -- 172.21.15.137:0/3531287737 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564938684d70 con 0x56493879eb30 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.280+0000 7fdead7fa640 10 client.17505 handle_command_reply: tid=0 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.280+0000 7fdece9d4740 1 client.17505 shutdown 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938719410 msgr2=0x56493871d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938719410 0x56493871d6d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b840 tx=0x7fdec0004560 comp rx=0 tx=0).stop 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 shutdown_connections 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564938723cb0 0x56493871dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56493879eb30 0x5649387a0f50 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564938719410 0x56493871d6d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 --2- 172.21.15.137:0/3531287737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564938718af0 0x56493872c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdead7fa640 0 client.17505 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 >> 172.21.15.137:0/3531287737 conn(0x564938715060 msgr2=0x56493872eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:19.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 shutdown_connections 2024-09-12T12:30:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 1 -- 172.21.15.137:0/3531287737 wait complete. 2024-09-12T12:30:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdece9d4740 20 client.17505 trim_cache size 0 max 16384 2024-09-12T12:30:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:30:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:30:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:30:19.282 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fdeb00a2da0 msgr2=0x7fdeb00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fdeb00a2da0 0x7fdeb00a5260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdebc097c20 tx=0x7fdebc06e830 comp rx=0 tx=0).stop 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41069c0 msgr2=0x7fdec41a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41069c0 0x7fdec41a4270 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009fd0 tx=0x7fdebc00b260 comp rx=0 tx=0).stop 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 shutdown_connections 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fdeb00a2da0 0x7fdeb00a5260 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdec41069c0 0x7fdec41a4270 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec4101350 0x7fdec4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 --2- 172.21.15.137:0/3191561888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdec41009b0 0x7fdec41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.281+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 >> 172.21.15.137:0/3191561888 conn(0x7fdec4077990 msgr2=0x7fdec4103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.282+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 shutdown_connections 2024-09-12T12:30:19.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:19.282+0000 7fdecbb5f640 1 -- 172.21.15.137:0/3191561888 wait complete. 2024-09-12T12:30:19.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:18 smithi137 ceph-mon[30950]: pgmap v543: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 106 KiB/s wr, 68 op/s 2024-09-12T12:30:19.301 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:30:19.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:18 smithi189 ceph-mon[35475]: pgmap v543: 97 pgs: 97 active+clean; 875 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 106 KiB/s wr, 68 op/s 2024-09-12T12:30:19.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 -- 172.21.15.137:0/4256290501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 msgr2=0x7fcb2c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/4256290501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10da40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009900 tx=0x7fcb1402a480 comp rx=0 tx=0).stop 2024-09-12T12:30:19.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 -- 172.21.15.137:0/4256290501 shutdown_connections 2024-09-12T12:30:19.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/4256290501 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10da40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/4256290501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2c102570 0x7fcb2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/4256290501 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb2c100e20 0x7fcb2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 -- 172.21.15.137:0/4256290501 >> 172.21.15.137:0/4256290501 conn(0x7fcb2c0fc7d0 msgr2=0x7fcb2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 -- 172.21.15.137:0/4256290501 shutdown_connections 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.792+0000 7fcb32b1b640 1 -- 172.21.15.137:0/4256290501 wait complete. 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 Processor -- start 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 -- start start 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb2c100e20 0x7fcb2c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2c102570 0x7fcb2c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1120a0 con 0x7fcb2c102ef0 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c111f20 con 0x7fcb2c100e20 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb32b1b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c112220 con 0x7fcb2c102570 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59430/0 (socket says 172.21.15.137:59430) 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 -- 172.21.15.137:0/2259380122 learned_addr learned my addr 172.21.15.137:0/2259380122 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 -- 172.21.15.137:0/2259380122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2c102570 msgr2=0x7fcb2c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2c102570 0x7fcb2c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 -- 172.21.15.137:0/2259380122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb2c100e20 msgr2=0x7fcb2c10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb2c100e20 0x7fcb2c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 -- 172.21.15.137:0/2259380122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c1a68f0 con 0x7fcb2c102ef0 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb310b8640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10ef50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005ab0 tx=0x7fcb14003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.793+0000 7fcb29ffb640 1 -- 172.21.15.137:0/2259380122 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1403f070 con 0x7fcb2c102ef0 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.794+0000 7fcb29ffb640 1 -- 172.21.15.137:0/2259380122 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb14031e30 con 0x7fcb2c102ef0 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.794+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c1a6b80 con 0x7fcb2c102ef0 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.794+0000 7fcb29ffb640 1 -- 172.21.15.137:0/2259380122 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14035030 con 0x7fcb2c102ef0 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.794+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a7060 con 0x7fcb2c102ef0 2024-09-12T12:30:19.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.795+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c101200 con 0x7fcb2c102ef0 2024-09-12T12:30:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.796+0000 7fcb29ffb640 1 -- 172.21.15.137:0/2259380122 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcb14042070 con 0x7fcb2c102ef0 2024-09-12T12:30:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.797+0000 7fcb29ffb640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcb180a2cf0 0x7fcb180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.797+0000 7fcb308b7640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcb180a2cf0 0x7fcb180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.797+0000 7fcb29ffb640 1 -- 172.21.15.137:0/2259380122 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7fcb140f04f0 con 0x7fcb2c102ef0 2024-09-12T12:30:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.797+0000 7fcb308b7640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcb180a2cf0 0x7fcb180a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400a990 tx=0x7fcb24005c70 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:19.800+0000 7fcb29ffb640 1 -- 172.21.15.137:0/2259380122 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb140bd130 con 0x7fcb2c102ef0 2024-09-12T12:30:20.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.001+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb2c1a7330 con 0x7fcb2c102ef0 2024-09-12T12:30:20.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.002+0000 7fcb29ffb640 1 -- 172.21.15.137:0/2259380122 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11940 (secure 0 0 0) 0x7fcb140bca30 con 0x7fcb2c102ef0 2024-09-12T12:30:20.002 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:30:20.002 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":50,"btime":"2024-09-12T12:30:03:892861+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:30:03.181716+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26647":{"gid":26647,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/3747343886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3747343886},{"type":"v1","addr":"172.21.15.189:6835","nonce":3747343886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:30:20.002 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 50 2024-09-12T12:30:20.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcb180a2cf0 msgr2=0x7fcb180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:20.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcb180a2cf0 0x7fcb180a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400a990 tx=0x7fcb24005c70 comp rx=0 tx=0).stop 2024-09-12T12:30:20.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 msgr2=0x7fcb2c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:20.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10ef50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005ab0 tx=0x7fcb14003c30 comp rx=0 tx=0).stop 2024-09-12T12:30:20.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 shutdown_connections 2024-09-12T12:30:20.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fcb180a2cf0 0x7fcb180a51b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:20.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb2c102ef0 0x7fcb2c10ef50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:20.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb2c102570 0x7fcb2c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:20.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 --2- 172.21.15.137:0/2259380122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb2c100e20 0x7fcb2c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:20.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 >> 172.21.15.137:0/2259380122 conn(0x7fcb2c0fc7d0 msgr2=0x7fcb2c106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:20.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 shutdown_connections 2024-09-12T12:30:20.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:20.004+0000 7fcb32b1b640 1 -- 172.21.15.137:0/2259380122 wait complete. 2024-09-12T12:30:20.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 21.20 MB/sec execute 112 sec latency 22587.551 ms 2024-09-12T12:30:20.803 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id c66e0300 --release --await' 2024-09-12T12:30:20.803 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id c66e0300 --release --await 2024-09-12T12:30:21.052 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:20 smithi137 ceph-mon[30950]: pgmap v544: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 53 op/s 2024-09-12T12:30:21.052 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:20 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2259380122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:30:21.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:20 smithi171 ceph-mon[35098]: pgmap v544: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 53 op/s 2024-09-12T12:30:21.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:20 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2259380122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:30:21.210 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:30:21.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 75742 21.01 MB/sec execute 113 sec latency 23587.605 ms 2024-09-12T12:30:21.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:20 smithi189 ceph-mon[35475]: pgmap v544: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 87 KiB/s wr, 53 op/s 2024-09-12T12:30:21.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:20 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2259380122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:30:21.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/3649043062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 msgr2=0x7f8b481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/3649043062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b481021c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009900 tx=0x7f8b3802a410 comp rx=0 tx=0).stop 2024-09-12T12:30:21.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/3649043062 shutdown_connections 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/3649043062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b48106f10 0x7f8b4810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/3649043062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b48102790 0x7f8b481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/3649043062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b481021c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/3649043062 >> 172.21.15.137:0/3649043062 conn(0x7f8b480fd8c0 msgr2=0x7f8b480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/3649043062 shutdown_connections 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.699+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/3649043062 wait complete. 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.700+0000 7f8b4e3b4640 1 Processor -- start 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.700+0000 7f8b4e3b4640 1 -- start start 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.700+0000 7f8b4e3b4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b48198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.700+0000 7f8b4e3b4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b48102790 0x7f8b48198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.700+0000 7f8b4e3b4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b48106f10 0x7f8b4819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.700+0000 7f8b4e3b4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48111f90 con 0x7f8b48101de0 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b4e3b4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48111e10 con 0x7f8b48102790 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b4e3b4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48112110 con 0x7f8b48106f10 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b48198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b48198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59446/0 (socket says 172.21.15.137:59446) 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 -- 172.21.15.137:0/2120825002 learned_addr learned my addr 172.21.15.137:0/2120825002 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b477fe640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b48102790 0x7f8b48198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b48106f10 0x7f8b4819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 -- 172.21.15.137:0/2120825002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b48106f10 msgr2=0x7f8b4819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b48106f10 0x7f8b4819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 -- 172.21.15.137:0/2120825002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b48102790 msgr2=0x7f8b48198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b48102790 0x7f8b48198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 -- 172.21.15.137:0/2120825002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4819d4c0 con 0x7f8b48101de0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b48198510 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006010 tx=0x7f8b380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b457fa640 1 -- 172.21.15.137:0/2120825002 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3803f030 con 0x7f8b48101de0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4819d750 con 0x7f8b48101de0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b457fa640 1 -- 172.21.15.137:0/2120825002 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b38003ce0 con 0x7f8b48101de0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b457fa640 1 -- 172.21.15.137:0/2120825002 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38035070 con 0x7f8b48101de0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.701+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b481a5090 con 0x7f8b48101de0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b457fa640 1 -- 172.21.15.137:0/2120825002 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8b38042030 con 0x7f8b48101de0 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 Processor -- start 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 -- start start 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a6015fd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a601607d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a60160a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a601562620 con 0x55a601608240 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a601509e50 con 0x55a6015fd980 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b5078d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a60155e5e0 con 0x55a6015fd060 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b457fa640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8b300a2cf0 0x7f8b300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a60160a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a60160a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59448/0 (socket says 172.21.15.137:59448) 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.704+0000 7f8b477fe640 1 -- 172.21.15.137:0/2241568522 learned_addr learned my addr 172.21.15.137:0/2241568522 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a6015fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8b300a2cf0 0x7f8b300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a601607d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b457fa640 1 -- 172.21.15.137:0/2120825002 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7f8b380ef910 con 0x7f8b48101de0 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 -- 172.21.15.137:0/2241568522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 msgr2=0x55a601607d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a601607d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 -- 172.21.15.137:0/2241568522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 msgr2=0x55a60160a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a60160a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 -- 172.21.15.137:0/2241568522 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6014fad40 con 0x55a6015fd060 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b477fe640 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a60160a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8b300a2cf0 0x7f8b300a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009900 tx=0x7f8b4001b000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a6015fd440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bf80 tx=0x7f8b40008750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a3d206efd44dbb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b2e7fc640 1 -- 172.21.15.137:0/2241568522 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40027070 con 0x55a6015fd060 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b2e7fc640 1 -- 172.21.15.137:0/2241568522 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b40009d90 con 0x55a6015fd060 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.705+0000 7f8b2e7fc640 1 -- 172.21.15.137:0/2241568522 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4001e070 con 0x55a6015fd060 2024-09-12T12:30:21.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 -- 172.21.15.137:0/2241568522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 msgr2=0x55a6015fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a6015fd440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bf80 tx=0x7f8b40008750 comp rx=0 tx=0).stop 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 -- 172.21.15.137:0/2241568522 shutdown_connections 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a60160a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a601607d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2241568522 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a6015fd440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 -- 172.21.15.137:0/2241568522 >> 172.21.15.137:0/2241568522 conn(0x55a6015f95f0 msgr2=0x55a6015f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 -- 172.21.15.137:0/2241568522 shutdown_connections 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 -- 172.21.15.137:0/2241568522 wait complete. 2024-09-12T12:30:21.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.706+0000 7f8b5078d740 1 -- start start 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 1 Processor -- start 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd060 0x55a601605140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd980 0x55a601605680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601605bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a60160ae10 con 0x55a601608240 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a6014eaf10 con 0x55a6015fd060 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a60160af90 con 0x55a6015fd980 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b5078d740 10 client.0 _ll_register_callbacks cb 0x55a60150edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd980 0x55a601605680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b4c951640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601605bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd980 0x55a601605680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:43054/0 (socket says 172.21.15.137:43054) 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b477fe640 1 -- 172.21.15.137:0/2096794201 learned_addr learned my addr 172.21.15.137:0/2096794201 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:21.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd060 0x55a601605140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b4c951640 1 -- 172.21.15.137:0/2096794201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd980 msgr2=0x55a601605680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd980 0x55a601605680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b4c951640 1 -- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd060 msgr2=0x55a601605140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd060 0x55a601605140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.707+0000 7f8b4c951640 1 -- 172.21.15.137:0/2096794201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a601271580 con 0x55a601608240 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601605bc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009690 tx=0x7f8b4000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94059a13a7afc85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b277fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b277fe640 1 -- 172.21.15.137:0/2096794201 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40027070 con 0x55a601608240 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b277fe640 1 -- 172.21.15.137:0/2096794201 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7f8b40022610 con 0x55a601608240 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b40009d90 con 0x55a601608240 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b277fe640 1 -- 172.21.15.137:0/2096794201 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8b4001e040 con 0x55a601608240 2024-09-12T12:30:21.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 10 client.17523 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a60133b960 con 0x55a601608240 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b277fe640 1 -- 172.21.15.137:0/2096794201 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b40009d90 con 0x55a601608240 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b277fe640 1 -- 172.21.15.137:0/2096794201 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f8b4002da20 con 0x55a601608240 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 10 client.17523 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 10 client.17523 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 20 client.17523 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a601617d10 0x55a60161a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 4 client.17523 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.708+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a60156fb90 con 0x55a601617d10 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.709+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a601617d10 0x55a60161a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.709+0000 7f8b277fe640 10 client.17523 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:21.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.709+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a601617d10 0x55a60161a100 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c859e879ef6696 server_cookie=2823e6087917846e in_seq=0 out_seq=0 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b277fe640 1 -- 172.21.15.137:0/2096794201 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a60156fb90 con 0x55a601617d10 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b277fe640 10 client.17523 handle_command_reply: tid=0 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 client.17523 shutdown 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 msgr2=0x55a601605bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601605bc0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009690 tx=0x7f8b4000b680 comp rx=0 tx=0).stop 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 shutdown_connections 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a601617d10 0x55a60161a100 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601605bc0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd980 0x55a601605680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2096794201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd060 0x55a601605140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b277fe640 0 client.17523 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 >> 172.21.15.137:0/2096794201 conn(0x55a6015f95f0 msgr2=0x55a6015ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:21.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 shutdown_connections 2024-09-12T12:30:21.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 1 -- 172.21.15.137:0/2096794201 wait complete. 2024-09-12T12:30:21.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.710+0000 7f8b5078d740 20 client.17523 trim_cache size 0 max 16384 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.727+0000 7f8b5078d740 1 Processor -- start 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.727+0000 7f8b5078d740 1 -- start start 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.727+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a601607010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.727+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6015fd980 0x55a6015ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a601608240 0x55a60161f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b5078d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8b481b0160 con 0x55a6015fd980 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b5078d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8b481ad9a0 con 0x55a601608240 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b5078d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b481ab810 con 0x55a6015fd060 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6015fd980 0x55a6015ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6015fd980 0x55a6015ffb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59466/0 (socket says 172.21.15.137:59466) 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 -- 172.21.15.137:0/2721353650 learned_addr learned my addr 172.21.15.137:0/2721353650 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a601607010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a601608240 0x55a60161f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 -- 172.21.15.137:0/2721353650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 msgr2=0x55a601607010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a601607010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 -- 172.21.15.137:0/2721353650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a601608240 msgr2=0x55a60161f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a601608240 0x55a60161f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 -- 172.21.15.137:0/2721353650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6015fa6a0 con 0x55a6015fd980 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b47fff640 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a601607010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b4c951640 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a601608240 0x55a60161f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b477fe640 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6015fd980 0x55a6015ffb10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00bc60 tx=0x7f8b3c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651927891884877 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:21.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b2dffb640 1 -- 172.21.15.137:0/2721353650 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00be10 con 0x55a6015fd980 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b2dffb640 1 -- 172.21.15.137:0/2721353650 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3c005020 con 0x55a6015fd980 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.728+0000 7f8b2dffb640 1 -- 172.21.15.137:0/2721353650 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004bc0 con 0x55a6015fd980 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 -- 172.21.15.137:0/2721353650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6015fd980 msgr2=0x55a6015ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6015fd980 0x55a6015ffb10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00bc60 tx=0x7f8b3c011810 comp rx=0 tx=0).stop 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 -- 172.21.15.137:0/2721353650 shutdown_connections 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a601608240 0x55a60161f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a6015fd980 0x55a6015ffb10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 --2- 172.21.15.137:0/2721353650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a601607010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 -- 172.21.15.137:0/2721353650 >> 172.21.15.137:0/2721353650 conn(0x55a6015f95f0 msgr2=0x55a6015f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 -- 172.21.15.137:0/2721353650 shutdown_connections 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 -- 172.21.15.137:0/2721353650 wait complete. 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.729+0000 7f8b5078d740 1 -- start start 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 1 Processor -- start 2024-09-12T12:30:21.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a60167e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a6016101e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601610720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8b481abe60 con 0x55a601608240 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8b481ad2d0 con 0x55a6015fd980 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b481ac530 con 0x55a6015fd060 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b5078d740 10 client.0 _ll_register_callbacks cb 0x7f8b4000bd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601610720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601610720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59478/0 (socket says 172.21.15.137:59478) 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 -- 172.21.15.137:0/4206547800 learned_addr learned my addr 172.21.15.137:0/4206547800 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:21.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b47fff640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a60167e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b477fe640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a6016101e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 -- 172.21.15.137:0/4206547800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 msgr2=0x55a60167e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a60167e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 -- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 msgr2=0x55a6016101e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a6016101e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b4c951640 1 -- 172.21.15.137:0/4206547800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a60167b8e0 con 0x55a601608240 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b47fff640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a60167e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.730+0000 7f8b477fe640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a6016101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b4c951640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601610720 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bfd0 tx=0x7f8b40036740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec84b1d2457996 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b277fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b277fe640 1 -- 172.21.15.137:0/4206547800 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40042020 con 0x55a601608240 2024-09-12T12:30:21.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b277fe640 1 -- 172.21.15.137:0/4206547800 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7f8b4000b240 con 0x55a601608240 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b3c005020 con 0x55a601608240 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b277fe640 1 -- 172.21.15.137:0/4206547800 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f8b40027050 con 0x55a601608240 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 10 client.17529 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a60168b6d0 con 0x55a601608240 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b277fe640 1 -- 172.21.15.137:0/4206547800 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3c005020 con 0x55a601608240 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b277fe640 1 -- 172.21.15.137:0/4206547800 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7f8b4003d030 con 0x55a601608240 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 10 client.17529 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 10 client.17529 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 20 client.17529 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a60168bed0 0x55a60168e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 4 client.17529 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c66e0300", "release": true, "await": true}] 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.731+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c66e0300", "release": true, "await": true}) -- 0x55a60156a0a0 con 0x55a60168bed0 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.732+0000 7f8b47fff640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a60168bed0 0x55a60168e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:21.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.732+0000 7f8b47fff640 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a60168bed0 0x55a60168e2f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=465afdd7e633f58e server_cookie=441b6619b52bdc54 in_seq=0 out_seq=0 2024-09-12T12:30:21.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.732+0000 7f8b277fe640 10 client.17529 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.774+0000 7f8b277fe640 1 -- 172.21.15.137:0/4206547800 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a60156a0a0 con 0x55a60168bed0 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.774+0000 7f8b277fe640 10 client.17529 handle_command_reply: tid=0 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.774+0000 7f8b5078d740 1 client.17529 shutdown 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 msgr2=0x55a601610720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601610720 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000bfd0 tx=0x7f8b40036740 comp rx=0 tx=0).stop 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 shutdown_connections 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a60168bed0 0x55a60168e2f0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a601608240 0x55a601610720 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a6015fd980 0x55a6016101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b277fe640 0 client.17529 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 --2- 172.21.15.137:0/4206547800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a6015fd060 0x55a60167e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 >> 172.21.15.137:0/4206547800 conn(0x55a6015f95f0 msgr2=0x55a6015f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 shutdown_connections 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 1 -- 172.21.15.137:0/4206547800 wait complete. 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.775+0000 7f8b5078d740 20 client.17529 trim_cache size 0 max 16384 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:30:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 50, 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 64, 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "c66e0300": { 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 64, 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:30:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 144.2, 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:30:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:30:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:30:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:30:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:30:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:30:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:30:21.780 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:30:21.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8b300a2cf0 msgr2=0x7f8b300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8b300a2cf0 0x7f8b300a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009900 tx=0x7f8b4001b000 comp rx=0 tx=0).stop 2024-09-12T12:30:21.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 msgr2=0x7f8b48198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:21.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b48198510 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8b38006010 tx=0x7f8b380042f0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 shutdown_connections 2024-09-12T12:30:21.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8b300a2cf0 0x7f8b300a51b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b48106f10 0x7f8b4819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b48102790 0x7f8b48198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 --2- 172.21.15.137:0/2120825002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b48101de0 0x7f8b48198510 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:21.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 >> 172.21.15.137:0/2120825002 conn(0x7f8b480fd8c0 msgr2=0x7f8b480fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:21.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 shutdown_connections 2024-09-12T12:30:21.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:30:21.776+0000 7f8b4e3b4640 1 -- 172.21.15.137:0/2120825002 wait complete. 2024-09-12T12:30:22.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 76006 20.98 MB/sec execute 114 sec latency 24147.209 ms 2024-09-12T12:30:22.593 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c66e0300', total seconds elapsed: 27.9 2024-09-12T12:30:23.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:22 smithi171 ceph-mon[35098]: pgmap v545: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-12T12:30:23.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 76891 21.20 MB/sec execute 115 sec latency 58.136 ms 2024-09-12T12:30:23.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:22 smithi137 ceph-mon[30950]: pgmap v545: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-12T12:30:23.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:22 smithi189 ceph-mon[35475]: pgmap v545: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 58 KiB/s wr, 15 op/s 2024-09-12T12:30:24.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 77579 21.35 MB/sec execute 116 sec latency 52.161 ms 2024-09-12T12:30:25.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:24 smithi171 ceph-mon[35098]: pgmap v546: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-09-12T12:30:25.226 INFO:tasks.workunit.client.0.smithi137.stdout: 10 78407 21.60 MB/sec execute 117 sec latency 63.298 ms 2024-09-12T12:30:25.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:24 smithi137 ceph-mon[30950]: pgmap v546: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-09-12T12:30:25.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:24 smithi189 ceph-mon[35475]: pgmap v546: 97 pgs: 97 active+clean; 875 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 14 op/s 2024-09-12T12:30:26.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 79139 21.73 MB/sec execute 118 sec latency 48.633 ms 2024-09-12T12:30:27.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:26 smithi171 ceph-mon[35098]: pgmap v547: 97 pgs: 97 active+clean; 889 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2024-09-12T12:30:27.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 79852 21.88 MB/sec execute 119 sec latency 87.319 ms 2024-09-12T12:30:27.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:26 smithi137 ceph-mon[30950]: pgmap v547: 97 pgs: 97 active+clean; 889 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2024-09-12T12:30:27.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:26 smithi189 ceph-mon[35475]: pgmap v547: 97 pgs: 97 active+clean; 889 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 235 op/s 2024-09-12T12:30:28.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 80490 21.95 MB/sec execute 120 sec latency 843.608 ms 2024-09-12T12:30:29.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:28 smithi171 ceph-mon[35098]: pgmap v548: 97 pgs: 97 active+clean; 889 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 230 op/s 2024-09-12T12:30:29.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 81282 22.05 MB/sec execute 121 sec latency 1843.637 ms 2024-09-12T12:30:29.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:28 smithi137 ceph-mon[30950]: pgmap v548: 97 pgs: 97 active+clean; 889 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 230 op/s 2024-09-12T12:30:29.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:28 smithi189 ceph-mon[35475]: pgmap v548: 97 pgs: 97 active+clean; 889 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.7 MiB/s wr, 230 op/s 2024-09-12T12:30:30.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 82064 22.31 MB/sec execute 122 sec latency 2843.664 ms 2024-09-12T12:30:31.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:30 smithi171 ceph-mon[35098]: pgmap v549: 97 pgs: 97 active+clean; 895 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-09-12T12:30:31.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 82476 22.33 MB/sec execute 123 sec latency 3843.691 ms 2024-09-12T12:30:31.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:30 smithi137 ceph-mon[30950]: pgmap v549: 97 pgs: 97 active+clean; 895 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-09-12T12:30:31.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:30 smithi189 ceph-mon[35475]: pgmap v549: 97 pgs: 97 active+clean; 895 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-09-12T12:30:32.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 82807 22.25 MB/sec execute 124 sec latency 4843.719 ms 2024-09-12T12:30:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 -- 172.21.15.137:0/96407610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 msgr2=0x7fb40810ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/96407610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb40810ea40 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb400009ca0 tx=0x7fb40002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 -- 172.21.15.137:0/96407610 shutdown_connections 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/96407610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb40810ea40 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/96407610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4080fff60 0x7fb4081003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/96407610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb408103aa0 0x7fb408103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 -- 172.21.15.137:0/96407610 >> 172.21.15.137:0/96407610 conn(0x7fb4080770f0 msgr2=0x7fb4080fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 -- 172.21.15.137:0/96407610 shutdown_connections 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.377+0000 7fb40d0a5640 1 -- 172.21.15.137:0/96407610 wait complete. 2024-09-12T12:30:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.378+0000 7fb40d0a5640 1 Processor -- start 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.378+0000 7fb40d0a5640 1 -- start start 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.378+0000 7fb40d0a5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4080fff60 0x7fb4081984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.378+0000 7fb40d0a5640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb408198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40d0a5640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb408103aa0 0x7fb40819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40d0a5640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081120a0 con 0x7fb4081008e0 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40d0a5640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb408111f20 con 0x7fb408103aa0 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40d0a5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb408112220 con 0x7fb4080fff60 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb408198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb408198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59816/0 (socket says 172.21.15.137:59816) 2024-09-12T12:30:32.381 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb406d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4080fff60 0x7fb4081984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40759e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb408103aa0 0x7fb40819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 -- 172.21.15.137:0/1540899932 learned_addr learned my addr 172.21.15.137:0/1540899932 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 -- 172.21.15.137:0/1540899932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4080fff60 msgr2=0x7fb4081984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4080fff60 0x7fb4081984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 -- 172.21.15.137:0/1540899932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb408103aa0 msgr2=0x7fb40819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb408103aa0 0x7fb40819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 -- 172.21.15.137:0/1540899932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40819d4e0 con 0x7fb4081008e0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40659c640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb408198a30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00e9f0 tx=0x7fb3fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb3effff640 1 -- 172.21.15.137:0/1540899932 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc00cd80 con 0x7fb4081008e0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40819d7d0 con 0x7fb4081008e0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb3effff640 1 -- 172.21.15.137:0/1540899932 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3fc010910 con 0x7fb4081008e0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb3effff640 1 -- 172.21.15.137:0/1540899932 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0188f0 con 0x7fb4081008e0 2024-09-12T12:30:32.382 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.379+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4081a5060 con 0x7fb4081008e0 2024-09-12T12:30:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.382+0000 7fb3effff640 1 -- 172.21.15.137:0/1540899932 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb3fc010ac0 con 0x7fb4081008e0 2024-09-12T12:30:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.382+0000 7fb40fe43740 1 Processor -- start 2024-09-12T12:30:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.382+0000 7fb40fe43740 1 -- start start 2024-09-12T12:30:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a18ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a23770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a23cb0 0x560b55a260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40fe43740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560b5597d2f0 con 0x560b55a18af0 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40fe43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560b55925270 con 0x560b55a23cb0 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40fe43740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b559792b0 con 0x560b55a19410 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a23770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a18ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a23770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:48270/0 (socket says 172.21.15.137:48270) 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 -- 172.21.15.137:0/12670909 learned_addr learned my addr 172.21.15.137:0/12670909 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40659c640 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a23cb0 0x560b55a260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 -- 172.21.15.137:0/12670909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a23cb0 msgr2=0x560b55a260a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a23cb0 0x560b55a260a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 -- 172.21.15.137:0/12670909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 msgr2=0x560b55a18ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a18ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 -- 172.21.15.137:0/12670909 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b55924340 con 0x560b55a19410 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb3effff640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb3f00a2da0 0x7fb3f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb40659c640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb3f00a2da0 0x7fb3f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb406d9d640 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a23770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009ca0 tx=0x7fb3f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=677c9c052ddd5ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb3effff640 1 -- 172.21.15.137:0/1540899932 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7fb3fc0cf7e0 con 0x7fb4081008e0 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.383+0000 7fb3ed7fa640 1 -- 172.21.15.137:0/12670909 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f801e070 con 0x560b55a19410 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb3ed7fa640 1 -- 172.21.15.137:0/12670909 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f8019b10 con 0x560b55a19410 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb3ed7fa640 1 -- 172.21.15.137:0/12670909 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800ea60 con 0x560b55a19410 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 -- 172.21.15.137:0/12670909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 msgr2=0x560b55a23770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40659c640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb3f00a2da0 0x7fb3f00a5260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0cd000 tx=0x7fb3fc0d5550 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a23770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009ca0 tx=0x7fb3f8011810 comp rx=0 tx=0).stop 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 -- 172.21.15.137:0/12670909 shutdown_connections 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a23cb0 0x560b55a260a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a23770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 --2- 172.21.15.137:0/12670909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a18ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 -- 172.21.15.137:0/12670909 >> 172.21.15.137:0/12670909 conn(0x560b55a15060 msgr2=0x560b55a15440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 -- 172.21.15.137:0/12670909 shutdown_connections 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.384+0000 7fb40fe43740 1 -- 172.21.15.137:0/12670909 wait complete. 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 -- start start 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 Processor -- start 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a20b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a21070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560b55a26880 con 0x560b55a18af0 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560b55906290 con 0x560b55a19410 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560b55a26a00 con 0x560b55a23cb0 2024-09-12T12:30:32.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40fe43740 10 client.0 _ll_register_callbacks cb 0x560b55937140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a20b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a20b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59836/0 (socket says 172.21.15.137:59836) 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 -- 172.21.15.137:0/3268134299 learned_addr learned my addr 172.21.15.137:0/3268134299 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb406d9d640 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a21070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 -- 172.21.15.137:0/3268134299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 msgr2=0x560b55a215b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a215b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 -- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 msgr2=0x560b55a21070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.385+0000 7fb40759e640 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a21070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40759e640 1 -- 172.21.15.137:0/3268134299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b55690210 con 0x560b55a18af0 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40759e640 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a20b30 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb400009430 tx=0x7fb4000093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae6d50fc5c2df00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb3eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb3eaff5640 1 -- 172.21.15.137:0/3268134299 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400030c60 con 0x560b55a18af0 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb3eaff5640 1 -- 172.21.15.137:0/3268134299 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7fb400030e00 con 0x560b55a18af0 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f8019b10 con 0x560b55a18af0 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb3eaff5640 1 -- 172.21.15.137:0/3268134299 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb400004b40 con 0x560b55a18af0 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 10 client.17541 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560b5575c620 con 0x560b55a18af0 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb3eaff5640 1 -- 172.21.15.137:0/3268134299 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f8019b10 con 0x560b55a18af0 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb3eaff5640 1 -- 172.21.15.137:0/3268134299 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7fb40004e020 con 0x560b55a18af0 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 10 client.17541 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 20 client.17541 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a33bc0 0x560b55a35ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 4 client.17541 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:30:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.386+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b5598aa20 con 0x560b55a33bc0 2024-09-12T12:30:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.387+0000 7fb406d9d640 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a33bc0 0x560b55a35ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.387+0000 7fb406d9d640 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a33bc0 0x560b55a35ff0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1578c28504350c server_cookie=e4f2cf016e2157ea in_seq=0 out_seq=0 2024-09-12T12:30:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.387+0000 7fb3eaff5640 10 client.17541 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb3eaff5640 1 -- 172.21.15.137:0/3268134299 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b5598aa20 con 0x560b55a33bc0 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb3eaff5640 10 client.17541 handle_command_reply: tid=0 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 client.17541 shutdown 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 msgr2=0x560b55a20b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a20b30 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb400009430 tx=0x7fb4000093a0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 shutdown_connections 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a215b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a21070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a33bc0 0x560b55a35ff0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3268134299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a20b30 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb3eaff5640 0 client.17541 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 >> 172.21.15.137:0/3268134299 conn(0x560b55a15060 msgr2=0x560b55a1adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 shutdown_connections 2024-09-12T12:30:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 1 -- 172.21.15.137:0/3268134299 wait complete. 2024-09-12T12:30:32.396 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.390+0000 7fb40fe43740 20 client.17541 trim_cache size 0 max 16384 2024-09-12T12:30:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 Processor -- start 2024-09-12T12:30:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 -- start start 2024-09-12T12:30:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a1b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.407 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a22aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081adea0 con 0x560b55a18af0 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081ab6e0 con 0x560b55a19410 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40fe43740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081a9550 con 0x560b55a23cb0 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a1b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb406d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40759e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a1b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59844/0 (socket says 172.21.15.137:59844) 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb406d9d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a213e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:41936/0 (socket says 172.21.15.137:41936) 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40759e640 1 -- 172.21.15.137:0/3635088098 learned_addr learned my addr 172.21.15.137:0/3635088098 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40759e640 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a22aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb406d9d640 1 -- 172.21.15.137:0/3635088098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 msgr2=0x560b55a22aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.408 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb406d9d640 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a22aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb406d9d640 1 -- 172.21.15.137:0/3635088098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 msgr2=0x560b55a1b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb406d9d640 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a1b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb406d9d640 1 -- 172.21.15.137:0/3635088098 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b55a14e90 con 0x560b55a19410 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.406+0000 7fb40759e640 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a1b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.407+0000 7fb406d9d640 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a213e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8011d10 tx=0x7fb3f8005220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1867868307c2458d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.407+0000 7fb3ecff9640 1 -- 172.21.15.137:0/3635088098 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f801e040 con 0x560b55a19410 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.407+0000 7fb3ecff9640 1 -- 172.21.15.137:0/3635088098 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f801f040 con 0x560b55a19410 2024-09-12T12:30:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.407+0000 7fb3ecff9640 1 -- 172.21.15.137:0/3635088098 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8004680 con 0x560b55a19410 2024-09-12T12:30:32.412 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- 172.21.15.137:0/3635088098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 msgr2=0x560b55a213e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a213e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8011d10 tx=0x7fb3f8005220 comp rx=0 tx=0).stop 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- 172.21.15.137:0/3635088098 shutdown_connections 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a23cb0 0x560b55a22aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a19410 0x560b55a213e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 --2- 172.21.15.137:0/3635088098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a18af0 0x560b55a1b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- 172.21.15.137:0/3635088098 >> 172.21.15.137:0/3635088098 conn(0x560b55a15060 msgr2=0x560b55a22390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- 172.21.15.137:0/3635088098 shutdown_connections 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- 172.21.15.137:0/3635088098 wait complete. 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- start start 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 Processor -- start 2024-09-12T12:30:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a18af0 0x560b55a27400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a27940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a23cb0 0x560b55a96900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081a9ba0 con 0x560b55a23cb0 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081ab010 con 0x560b55a18af0 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081aa270 con 0x560b55a19410 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40fe43740 10 client.0 _ll_register_callbacks cb 0x7fb3f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40759e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a18af0 0x560b55a27400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40759e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a18af0 0x560b55a27400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:41942/0 (socket says 172.21.15.137:41942) 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40759e640 1 -- 172.21.15.137:0/986078869 learned_addr learned my addr 172.21.15.137:0/986078869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb40759e640 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a23cb0 0x560b55a96900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.408+0000 7fb406d9d640 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a27940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40759e640 1 -- 172.21.15.137:0/986078869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 msgr2=0x560b55a27940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.414 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40759e640 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a27940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40759e640 1 -- 172.21.15.137:0/986078869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a23cb0 msgr2=0x560b55a96900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40759e640 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a23cb0 0x560b55a96900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40759e640 1 -- 172.21.15.137:0/986078869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b55a102d0 con 0x560b55a18af0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40759e640 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a18af0 0x560b55a27400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4000318b0 tx=0x7fb40002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbef0a3d6559e05d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb3eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb3eaff5640 1 -- 172.21.15.137:0/986078869 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400030e90 con 0x560b55a18af0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb3eaff5640 1 -- 172.21.15.137:0/986078869 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7fb40003fa20 con 0x560b55a18af0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f801f040 con 0x560b55a18af0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb3eaff5640 1 -- 172.21.15.137:0/986078869 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb40003fc20 con 0x560b55a18af0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40fe43740 10 client.26783 fetch_fsmap learned FSMap version 50 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x560b55a9e140 con 0x560b55a18af0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.409+0000 7fb3eaff5640 1 -- 172.21.15.137:0/986078869 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f801f040 con 0x560b55a18af0 2024-09-12T12:30:32.415 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb3eaff5640 1 -- 172.21.15.137:0/986078869 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 50) ==== 8358+0+0 (secure 0 0 0) 0x7fb40004e040 con 0x560b55a18af0 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb40fe43740 10 client.26783 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb40fe43740 10 client.26783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb40fe43740 20 client.26783 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb40fe43740 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a9e800 0x560b55aa0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb40fe43740 4 client.26783 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b55984d70 con 0x560b55a9e800 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb406d9d640 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a9e800 0x560b55aa0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb3eaff5640 10 client.26783 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:32.416 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:32.410+0000 7fb406d9d640 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a9e800 0x560b55aa0c20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87da05b555f39c36 server_cookie=ffe9d47c2156fa21 in_seq=0 out_seq=0 2024-09-12T12:30:33.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:32 smithi171 ceph-mon[35098]: pgmap v550: 97 pgs: 97 active+clean; 924 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 447 op/s 2024-09-12T12:30:33.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 83209 22.28 MB/sec execute 125 sec latency 5843.748 ms 2024-09-12T12:30:33.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:32 smithi137 ceph-mon[30950]: pgmap v550: 97 pgs: 97 active+clean; 924 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 447 op/s 2024-09-12T12:30:33.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:32 smithi189 ceph-mon[35475]: pgmap v550: 97 pgs: 97 active+clean; 924 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 447 op/s 2024-09-12T12:30:34.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:33 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:30:34.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 83724 22.25 MB/sec execute 126 sec latency 6566.416 ms 2024-09-12T12:30:34.300 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:33 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:30:34.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:33 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:30:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:34 smithi171 ceph-mon[35098]: pgmap v551: 97 pgs: 97 active+clean; 924 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 440 op/s 2024-09-12T12:30:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:34 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:30:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:34 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:30:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:34 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:30:35.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 84321 22.39 MB/sec execute 127 sec latency 7566.444 ms 2024-09-12T12:30:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:34 smithi137 ceph-mon[30950]: pgmap v551: 97 pgs: 97 active+clean; 924 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 440 op/s 2024-09-12T12:30:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:34 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:30:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:34 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:30:35.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:34 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:30:35.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:34 smithi189 ceph-mon[35475]: pgmap v551: 97 pgs: 97 active+clean; 924 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 440 op/s 2024-09-12T12:30:35.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:34 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:30:35.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:34 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:30:35.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:34 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:30:36.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 84994 22.51 MB/sec execute 128 sec latency 8566.472 ms 2024-09-12T12:30:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.174+0000 7fb3eaff5640 1 -- 172.21.15.137:0/986078869 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b55984d70 con 0x560b55a9e800 2024-09-12T12:30:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.174+0000 7fb3eaff5640 10 client.26783 handle_command_reply: tid=0 2024-09-12T12:30:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.174+0000 7fb40fe43740 1 client.26783 shutdown 2024-09-12T12:30:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.174+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a18af0 msgr2=0x560b55a27400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.174+0000 7fb40fe43740 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a18af0 0x560b55a27400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4000318b0 tx=0x7fb40002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 shutdown_connections 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x560b55a9e800 0x560b55aa0c20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560b55a23cb0 0x560b55a96900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560b55a19410 0x560b55a27940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 --2- 172.21.15.137:0/986078869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560b55a18af0 0x560b55a27400 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 >> 172.21.15.137:0/986078869 conn(0x560b55a15060 msgr2=0x560b55a22bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb3eaff5640 0 client.26783 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 shutdown_connections 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 1 -- 172.21.15.137:0/986078869 wait complete. 2024-09-12T12:30:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.175+0000 7fb40fe43740 20 client.26783 trim_cache size 0 max 16384 2024-09-12T12:30:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:30:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:30:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:30:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:30:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb3f00a2da0 msgr2=0x7fb3f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb3f00a2da0 0x7fb3f00a5260 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0cd000 tx=0x7fb3fc0d5550 comp rx=0 tx=0).stop 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 msgr2=0x7fb408198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb408198a30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00e9f0 tx=0x7fb3fc00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 shutdown_connections 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb3f00a2da0 0x7fb3f00a5260 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb408103aa0 0x7fb40819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4081008e0 0x7fb408198a30 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 --2- 172.21.15.137:0/1540899932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4080fff60 0x7fb4081984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 >> 172.21.15.137:0/1540899932 conn(0x7fb4080770f0 msgr2=0x7fb408105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 shutdown_connections 2024-09-12T12:30:37.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:37.176+0000 7fb40d0a5640 1 -- 172.21.15.137:0/1540899932 wait complete. 2024-09-12T12:30:37.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:36 smithi171 ceph-mon[35098]: pgmap v552: 97 pgs: 97 active+clean; 948 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 10 MiB/s wr, 695 op/s 2024-09-12T12:30:37.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 85471 22.50 MB/sec execute 129 sec latency 9566.500 ms 2024-09-12T12:30:37.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:36 smithi137 ceph-mon[30950]: pgmap v552: 97 pgs: 97 active+clean; 948 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 10 MiB/s wr, 695 op/s 2024-09-12T12:30:37.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:36 smithi189 ceph-mon[35475]: pgmap v552: 97 pgs: 97 active+clean; 948 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 10 MiB/s wr, 695 op/s 2024-09-12T12:30:38.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 86041 22.58 MB/sec execute 130 sec latency 10368.271 ms 2024-09-12T12:30:39.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 86733 22.65 MB/sec execute 131 sec latency 7454.863 ms 2024-09-12T12:30:39.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:38 smithi137 ceph-mon[30950]: pgmap v553: 97 pgs: 97 active+clean; 948 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.6 MiB/s wr, 473 op/s 2024-09-12T12:30:39.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:38 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:39.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:38 smithi189 ceph-mon[35475]: pgmap v553: 97 pgs: 97 active+clean; 948 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.6 MiB/s wr, 473 op/s 2024-09-12T12:30:39.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:38 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:39.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:38 smithi171 ceph-mon[35098]: pgmap v553: 97 pgs: 97 active+clean; 948 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.6 MiB/s wr, 473 op/s 2024-09-12T12:30:39.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:38 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:40.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 87550 22.87 MB/sec execute 132 sec latency 84.787 ms 2024-09-12T12:30:40.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:39 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:40.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:39 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:40.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:39 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:41.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 88171 22.93 MB/sec execute 133 sec latency 86.473 ms 2024-09-12T12:30:41.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:40 smithi137 ceph-mon[30950]: pgmap v554: 97 pgs: 97 active+clean; 949 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 514 op/s 2024-09-12T12:30:41.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:40 smithi189 ceph-mon[35475]: pgmap v554: 97 pgs: 97 active+clean; 949 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 514 op/s 2024-09-12T12:30:41.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:40 smithi171 ceph-mon[35098]: pgmap v554: 97 pgs: 97 active+clean; 949 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 514 op/s 2024-09-12T12:30:42.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 88895 23.05 MB/sec execute 134 sec latency 88.767 ms 2024-09-12T12:30:43.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 89578 23.15 MB/sec execute 135 sec latency 87.397 ms 2024-09-12T12:30:43.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:42 smithi137 ceph-mon[30950]: pgmap v555: 97 pgs: 97 active+clean; 966 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 641 op/s 2024-09-12T12:30:43.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:42 smithi189 ceph-mon[35475]: pgmap v555: 97 pgs: 97 active+clean; 966 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 641 op/s 2024-09-12T12:30:43.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:42 smithi171 ceph-mon[35098]: pgmap v555: 97 pgs: 97 active+clean; 966 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 641 op/s 2024-09-12T12:30:44.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 90321 23.26 MB/sec execute 136 sec latency 60.698 ms 2024-09-12T12:30:45.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 91115 23.43 MB/sec execute 137 sec latency 71.651 ms 2024-09-12T12:30:45.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:44 smithi137 ceph-mon[30950]: pgmap v556: 97 pgs: 97 active+clean; 966 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.5 MiB/s wr, 469 op/s 2024-09-12T12:30:45.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:44 smithi189 ceph-mon[35475]: pgmap v556: 97 pgs: 97 active+clean; 966 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.5 MiB/s wr, 469 op/s 2024-09-12T12:30:45.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:44 smithi171 ceph-mon[35098]: pgmap v556: 97 pgs: 97 active+clean; 966 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.5 MiB/s wr, 469 op/s 2024-09-12T12:30:46.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 91721 23.49 MB/sec execute 138 sec latency 103.722 ms 2024-09-12T12:30:47.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 92399 23.56 MB/sec execute 139 sec latency 167.255 ms 2024-09-12T12:30:47.234 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:46 smithi171 ceph-mon[35098]: pgmap v557: 97 pgs: 97 active+clean; 999 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2024-09-12T12:30:47.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:46 smithi137 ceph-mon[30950]: pgmap v557: 97 pgs: 97 active+clean; 999 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2024-09-12T12:30:47.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:46 smithi189 ceph-mon[35475]: pgmap v557: 97 pgs: 97 active+clean; 999 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.6 MiB/s wr, 684 op/s 2024-09-12T12:30:48.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 93047 23.64 MB/sec execute 140 sec latency 240.621 ms 2024-09-12T12:30:49.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 93841 23.78 MB/sec execute 141 sec latency 114.250 ms 2024-09-12T12:30:49.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:48 smithi137 ceph-mon[30950]: pgmap v558: 97 pgs: 97 active+clean; 999 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 430 op/s 2024-09-12T12:30:49.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:48 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:49.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:48 smithi189 ceph-mon[35475]: pgmap v558: 97 pgs: 97 active+clean; 999 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 430 op/s 2024-09-12T12:30:49.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:48 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:49.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:48 smithi171 ceph-mon[35098]: pgmap v558: 97 pgs: 97 active+clean; 999 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.3 MiB/s wr, 430 op/s 2024-09-12T12:30:49.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:48 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:30:50.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 94581 23.90 MB/sec execute 142 sec latency 80.073 ms 2024-09-12T12:30:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1028806114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc101350 msgr2=0x7ff3dc10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1028806114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc101350 0x7ff3dc10d880 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009930 tx=0x7ff3d802a420 comp rx=0 tx=0).stop 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1028806114 shutdown_connections 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1028806114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc101350 0x7ff3dc10d880 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1028806114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3dc1009b0 0x7ff3dc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1028806114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1069c0 0x7ff3dc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1028806114 >> 172.21.15.137:0/1028806114 conn(0x7ff3dc077990 msgr2=0x7ff3dc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1028806114 shutdown_connections 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.276+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1028806114 wait complete. 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 Processor -- start 2024-09-12T12:30:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 -- start start 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1009b0 0x7ff3dc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3dc101350 0x7ff3dc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc1069c0 0x7ff3dc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc110e00 con 0x7ff3dc1069c0 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc110c80 con 0x7ff3dc1009b0 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e38d7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc110f80 con 0x7ff3dc101350 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc1069c0 0x7ff3dc19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1673640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1009b0 0x7ff3dc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1673640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1009b0 0x7ff3dc198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55138/0 (socket says 172.21.15.137:55138) 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3dc101350 0x7ff3dc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1673640 1 -- 172.21.15.137:0/1051375277 learned_addr learned my addr 172.21.15.137:0/1051375277 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:50.280 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc1069c0 0x7ff3dc19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48830/0 (socket says 172.21.15.137:48830) 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 -- 172.21.15.137:0/1051375277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3dc101350 msgr2=0x7ff3dc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3dc101350 0x7ff3dc1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 -- 172.21.15.137:0/1051375277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1009b0 msgr2=0x7ff3dc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1009b0 0x7ff3dc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 -- 172.21.15.137:0/1051375277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc19d360 con 0x7ff3dc1069c0 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1673640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1009b0 0x7ff3dc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.277+0000 7ff3e1e74640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc1069c0 0x7ff3dc19cc30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009900 tx=0x7ff3d8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.278+0000 7ff3d27fc640 1 -- 172.21.15.137:0/1051375277 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d803f030 con 0x7ff3dc1069c0 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.278+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3dc19d5f0 con 0x7ff3dc1069c0 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.278+0000 7ff3d27fc640 1 -- 172.21.15.137:0/1051375277 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3d8004350 con 0x7ff3dc1069c0 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.278+0000 7ff3d27fc640 1 -- 172.21.15.137:0/1051375277 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8035070 con 0x7ff3dc1069c0 2024-09-12T12:30:50.281 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.278+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3dc1a4ee0 con 0x7ff3dc1069c0 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.280+0000 7ff3d27fc640 1 -- 172.21.15.137:0/1051375277 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff3d8042030 con 0x7ff3dc1069c0 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 Processor -- start 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 -- start start 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfe7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56185cf4c2f0 con 0x56185cff2cb0 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56185cef4270 con 0x56185cfe8410 2024-09-12T12:30:50.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e6673740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56185cf482b0 con 0x56185cfe7af0 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3d27fc640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff3c80a2e20 0x7ff3c80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e1673640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfe7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfe7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:35822/0 (socket says 172.21.15.137:35822) 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 -- 172.21.15.137:0/2158636544 learned_addr learned my addr 172.21.15.137:0/2158636544 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e1e74640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff3c80a2e20 0x7ff3c80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3d27fc640 1 -- 172.21.15.137:0/1051375277 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7ff3d80ef300 con 0x7ff3dc1069c0 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 -- 172.21.15.137:0/2158636544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 msgr2=0x56185cff2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 -- 172.21.15.137:0/2158636544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 msgr2=0x56185cff50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.281+0000 7ff3e0e72640 1 -- 172.21.15.137:0/2158636544 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185cef3340 con 0x56185cfe7af0 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.282+0000 7ff3e1e74640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff3c80a2e20 0x7ff3c80a52e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80bb820 tx=0x7ff3d808a7c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.282+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfe7ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc011810 tx=0x7ff3cc0097d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56809ccd516bdb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.282+0000 7ff3c77fe640 1 -- 172.21.15.137:0/2158636544 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc004030 con 0x56185cfe7af0 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.282+0000 7ff3c77fe640 1 -- 172.21.15.137:0/2158636544 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3cc00fca0 con 0x56185cfe7af0 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.282+0000 7ff3c77fe640 1 -- 172.21.15.137:0/2158636544 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc003970 con 0x56185cfe7af0 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 -- 172.21.15.137:0/2158636544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 msgr2=0x56185cfe7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfe7ed0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc011810 tx=0x7ff3cc0097d0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 -- 172.21.15.137:0/2158636544 shutdown_connections 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 --2- 172.21.15.137:0/2158636544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfe7ed0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 -- 172.21.15.137:0/2158636544 >> 172.21.15.137:0/2158636544 conn(0x56185cfe4060 msgr2=0x56185cfe4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 -- 172.21.15.137:0/2158636544 shutdown_connections 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 -- 172.21.15.137:0/2158636544 wait complete. 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 -- start start 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 Processor -- start 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.283+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e6673740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56185cff5880 con 0x56185cff2cb0 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e6673740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56185ced5290 con 0x56185cfe8410 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e6673740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56185cff5a00 con 0x56185cfe7af0 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e6673740 10 client.0 _ll_register_callbacks cb 0x56185cf06140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:50.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfefbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfefbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:35838/0 (socket says 172.21.15.137:35838) 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55160/0 (socket says 172.21.15.137:55160) 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e0e72640 1 -- 172.21.15.137:0/3860018305 learned_addr learned my addr 172.21.15.137:0/3860018305 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 -- 172.21.15.137:0/3860018305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 msgr2=0x56185cfefbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfefbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 -- 172.21.15.137:0/3860018305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 msgr2=0x56185cff0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 -- 172.21.15.137:0/3860018305 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185cc5f210 con 0x56185cfe8410 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e1673640 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff0130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400fe20 tx=0x7ff3d4012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbfa044e8696c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3860018305 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4003970 con 0x56185cfe8410 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3cc00fca0 con 0x56185cfe8410 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.284+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3860018305 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7ff3d4003b10 con 0x56185cfe8410 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3860018305 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff3d4003d10 con 0x56185cfe8410 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 10 client.26795 fetch_fsmap learned FSMap version 53 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x56185cd2b620 con 0x56185cfe8410 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3860018305 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3cc00fca0 con 0x56185cfe8410 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3860018305 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ff3d4023020 con 0x56185cfe8410 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 10 client.26795 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 10 client.26795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 20 client.26795 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d002ce0 0x56185d0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 4 client.26795 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:30:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56185cf59a20 con 0x56185d002ce0 2024-09-12T12:30:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.285+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d002ce0 0x56185d0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.286+0000 7ff3c4ff9640 10 client.26795 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.286+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d002ce0 0x56185d0050d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492c6441de2412bc server_cookie=324913756e157758 in_seq=0 out_seq=0 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3860018305 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56185cf59a20 con 0x56185d002ce0 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3c4ff9640 10 client.26795 handle_command_reply: tid=0 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 client.26795 shutdown 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 msgr2=0x56185cff0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff0130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400fe20 tx=0x7ff3d4012820 comp rx=0 tx=0).stop 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 shutdown_connections 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d002ce0 0x56185d0050d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185cff0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3c4ff9640 0 client.26795 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185cff0130 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3860018305 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cfefbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 >> 172.21.15.137:0/3860018305 conn(0x56185cfe4060 msgr2=0x56185cfe9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 shutdown_connections 2024-09-12T12:30:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 1 -- 172.21.15.137:0/3860018305 wait complete. 2024-09-12T12:30:50.295 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.290+0000 7ff3e6673740 20 client.26795 trim_cache size 0 max 16384 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 Processor -- start 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 -- start start 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cff04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185d00a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185d00aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc1afff0 con 0x56185cff2cb0 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc1ad8c0 con 0x56185cfe8410 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e6673740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc1ab730 con 0x56185cfe7af0 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185d00aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185d00aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48864/0 (socket says 172.21.15.137:48864) 2024-09-12T12:30:50.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e0e72640 1 -- 172.21.15.137:0/4163038534 learned_addr learned my addr 172.21.15.137:0/4163038534 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e1673640 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185d00a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.307+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cff04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3e0e72640 1 -- 172.21.15.137:0/4163038534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 msgr2=0x56185cff04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cff04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3e0e72640 1 -- 172.21.15.137:0/4163038534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 msgr2=0x56185d00a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185d00a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3e0e72640 1 -- 172.21.15.137:0/4163038534 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56185d005440 con 0x56185cff2cb0 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185d00aed0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00f790 tx=0x7ff3cc013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8347927092a23839 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3c6ffd640 1 -- 172.21.15.137:0/4163038534 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc011070 con 0x56185cff2cb0 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3c6ffd640 1 -- 172.21.15.137:0/4163038534 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3cc034070 con 0x56185cff2cb0 2024-09-12T12:30:50.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.308+0000 7ff3c6ffd640 1 -- 172.21.15.137:0/4163038534 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc02f6a0 con 0x56185cff2cb0 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 -- 172.21.15.137:0/4163038534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 msgr2=0x56185d00aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185d00aed0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00f790 tx=0x7ff3cc013040 comp rx=0 tx=0).stop 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 -- 172.21.15.137:0/4163038534 shutdown_connections 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cff2cb0 0x56185d00aed0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cfe8410 0x56185d00a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 --2- 172.21.15.137:0/4163038534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185cff04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 -- 172.21.15.137:0/4163038534 >> 172.21.15.137:0/4163038534 conn(0x56185cfe4060 msgr2=0x56185cff1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 -- 172.21.15.137:0/4163038534 shutdown_connections 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 -- 172.21.15.137:0/4163038534 wait complete. 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.309+0000 7ff3e6673740 1 -- start start 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 1 Processor -- start 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185d068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cfe8410 0x56185d069200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cff2cb0 0x56185cffacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc1abd80 con 0x56185cfe8410 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc1ad1f0 con 0x56185cff2cb0 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc1ac450 con 0x56185cfe7af0 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e6673740 10 client.0 _ll_register_callbacks cb 0x7ff3d400f4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cfe8410 0x56185d069200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cfe8410 0x56185d069200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48878/0 (socket says 172.21.15.137:48878) 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 -- 172.21.15.137:0/3803166702 learned_addr learned my addr 172.21.15.137:0/3803166702 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e0e72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185d068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cff2cb0 0x56185cffacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 -- 172.21.15.137:0/3803166702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 msgr2=0x56185d068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185d068cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 -- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cff2cb0 msgr2=0x56185cffacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cff2cb0 0x56185cffacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.310+0000 7ff3e1673640 1 -- 172.21.15.137:0/3803166702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56185cffb4f0 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3e1673640 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cfe8410 0x56185d069200 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009aa0 tx=0x7ff3d4026b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d02ca19f06f804 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3803166702 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4026e50 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3803166702 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7ff3d4018040 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3cc034070 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3803166702 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff3d4025460 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3e6673740 10 client.17565 fetch_fsmap learned FSMap version 53 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.311+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x56185cfed1d0 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3803166702 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3cc034070 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3803166702 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ff3d402d030 con 0x56185cfe8410 2024-09-12T12:30:50.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e6673740 10 client.17565 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e6673740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e6673740 20 client.17565 populate_metadata read hostname 'smithi137' 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d0721b0 0x56185d0745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e6673740 4 client.17565 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56185cf53d70 con 0x56185d0721b0 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d0721b0 0x56185d0745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3c4ff9640 10 client.17565 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.312+0000 7ff3e0e72640 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d0721b0 0x56185d0745d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e82118d40ab9eb server_cookie=78b994695c98a713 in_seq=0 out_seq=0 2024-09-12T12:30:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3c4ff9640 1 -- 172.21.15.137:0/3803166702 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56185cf53d70 con 0x56185d0721b0 2024-09-12T12:30:50.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3c4ff9640 10 client.17565 handle_command_reply: tid=0 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 client.17565 shutdown 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cfe8410 msgr2=0x56185d069200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cfe8410 0x56185d069200 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009aa0 tx=0x7ff3d4026b50 comp rx=0 tx=0).stop 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 shutdown_connections 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56185cff2cb0 0x56185cffacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56185d0721b0 0x56185d0745d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3c4ff9640 0 client.17565 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56185cfe8410 0x56185d069200 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 --2- 172.21.15.137:0/3803166702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56185cfe7af0 0x56185d068cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 >> 172.21.15.137:0/3803166702 conn(0x56185cfe4060 msgr2=0x56185cfe29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.675+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 shutdown_connections 2024-09-12T12:30:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e6673740 1 -- 172.21.15.137:0/3803166702 wait complete. 2024-09-12T12:30:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e6673740 20 client.17565 trim_cache size 0 max 16384 2024-09-12T12:30:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:30:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:30:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:30:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff3c80a2e20 msgr2=0x7ff3c80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff3c80a2e20 0x7ff3c80a52e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3d80bb820 tx=0x7ff3d808a7c0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc1069c0 msgr2=0x7ff3dc19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc1069c0 0x7ff3dc19cc30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009900 tx=0x7ff3d8005e70 comp rx=0 tx=0).stop 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 shutdown_connections 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff3c80a2e20 0x7ff3c80a52e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff3dc1069c0 0x7ff3dc19cc30 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.676+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3dc101350 0x7ff3dc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.677+0000 7ff3e38d7640 1 --2- 172.21.15.137:0/1051375277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff3dc1009b0 0x7ff3dc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:30:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.677+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 >> 172.21.15.137:0/1051375277 conn(0x7ff3dc077990 msgr2=0x7ff3dc0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:30:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.677+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 shutdown_connections 2024-09-12T12:30:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:30:50.677+0000 7ff3e38d7640 1 -- 172.21.15.137:0/1051375277 wait complete. 2024-09-12T12:30:51.227 INFO:tasks.workunit.client.0.smithi137.stdout: 10 95091 23.93 MB/sec execute 143 sec latency 230.560 ms 2024-09-12T12:30:51.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:50 smithi137 ceph-mon[30950]: pgmap v559: 97 pgs: 97 active+clean; 1003 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 468 op/s 2024-09-12T12:30:51.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:50 smithi189 ceph-mon[35475]: pgmap v559: 97 pgs: 97 active+clean; 1003 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 468 op/s 2024-09-12T12:30:51.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:50 smithi171 ceph-mon[35098]: pgmap v559: 97 pgs: 97 active+clean; 1003 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.7 MiB/s wr, 468 op/s 2024-09-12T12:30:52.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 95782 24.07 MB/sec execute 144 sec latency 82.563 ms 2024-09-12T12:30:53.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 96300 24.05 MB/sec execute 145 sec latency 170.874 ms 2024-09-12T12:30:53.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:52 smithi137 ceph-mon[30950]: pgmap v560: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 603 op/s 2024-09-12T12:30:53.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:52 smithi189 ceph-mon[35475]: pgmap v560: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 603 op/s 2024-09-12T12:30:53.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:52 smithi171 ceph-mon[35098]: pgmap v560: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 8.9 MiB/s wr, 603 op/s 2024-09-12T12:30:54.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 96968 24.12 MB/sec execute 146 sec latency 109.729 ms 2024-09-12T12:30:55.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 97710 24.20 MB/sec execute 147 sec latency 98.152 ms 2024-09-12T12:30:55.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:54 smithi137 ceph-mon[30950]: pgmap v561: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 429 op/s 2024-09-12T12:30:55.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:54 smithi189 ceph-mon[35475]: pgmap v561: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 429 op/s 2024-09-12T12:30:55.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:54 smithi171 ceph-mon[35098]: pgmap v561: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 429 op/s 2024-09-12T12:30:56.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 98445 24.36 MB/sec execute 148 sec latency 127.921 ms 2024-09-12T12:30:57.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 99067 24.41 MB/sec execute 149 sec latency 99.020 ms 2024-09-12T12:30:57.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:56 smithi137 ceph-mon[30950]: pgmap v562: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-12T12:30:57.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:56 smithi189 ceph-mon[35475]: pgmap v562: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-12T12:30:57.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:56 smithi171 ceph-mon[35098]: pgmap v562: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-12T12:30:58.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 99457 24.38 MB/sec execute 150 sec latency 325.966 ms 2024-09-12T12:30:59.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 99989 24.42 MB/sec execute 151 sec latency 428.611 ms 2024-09-12T12:30:59.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:30:58 smithi137 ceph-mon[30950]: pgmap v563: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 569 op/s 2024-09-12T12:30:59.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:30:58 smithi189 ceph-mon[35475]: pgmap v563: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 569 op/s 2024-09-12T12:30:59.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:30:58 smithi171 ceph-mon[35098]: pgmap v563: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 569 op/s 2024-09-12T12:31:00.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 100693 24.51 MB/sec execute 152 sec latency 80.643 ms 2024-09-12T12:31:01.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 100835 24.37 MB/sec execute 153 sec latency 827.624 ms 2024-09-12T12:31:01.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:00 smithi137 ceph-mon[30950]: pgmap v564: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.3 MiB/s wr, 622 op/s 2024-09-12T12:31:01.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:00 smithi189 ceph-mon[35475]: pgmap v564: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.3 MiB/s wr, 622 op/s 2024-09-12T12:31:01.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:00 smithi171 ceph-mon[35098]: pgmap v564: 97 pgs: 97 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.3 MiB/s wr, 622 op/s 2024-09-12T12:31:02.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 100920 24.23 MB/sec execute 154 sec latency 1668.553 ms 2024-09-12T12:31:03.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 101253 24.16 MB/sec execute 155 sec latency 252.125 ms 2024-09-12T12:31:03.251 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:02 smithi171 ceph-mon[35098]: pgmap v565: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 820 op/s 2024-09-12T12:31:03.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:02 smithi137 ceph-mon[30950]: pgmap v565: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 820 op/s 2024-09-12T12:31:03.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:02 smithi189 ceph-mon[35475]: pgmap v565: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 12 MiB/s wr, 820 op/s 2024-09-12T12:31:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 -- 172.21.15.137:0/4174460405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f41035a0 msgr2=0x7f06f410ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 --2- 172.21.15.137:0/4174460405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f41035a0 0x7f06f410ba80 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009900 tx=0x7f06e802a480 comp rx=0 tx=0).stop 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 -- 172.21.15.137:0/4174460405 shutdown_connections 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 --2- 172.21.15.137:0/4174460405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f410c0d0 0x7f06f410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 --2- 172.21.15.137:0/4174460405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f41035a0 0x7f06f410ba80 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 --2- 172.21.15.137:0/4174460405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f4102c80 0x7f06f4103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 -- 172.21.15.137:0/4174460405 >> 172.21.15.137:0/4174460405 conn(0x7f06f4077990 msgr2=0x7f06f40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 -- 172.21.15.137:0/4174460405 shutdown_connections 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.786+0000 7f06f9b59640 1 -- 172.21.15.137:0/4174460405 wait complete. 2024-09-12T12:31:03.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.787+0000 7f06f9b59640 1 Processor -- start 2024-09-12T12:31:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.787+0000 7f06f9b59640 1 -- start start 2024-09-12T12:31:03.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.787+0000 7f06f9b59640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f4102c80 0x7f06f4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.787+0000 7f06f9b59640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f41035a0 0x7f06f4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f9b59640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f410c0d0 0x7f06f419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f9b59640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4111eb0 con 0x7f06f410c0d0 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f9b59640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4111d30 con 0x7f06f41035a0 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f9b59640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f4112030 con 0x7f06f4102c80 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f4102c80 0x7f06f4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f3fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f410c0d0 0x7f06f419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f4102c80 0x7f06f4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:43644/0 (socket says 172.21.15.137:43644) 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f37fe640 1 -- 172.21.15.137:0/522203669 learned_addr learned my addr 172.21.15.137:0/522203669 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f41035a0 0x7f06f4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f3fff640 1 -- 172.21.15.137:0/522203669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f4102c80 msgr2=0x7f06f4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f3fff640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f4102c80 0x7f06f4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f3fff640 1 -- 172.21.15.137:0/522203669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f41035a0 msgr2=0x7f06f4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f3fff640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f41035a0 0x7f06f4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f3fff640 1 -- 172.21.15.137:0/522203669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f419d330 con 0x7f06f410c0d0 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f41035a0 0x7f06f4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f3fff640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f410c0d0 0x7f06f419cc00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009fd0 tx=0x7f06ec00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f0ff9640 1 -- 172.21.15.137:0/522203669 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00ce20 con 0x7f06f410c0d0 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f0ff9640 1 -- 172.21.15.137:0/522203669 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06ec005020 con 0x7f06f410c0d0 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f419d620 con 0x7f06f410c0d0 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.788+0000 7f06f0ff9640 1 -- 172.21.15.137:0/522203669 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec010720 con 0x7f06f410c0d0 2024-09-12T12:31:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.789+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f41a4eb0 con 0x7f06f410c0d0 2024-09-12T12:31:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.791+0000 7f06f0ff9640 1 -- 172.21.15.137:0/522203669 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f06ec005540 con 0x7f06f410c0d0 2024-09-12T12:31:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 Processor -- start 2024-09-12T12:31:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 -- start start 2024-09-12T12:31:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bb9a10 0x556755bb9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.795 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755bcb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556755b1e2f0 con 0x556755bba330 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556755ac6270 con 0x556755bb9a10 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556755b1a2b0 con 0x556755bc90c0 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f0ff9640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bb9a10 0x556755bb9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bb9a10 0x556755bb9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39782/0 (socket says 172.21.15.137:39782) 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 -- 172.21.15.137:0/2734993010 learned_addr learned my addr 172.21.15.137:0/2734993010 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755bcb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f3fff640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 -- 172.21.15.137:0/2734993010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 msgr2=0x556755bcb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755bcb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 -- 172.21.15.137:0/2734993010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 msgr2=0x556755bc8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f2ffd640 1 -- 172.21.15.137:0/2734993010 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556755ac5340 con 0x556755bb9a10 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.792+0000 7f06f37fe640 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.793+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bb9a10 0x556755bb9df0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06e802a9a0 tx=0x7f06e80314e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b9f36e9d5394e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.793+0000 7f06f0ff9640 1 -- 172.21.15.137:0/522203669 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7f06ec0cfa20 con 0x7f06f410c0d0 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.793+0000 7f06f3fff640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06ec09bc60 tx=0x7f06ec0054b0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.793+0000 7f06d9ffb640 1 -- 172.21.15.137:0/2734993010 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8044070 con 0x556755bb9a10 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.793+0000 7f06d9ffb640 1 -- 172.21.15.137:0/2734993010 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8005020 con 0x556755bb9a10 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.793+0000 7f06d9ffb640 1 -- 172.21.15.137:0/2734993010 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8035040 con 0x556755bb9a10 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 -- 172.21.15.137:0/2734993010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bb9a10 msgr2=0x556755bb9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bb9a10 0x556755bb9df0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f06e802a9a0 tx=0x7f06e80314e0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 -- 172.21.15.137:0/2734993010 shutdown_connections 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755bcb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/2734993010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bb9a10 0x556755bb9df0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 -- 172.21.15.137:0/2734993010 >> 172.21.15.137:0/2734993010 conn(0x556755bb6060 msgr2=0x556755bb6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 -- 172.21.15.137:0/2734993010 shutdown_connections 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 -- 172.21.15.137:0/2734993010 wait complete. 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.794+0000 7f06fc9d3740 1 -- start start 2024-09-12T12:31:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 1 Processor -- start 2024-09-12T12:31:03.800 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bc90c0 0x556755bc6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556755bcbc90 con 0x556755bba330 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556755aa7290 con 0x556755bc90c0 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556755bcbe10 con 0x556755bb9a10 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06fc9d3740 10 client.0 _ll_register_callbacks cb 0x556755ad8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f2ffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bc90c0 0x556755bc6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40108/0 (socket says 172.21.15.137:40108) 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f37fe640 1 -- 172.21.15.137:0/4270446907 learned_addr learned my addr 172.21.15.137:0/4270446907 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f37fe640 1 -- 172.21.15.137:0/4270446907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 msgr2=0x556755bc5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f37fe640 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bc5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f37fe640 1 -- 172.21.15.137:0/4270446907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bc90c0 msgr2=0x556755bc6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.795+0000 7f06f37fe640 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bc90c0 0x556755bc6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06f37fe640 1 -- 172.21.15.137:0/4270446907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556755831210 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06f37fe640 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc64f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f06e40126f0 tx=0x7f06e4012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31175e9ca62c2464 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06d2ffd640 1 -- 172.21.15.137:0/4270446907 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e400f920 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06d2ffd640 1 -- 172.21.15.137:0/4270446907 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7f06e4012e00 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e8005020 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06d2ffd640 1 -- 172.21.15.137:0/4270446907 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f06e400fac0 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06fc9d3740 10 client.17577 fetch_fsmap learned FSMap version 53 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5567558fd620 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.796+0000 7f06d2ffd640 1 -- 172.21.15.137:0/4270446907 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8005020 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06d2ffd640 1 -- 172.21.15.137:0/4270446907 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f06e4023070 con 0x556755bba330 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06fc9d3740 10 client.17577 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:31:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06fc9d3740 10 client.17577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06fc9d3740 20 client.17577 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755bd9160 0x556755bdb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06fc9d3740 4 client.17577 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:31:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556755b2ba20 con 0x556755bd9160 2024-09-12T12:31:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.797+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755bd9160 0x556755bdb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.798+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755bd9160 0x556755bdb550 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9068458f99bee9a6 server_cookie=7973cb325c7558ec in_seq=0 out_seq=0 2024-09-12T12:31:03.803 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.798+0000 7f06d2ffd640 10 client.17577 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.800+0000 7f06d2ffd640 1 -- 172.21.15.137:0/4270446907 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556755b2ba20 con 0x556755bd9160 2024-09-12T12:31:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.800+0000 7f06d2ffd640 10 client.17577 handle_command_reply: tid=0 2024-09-12T12:31:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 client.17577 shutdown 2024-09-12T12:31:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 msgr2=0x556755bc64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc64f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f06e40126f0 tx=0x7f06e4012bb0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 shutdown_connections 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bc90c0 0x556755bc6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755bd9160 0x556755bdb550 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06d2ffd640 0 client.17577 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bba330 0x556755bc64f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/4270446907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bc5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 >> 172.21.15.137:0/4270446907 conn(0x556755bb6060 msgr2=0x556755bbbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 shutdown_connections 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 1 -- 172.21.15.137:0/4270446907 wait complete. 2024-09-12T12:31:03.805 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.801+0000 7f06fc9d3740 20 client.17577 trim_cache size 0 max 16384 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 Processor -- start 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 -- start start 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bb9a10 0x556755bc6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755be0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755be1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06f41add90 con 0x556755bb9a10 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06f41ab660 con 0x556755bba330 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f41a94d0 con 0x556755bc90c0 2024-09-12T12:31:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06f2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bb9a10 0x556755bc6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06f2ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bb9a10 0x556755bc6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40114/0 (socket says 172.21.15.137:40114) 2024-09-12T12:31:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06f2ffd640 1 -- 172.21.15.137:0/1651113315 learned_addr learned my addr 172.21.15.137:0/1651113315 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06f37fe640 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755be0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.822+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755be1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06f37fe640 1 -- 172.21.15.137:0/1651113315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 msgr2=0x556755be1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06f37fe640 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755be1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06f37fe640 1 -- 172.21.15.137:0/1651113315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bb9a10 msgr2=0x556755bc6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06f37fe640 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bb9a10 0x556755bc6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06f37fe640 1 -- 172.21.15.137:0/1651113315 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556755bba160 con 0x556755bba330 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bb9a10 0x556755bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06f37fe640 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755be0dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f06e4010e50 tx=0x7f06e400f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ba74f427c277b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06d97fa640 1 -- 172.21.15.137:0/1651113315 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4029030 con 0x556755bba330 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06d97fa640 1 -- 172.21.15.137:0/1651113315 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e40122e0 con 0x556755bba330 2024-09-12T12:31:03.826 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.823+0000 7f06d97fa640 1 -- 172.21.15.137:0/1651113315 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40107c0 con 0x556755bba330 2024-09-12T12:31:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.824+0000 7f06fc9d3740 1 -- 172.21.15.137:0/1651113315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 msgr2=0x556755be0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.824+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755be0dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f06e4010e50 tx=0x7f06e400f830 comp rx=0 tx=0).stop 2024-09-12T12:31:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 -- 172.21.15.137:0/1651113315 shutdown_connections 2024-09-12T12:31:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bc90c0 0x556755be1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755be0dd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/1651113315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bb9a10 0x556755bc6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.827 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 -- 172.21.15.137:0/1651113315 >> 172.21.15.137:0/1651113315 conn(0x556755bb6060 msgr2=0x556755bc8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 -- 172.21.15.137:0/1651113315 shutdown_connections 2024-09-12T12:31:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 -- 172.21.15.137:0/1651113315 wait complete. 2024-09-12T12:31:03.828 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 -- start start 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.825+0000 7f06fc9d3740 1 Processor -- start 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bd1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755bccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06fc9d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bc90c0 0x556755bcd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06f41a9b20 con 0x556755bc90c0 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06f41aaf90 con 0x556755bba330 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06fc9d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06f41aa1f0 con 0x556755bb9a10 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06fc9d3740 10 client.0 _ll_register_callbacks cb 0x7f06e4009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:03.830 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06f37fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755bccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06f37fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755bccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:39828/0 (socket says 172.21.15.137:39828) 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06f37fe640 1 -- 172.21.15.137:0/3447740043 learned_addr learned my addr 172.21.15.137:0/3447740043 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bd1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.826+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bc90c0 0x556755bcd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06f2ffd640 1 -- 172.21.15.137:0/3447740043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 msgr2=0x556755bccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755bccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06f2ffd640 1 -- 172.21.15.137:0/3447740043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bc90c0 msgr2=0x556755bcd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bc90c0 0x556755bcd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06f2ffd640 1 -- 172.21.15.137:0/3447740043 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556755bcd8b0 con 0x556755bb9a10 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06f2ffd640 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bd1850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f06e8032c00 tx=0x7f06e803c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b3eec20930688bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:31:03.831 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06d2ffd640 1 -- 172.21.15.137:0/3447740043 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8035040 con 0x556755bb9a10 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.827+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e40122e0 con 0x556755bb9a10 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.828+0000 7f06d2ffd640 1 -- 172.21.15.137:0/3447740043 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8484+0+0 (secure 0 0 0) 0x7f06e8004ca0 con 0x556755bb9a10 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.828+0000 7f06d2ffd640 1 -- 172.21.15.137:0/3447740043 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f06e8044040 con 0x556755bb9a10 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.828+0000 7f06fc9d3740 10 client.26881 fetch_fsmap learned FSMap version 53 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.828+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x556755c54df0 con 0x556755bb9a10 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.828+0000 7f06d2ffd640 1 -- 172.21.15.137:0/3447740043 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e40122e0 con 0x556755bb9a10 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.829+0000 7f06d2ffd640 1 -- 172.21.15.137:0/3447740043 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f06e8054ab0 con 0x556755bb9a10 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.829+0000 7f06fc9d3740 10 client.26881 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.829+0000 7f06fc9d3740 10 client.26881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.829+0000 7f06fc9d3740 20 client.26881 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.829+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755c55010 0x556755c57400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.829+0000 7f06fc9d3740 4 client.26881 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:31:03.832 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.829+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556755b25d70 con 0x556755c55010 2024-09-12T12:31:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.830+0000 7f06f37fe640 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755c55010 0x556755c57400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.831+0000 7f06f37fe640 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755c55010 0x556755c57400 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796f84522af43c28 server_cookie=7e56fbdf880a3897 in_seq=0 out_seq=0 2024-09-12T12:31:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:03.831+0000 7f06d2ffd640 10 client.26881 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.124+0000 7f06d2ffd640 1 -- 172.21.15.137:0/3447740043 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556755b25d70 con 0x556755c55010 2024-09-12T12:31:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.124+0000 7f06d2ffd640 10 client.26881 handle_command_reply: tid=0 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.124+0000 7f06fc9d3740 1 client.26881 shutdown 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 msgr2=0x556755bd1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bd1850 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f06e8032c00 tx=0x7f06e803c040 comp rx=0 tx=0).stop 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 shutdown_connections 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556755c55010 0x556755c57400 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556755bc90c0 0x556755bcd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06d2ffd640 0 client.26881 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556755bba330 0x556755bccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 --2- 172.21.15.137:0/3447740043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556755bb9a10 0x556755bd1850 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.125+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 >> 172.21.15.137:0/3447740043 conn(0x556755bb6060 msgr2=0x556755bb49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.126+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 shutdown_connections 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.126+0000 7f06fc9d3740 1 -- 172.21.15.137:0/3447740043 wait complete. 2024-09-12T12:31:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.126+0000 7f06fc9d3740 20 client.26881 trim_cache size 0 max 16384 2024-09-12T12:31:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:31:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:31:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:31:04.128 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 msgr2=0x7f06dc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f06ec09bc60 tx=0x7f06ec0054b0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f410c0d0 msgr2=0x7f06f419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f410c0d0 0x7f06f419cc00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009fd0 tx=0x7f06ec00ef10 comp rx=0 tx=0).stop 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 shutdown_connections 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f06dc0a2da0 0x7f06dc0a5260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06f410c0d0 0x7f06f419cc00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06f41035a0 0x7f06f4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 --2- 172.21.15.137:0/522203669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06f4102c80 0x7f06f4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 >> 172.21.15.137:0/522203669 conn(0x7f06f4077990 msgr2=0x7f06f40fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 shutdown_connections 2024-09-12T12:31:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:04.127+0000 7f06f9b59640 1 -- 172.21.15.137:0/522203669 wait complete. 2024-09-12T12:31:04.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 101677 24.18 MB/sec execute 156 sec latency 287.300 ms 2024-09-12T12:31:05.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 102327 24.29 MB/sec execute 157 sec latency 159.535 ms 2024-09-12T12:31:05.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:05 smithi137 ceph-mon[30950]: pgmap v566: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 644 op/s 2024-09-12T12:31:05.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:05 smithi189 ceph-mon[35475]: pgmap v566: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 644 op/s 2024-09-12T12:31:05.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:05 smithi171 ceph-mon[35098]: pgmap v566: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 644 op/s 2024-09-12T12:31:06.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 102906 24.35 MB/sec execute 158 sec latency 95.422 ms 2024-09-12T12:31:07.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 103444 24.33 MB/sec execute 159 sec latency 278.602 ms 2024-09-12T12:31:07.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:07 smithi137 ceph-mon[30950]: pgmap v567: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-12T12:31:07.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:07 smithi189 ceph-mon[35475]: pgmap v567: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-12T12:31:07.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:07 smithi171 ceph-mon[35098]: pgmap v567: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 1.06k op/s 2024-09-12T12:31:08.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 103979 24.36 MB/sec execute 160 sec latency 134.167 ms 2024-09-12T12:31:09.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 104227 24.30 MB/sec execute 161 sec latency 384.766 ms 2024-09-12T12:31:09.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:09 smithi137 ceph-mon[30950]: pgmap v568: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 710 op/s 2024-09-12T12:31:09.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:09 smithi137 ceph-mon[30950]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T12:31:09.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:09 smithi189 ceph-mon[35475]: pgmap v568: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 710 op/s 2024-09-12T12:31:09.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:09 smithi189 ceph-mon[35475]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T12:31:09.453 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:09 smithi171 ceph-mon[35098]: pgmap v568: 97 pgs: 97 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 710 op/s 2024-09-12T12:31:09.453 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:09 smithi171 ceph-mon[35098]: osdmap e114: 12 total, 12 up, 12 in 2024-09-12T12:31:10.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 104556 24.24 MB/sec execute 162 sec latency 257.504 ms 2024-09-12T12:31:11.228 INFO:tasks.workunit.client.0.smithi137.stdout: 10 105216 24.34 MB/sec execute 163 sec latency 114.656 ms 2024-09-12T12:31:11.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:11 smithi137 ceph-mon[30950]: pgmap v570: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 848 op/s 2024-09-12T12:31:11.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:11 smithi189 ceph-mon[35475]: pgmap v570: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 848 op/s 2024-09-12T12:31:11.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:11 smithi171 ceph-mon[35098]: pgmap v570: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 848 op/s 2024-09-12T12:31:12.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 105912 24.44 MB/sec execute 164 sec latency 85.085 ms 2024-09-12T12:31:13.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 106482 24.45 MB/sec execute 165 sec latency 118.070 ms 2024-09-12T12:31:13.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:13 smithi137 ceph-mon[30950]: pgmap v571: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-12T12:31:13.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:13 smithi189 ceph-mon[35475]: pgmap v571: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-12T12:31:13.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:13 smithi171 ceph-mon[35098]: pgmap v571: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-12T12:31:14.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 106907 24.44 MB/sec execute 166 sec latency 250.717 ms 2024-09-12T12:31:15.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 107644 24.56 MB/sec execute 167 sec latency 94.109 ms 2024-09-12T12:31:15.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:15 smithi137 ceph-mon[30950]: pgmap v572: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-12T12:31:15.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:15 smithi189 ceph-mon[35475]: pgmap v572: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-12T12:31:15.449 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:15 smithi171 ceph-mon[35098]: pgmap v572: 97 pgs: 14 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 54 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 788 op/s 2024-09-12T12:31:16.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 108305 24.59 MB/sec execute 168 sec latency 136.197 ms 2024-09-12T12:31:17.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 108982 24.69 MB/sec execute 169 sec latency 96.427 ms 2024-09-12T12:31:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 -- 172.21.15.137:0/3036345275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a501011e0 msgr2=0x7f3a50104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 --2- 172.21.15.137:0/3036345275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a501011e0 0x7f3a50104dc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000d960 tx=0x7f3a4002c2b0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 -- 172.21.15.137:0/3036345275 shutdown_connections 2024-09-12T12:31:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 --2- 172.21.15.137:0/3036345275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a50105520 0x7f3a5010d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 --2- 172.21.15.137:0/3036345275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a501011e0 0x7f3a50104dc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 --2- 172.21.15.137:0/3036345275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a50100830 0x7f3a50100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 -- 172.21.15.137:0/3036345275 >> 172.21.15.137:0/3036345275 conn(0x7f3a50075f20 msgr2=0x7f3a50076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 -- 172.21.15.137:0/3036345275 shutdown_connections 2024-09-12T12:31:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.260+0000 7f3a58398640 1 -- 172.21.15.137:0/3036345275 wait complete. 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.261+0000 7f3a58398640 1 Processor -- start 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.261+0000 7f3a58398640 1 -- start start 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a58398640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a50100830 0x7f3a5010d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a58398640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a501011e0 0x7f3a5010b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a58398640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a50105520 0x7f3a5010b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a58398640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5010f2c0 con 0x7f3a50105520 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a58398640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5010f140 con 0x7f3a501011e0 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a58398640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5010f440 con 0x7f3a50100830 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a50105520 0x7f3a5010b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a50105520 0x7f3a5010b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34586/0 (socket says 172.21.15.137:34586) 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 -- 172.21.15.137:0/1122069911 learned_addr learned my addr 172.21.15.137:0/1122069911 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a55933640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a501011e0 0x7f3a5010b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56134640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a50100830 0x7f3a5010d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 -- 172.21.15.137:0/1122069911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a50100830 msgr2=0x7f3a5010d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a50100830 0x7f3a5010d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 -- 172.21.15.137:0/1122069911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a501011e0 msgr2=0x7f3a5010b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a501011e0 0x7f3a5010b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.262+0000 7f3a56935640 1 -- 172.21.15.137:0/1122069911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a501aac90 con 0x7f3a50105520 2024-09-12T12:31:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.263+0000 7f3a56935640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a50105520 0x7f3a5010b8c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3a48004410 tx=0x7f3a480048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.263+0000 7f3a477fe640 1 -- 172.21.15.137:0/1122069911 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48004b20 con 0x7f3a50105520 2024-09-12T12:31:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.263+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a501aaf80 con 0x7f3a50105520 2024-09-12T12:31:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.263+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a501ab4a0 con 0x7f3a50105520 2024-09-12T12:31:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.263+0000 7f3a477fe640 1 -- 172.21.15.137:0/1122069911 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a48004cc0 con 0x7f3a50105520 2024-09-12T12:31:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.263+0000 7f3a477fe640 1 -- 172.21.15.137:0/1122069911 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a480157f0 con 0x7f3a50105520 2024-09-12T12:31:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.266+0000 7f3a5b20f740 1 Processor -- start 2024-09-12T12:31:17.267 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.266+0000 7f3a477fe640 1 -- 172.21.15.137:0/1122069911 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3a48015ab0 con 0x7f3a50105520 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a5b20f740 1 -- start start 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a586cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586d400 0x55a4a5877760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a4a57d12f0 con 0x55a4a586d400 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a4a5779270 con 0x55a4a586cae0 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a4a57cd2b0 con 0x55a4a5877ca0 2024-09-12T12:31:17.268 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a56134640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586d400 0x55a4a5877760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a55933640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a586cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a56134640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586d400 0x55a4a5877760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34594/0 (socket says 172.21.15.137:34594) 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.267+0000 7f3a56134640 1 -- 172.21.15.137:0/2109182600 learned_addr learned my addr 172.21.15.137:0/2109182600 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a55933640 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a56134640 1 -- 172.21.15.137:0/2109182600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 msgr2=0x55a4a587a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a56134640 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a56134640 1 -- 172.21.15.137:0/2109182600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 msgr2=0x55a4a586cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a56134640 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a586cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a56134640 1 -- 172.21.15.137:0/2109182600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a5778340 con 0x55a4a586d400 2024-09-12T12:31:17.269 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a55933640 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:31:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a56134640 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586d400 0x55a4a5877760 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009d20 tx=0x7f3a4c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcb3c5e30a4b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a477fe640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3a3c0a2e20 0x7f3a3c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a44bfe640 1 -- 172.21.15.137:0/2109182600 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c012650 con 0x55a4a586d400 2024-09-12T12:31:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a477fe640 1 -- 172.21.15.137:0/1122069911 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8528+0+0 (secure 0 0 0) 0x7f3a480cf8e0 con 0x7f3a50105520 2024-09-12T12:31:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a44bfe640 1 -- 172.21.15.137:0/2109182600 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a4c009e10 con 0x55a4a586d400 2024-09-12T12:31:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a44bfe640 1 -- 172.21.15.137:0/2109182600 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00fc70 con 0x55a4a586d400 2024-09-12T12:31:17.270 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.268+0000 7f3a55933640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3a3c0a2e20 0x7f3a3c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.269+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2109182600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586d400 msgr2=0x55a4a5877760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.269+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586d400 0x55a4a5877760 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009d20 tx=0x7f3a4c0117b0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.269+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2109182600 shutdown_connections 2024-09-12T12:31:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.269+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.272 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.269+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586d400 0x55a4a5877760 secure :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009d20 tx=0x7f3a4c0117b0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.269+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2109182600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a586cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.269+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2109182600 >> 172.21.15.137:0/2109182600 conn(0x55a4a5869050 msgr2=0x55a4a5869430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.271+0000 7f3a55933640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3a3c0a2e20 0x7f3a3c0a52e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005e00 tx=0x7f3a4002e040 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.271+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2109182600 shutdown_connections 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.272+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2109182600 wait complete. 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 -- start start 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 Processor -- start 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586cae0 0x55a4a5874d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a5875290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a58757d0 0x55a4a5883d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55a4a587a870 con 0x55a4a586cae0 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a4a575a290 con 0x55a4a5877ca0 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a4a587a9f0 con 0x55a4a58757d0 2024-09-12T12:31:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.273+0000 7f3a5b20f740 10 client.0 _ll_register_callbacks cb 0x55a4a578b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a5875290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a5875290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:49868/0 (socket says 172.21.15.137:49868) 2024-09-12T12:31:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 -- 172.21.15.137:0/3285536704 learned_addr learned my addr 172.21.15.137:0/3285536704 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a56134640 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586cae0 0x55a4a5874d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a56935640 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a58757d0 0x55a4a5883d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 -- 172.21.15.137:0/3285536704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a58757d0 msgr2=0x55a4a5883d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a58757d0 0x55a4a5883d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 -- 172.21.15.137:0/3285536704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586cae0 msgr2=0x55a4a5874d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586cae0 0x55a4a5874d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.274+0000 7f3a55933640 1 -- 172.21.15.137:0/3285536704 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a54e4210 con 0x55a4a5877ca0 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a55933640 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a5875290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a4002c8a0 tx=0x7f3a40000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf0e0060a4799f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a39ffb640 1 -- 172.21.15.137:0/3285536704 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40051060 con 0x55a4a5877ca0 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a39ffb640 1 -- 172.21.15.137:0/3285536704 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8528+0+0 (secure 0 0 0) 0x7f3a40005450 con 0x55a4a5877ca0 2024-09-12T12:31:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a4c009e10 con 0x55a4a5877ca0 2024-09-12T12:31:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a39ffb640 1 -- 172.21.15.137:0/3285536704 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3a40005650 con 0x55a4a5877ca0 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a5b20f740 10 client.26831 fetch_fsmap learned FSMap version 53 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55a4a55b0620 con 0x55a4a5877ca0 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.275+0000 7f3a39ffb640 1 -- 172.21.15.137:0/3285536704 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a4c009e10 con 0x55a4a5877ca0 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a39ffb640 1 -- 172.21.15.137:0/3285536704 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f3a40040960 con 0x55a4a5877ca0 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a5b20f740 10 client.26831 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a5b20f740 10 client.26831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a5b20f740 20 client.26831 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5899540 0x55a4a589ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a5b20f740 4 client.26831 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4a57dea20 con 0x55a4a5899540 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.276+0000 7f3a56134640 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5899540 0x55a4a589ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.277+0000 7f3a56134640 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5899540 0x55a4a589ba00 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd96713a89711e31 server_cookie=e757b8d942f63929 in_seq=0 out_seq=0 2024-09-12T12:31:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.277+0000 7f3a39ffb640 10 client.26831 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a39ffb640 1 -- 172.21.15.137:0/3285536704 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4a57dea20 con 0x55a4a5899540 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a39ffb640 10 client.26831 handle_command_reply: tid=0 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 client.26831 shutdown 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 msgr2=0x55a4a5875290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a5875290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a4002c8a0 tx=0x7f3a40000bc0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 shutdown_connections 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a58757d0 0x55a4a5883d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a5875290 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5899540 0x55a4a589ba00 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3285536704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a586cae0 0x55a4a5874d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 >> 172.21.15.137:0/3285536704 conn(0x55a4a5869050 msgr2=0x55a4a586edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a39ffb640 0 client.26831 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 shutdown_connections 2024-09-12T12:31:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3285536704 wait complete. 2024-09-12T12:31:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.282+0000 7f3a5b20f740 20 client.26831 trim_cache size 0 max 16384 2024-09-12T12:31:17.299 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:17 smithi137 ceph-mon[30950]: pgmap v573: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 624 op/s 2024-09-12T12:31:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.301+0000 7f3a5b20f740 1 Processor -- start 2024-09-12T12:31:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.301+0000 7f3a5b20f740 1 -- start start 2024-09-12T12:31:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.301+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a586cae0 0x55a4a5883b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a5868100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a58a1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a55933640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a5868100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a56134640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a586cae0 0x55a4a5883b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a55933640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a5868100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34618/0 (socket says 172.21.15.137:34618) 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a55933640 1 -- 172.21.15.137:0/3168499770 learned_addr learned my addr 172.21.15.137:0/3168499770 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a501adf80 con 0x55a4a58757d0 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a501ab850 con 0x55a4a5877ca0 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a50072340 con 0x55a4a586cae0 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a56935640 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a58a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a56134640 1 -- 172.21.15.137:0/3168499770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 msgr2=0x55a4a58a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a56134640 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a58a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a56134640 1 -- 172.21.15.137:0/3168499770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 msgr2=0x55a4a5868100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a56134640 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a5868100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a56134640 1 -- 172.21.15.137:0/3168499770 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4a586c910 con 0x55a4a586cae0 2024-09-12T12:31:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.302+0000 7f3a55933640 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a5868100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.303+0000 7f3a56134640 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a586cae0 0x55a4a5883b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55a4a5869fb0 tx=0x7f3a4c014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43fa532bbeec9966 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.303+0000 7f3a3bfff640 1 -- 172.21.15.137:0/3168499770 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00f640 con 0x55a4a586cae0 2024-09-12T12:31:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.303+0000 7f3a3bfff640 1 -- 172.21.15.137:0/3168499770 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a4c009e10 con 0x55a4a586cae0 2024-09-12T12:31:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.303+0000 7f3a3bfff640 1 -- 172.21.15.137:0/3168499770 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c0035f0 con 0x55a4a586cae0 2024-09-12T12:31:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a586cae0 msgr2=0x55a4a5883b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a586cae0 0x55a4a5883b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55a4a5869fb0 tx=0x7f3a4c014040 comp rx=0 tx=0).stop 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 shutdown_connections 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a5877ca0 0x55a4a58a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a5868100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/3168499770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a586cae0 0x55a4a5883b10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 >> 172.21.15.137:0/3168499770 conn(0x55a4a5869050 msgr2=0x55a4a58679f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 shutdown_connections 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.304+0000 7f3a5b20f740 1 -- 172.21.15.137:0/3168499770 wait complete. 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 -- start start 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 Processor -- start 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a587bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a587c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a500729b0 con 0x55a4a58757d0 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a56134640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a587bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a50073e20 con 0x55a4a586cae0 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.305+0000 7f3a5b20f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a50073080 con 0x55a4a5877ca0 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a56935640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a5b20f740 10 client.0 _ll_register_callbacks cb 0x7f3a4c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a56935640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:58956/0 (socket says 172.21.15.137:58956) 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a56134640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a587bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:49884/0 (socket says 172.21.15.137:49884) 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a56935640 1 -- 172.21.15.137:0/2307531378 learned_addr learned my addr 172.21.15.137:0/2307531378 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a55933640 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a587c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a55933640 1 -- 172.21.15.137:0/2307531378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 msgr2=0x55a4a587c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a55933640 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a55933640 1 -- 172.21.15.137:0/2307531378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 msgr2=0x55a4a587bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a55933640 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a587bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a55933640 1 -- 172.21.15.137:0/2307531378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4a5903480 con 0x55a4a58757d0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a56935640 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a55933640 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a587c150 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3a4002c7d0 tx=0x7f3a40032970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc911a56c282f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a39ffb640 1 -- 172.21.15.137:0/2307531378 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40051040 con 0x55a4a58757d0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.306+0000 7f3a39ffb640 1 -- 172.21.15.137:0/2307531378 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8528+0+0 (secure 0 0 0) 0x7f3a40055bf0 con 0x55a4a58757d0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a4c009e10 con 0x55a4a58757d0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a39ffb640 1 -- 172.21.15.137:0/2307531378 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f3a40055df0 con 0x55a4a58757d0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 10 client.17601 fetch_fsmap learned FSMap version 53 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55a4a5904770 con 0x55a4a58757d0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a39ffb640 1 -- 172.21.15.137:0/2307531378 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a4c009e10 con 0x55a4a58757d0 2024-09-12T12:31:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a39ffb640 1 -- 172.21.15.137:0/2307531378 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7f3a400047e0 con 0x55a4a58757d0 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 10 client.17601 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 10 client.17601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 20 client.17601 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5904e70 0x55a4a5907290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 4 client.17601 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a4a57d8d70 con 0x55a4a5904e70 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.307+0000 7f3a56134640 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5904e70 0x55a4a5907290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.308+0000 7f3a39ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.309+0000 7f3a56134640 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5904e70 0x55a4a5907290 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825920a6497ab33b server_cookie=6789e3effd8165d3 in_seq=0 out_seq=0 2024-09-12T12:31:17.340 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:17 smithi189 ceph-mon[35475]: pgmap v573: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 624 op/s 2024-09-12T12:31:17.450 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:17 smithi171 ceph-mon[35098]: pgmap v573: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 624 op/s 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.588+0000 7f3a39ffb640 1 -- 172.21.15.137:0/2307531378 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a4a57d8d70 con 0x55a4a5904e70 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.588+0000 7f3a39ffb640 10 client.17601 handle_command_reply: tid=0 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.588+0000 7f3a5b20f740 1 client.17601 shutdown 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.588+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 msgr2=0x55a4a587c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a587c150 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3a4002c7d0 tx=0x7f3a40032970 comp rx=0 tx=0).stop 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 shutdown_connections 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a4a5877ca0 0x55a4a587c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55a4a5904e70 0x55a4a5907290 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4a58757d0 0x55a4a587c150 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a39ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 --2- 172.21.15.137:0/2307531378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a4a586cae0 0x55a4a587bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 >> 172.21.15.137:0/2307531378 conn(0x55a4a5869050 msgr2=0x55a4a586ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 shutdown_connections 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.589+0000 7f3a5b20f740 1 -- 172.21.15.137:0/2307531378 wait complete. 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.590+0000 7f3a5b20f740 20 client.17601 trim_cache size 0 max 16384 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:31:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:31:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.590+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3a3c0a2e20 msgr2=0x7f3a3c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.590+0000 7f3a58398640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3a3c0a2e20 0x7f3a3c0a52e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005e00 tx=0x7f3a4002e040 comp rx=0 tx=0).stop 2024-09-12T12:31:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.590+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a50105520 msgr2=0x7f3a5010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.590+0000 7f3a58398640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a50105520 0x7f3a5010b8c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f3a48004410 tx=0x7f3a480048d0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 shutdown_connections 2024-09-12T12:31:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3a3c0a2e20 0x7f3a3c0a52e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a50105520 0x7f3a5010b8c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a501011e0 0x7f3a5010b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 --2- 172.21.15.137:0/1122069911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a50100830 0x7f3a5010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 >> 172.21.15.137:0/1122069911 conn(0x7f3a50075f20 msgr2=0x7f3a500fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 shutdown_connections 2024-09-12T12:31:17.593 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:17.591+0000 7f3a58398640 1 -- 172.21.15.137:0/1122069911 wait complete. 2024-09-12T12:31:18.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 109426 24.70 MB/sec execute 170 sec latency 229.103 ms 2024-09-12T12:31:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:18 smithi137 ceph-mon[30950]: pgmap v574: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 624 op/s 2024-09-12T12:31:19.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:18 smithi137 ceph-mon[30950]: osdmap e115: 12 total, 12 up, 12 in 2024-09-12T12:31:19.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:18 smithi189 ceph-mon[35475]: pgmap v574: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 624 op/s 2024-09-12T12:31:19.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:18 smithi189 ceph-mon[35475]: osdmap e115: 12 total, 12 up, 12 in 2024-09-12T12:31:19.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:18 smithi171 ceph-mon[35098]: pgmap v574: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 624 op/s 2024-09-12T12:31:19.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:18 smithi171 ceph-mon[35098]: osdmap e115: 12 total, 12 up, 12 in 2024-09-12T12:31:19.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 109823 24.63 MB/sec execute 171 sec latency 333.983 ms 2024-09-12T12:31:20.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 110536 24.75 MB/sec execute 172 sec latency 107.041 ms 2024-09-12T12:31:21.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:20 smithi137 ceph-mon[30950]: pgmap v576: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-09-12T12:31:21.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:20 smithi189 ceph-mon[35475]: pgmap v576: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-09-12T12:31:21.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:20 smithi171 ceph-mon[35098]: pgmap v576: 97 pgs: 97 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-09-12T12:31:21.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 111157 24.82 MB/sec execute 173 sec latency 203.982 ms 2024-09-12T12:31:22.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 111759 24.84 MB/sec execute 174 sec latency 137.722 ms 2024-09-12T12:31:23.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:22 smithi137 ceph-mon[30950]: pgmap v577: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 778 op/s 2024-09-12T12:31:23.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:22 smithi189 ceph-mon[35475]: pgmap v577: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 778 op/s 2024-09-12T12:31:23.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:22 smithi171 ceph-mon[35098]: pgmap v577: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 778 op/s 2024-09-12T12:31:23.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 112391 24.90 MB/sec execute 175 sec latency 105.892 ms 2024-09-12T12:31:24.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 112765 24.84 MB/sec execute 176 sec latency 257.694 ms 2024-09-12T12:31:25.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:24 smithi137 ceph-mon[30950]: pgmap v578: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 777 op/s 2024-09-12T12:31:25.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:24 smithi189 ceph-mon[35475]: pgmap v578: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 777 op/s 2024-09-12T12:31:25.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:24 smithi171 ceph-mon[35098]: pgmap v578: 97 pgs: 97 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 777 op/s 2024-09-12T12:31:25.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 113475 24.90 MB/sec execute 177 sec latency 150.182 ms 2024-09-12T12:31:26.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 114279 25.06 MB/sec execute 178 sec latency 74.419 ms 2024-09-12T12:31:27.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:26 smithi137 ceph-mon[30950]: pgmap v579: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 752 op/s 2024-09-12T12:31:27.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:26 smithi189 ceph-mon[35475]: pgmap v579: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 752 op/s 2024-09-12T12:31:27.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:26 smithi171 ceph-mon[35098]: pgmap v579: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 752 op/s 2024-09-12T12:31:27.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 114891 25.10 MB/sec execute 179 sec latency 41.337 ms 2024-09-12T12:31:28.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 115399 25.10 MB/sec execute 180 sec latency 222.798 ms 2024-09-12T12:31:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:28 smithi137 ceph-mon[30950]: pgmap v580: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 752 op/s 2024-09-12T12:31:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:28 smithi189 ceph-mon[35475]: pgmap v580: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 752 op/s 2024-09-12T12:31:29.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:28 smithi171 ceph-mon[35098]: pgmap v580: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 752 op/s 2024-09-12T12:31:29.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 115909 25.12 MB/sec execute 181 sec latency 278.325 ms 2024-09-12T12:31:30.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 116418 25.09 MB/sec execute 182 sec latency 106.402 ms 2024-09-12T12:31:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.702+0000 7ff88e6b7640 1 -- 172.21.15.137:0/1694153899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff888101350 msgr2=0x7ff88810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.702+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/1694153899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff888101350 0x7ff88810d880 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff880009cd0 tx=0x7ff88002a8e0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 -- 172.21.15.137:0/1694153899 shutdown_connections 2024-09-12T12:31:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/1694153899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff888101350 0x7ff88810d880 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/1694153899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881009b0 0x7ff888100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/1694153899 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8881069c0 0x7ff888106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 -- 172.21.15.137:0/1694153899 >> 172.21.15.137:0/1694153899 conn(0x7ff888077990 msgr2=0x7ff8880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 -- 172.21.15.137:0/1694153899 shutdown_connections 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 -- 172.21.15.137:0/1694153899 wait complete. 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 Processor -- start 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.703+0000 7ff88e6b7640 1 -- start start 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88e6b7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881009b0 0x7ff8881960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88e6b7640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff888101350 0x7ff888196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88e6b7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8881069c0 0x7ff88819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88e6b7640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff88810fd00 con 0x7ff8881069c0 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88e6b7640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff88810fb80 con 0x7ff888101350 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88e6b7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff88810fe80 con 0x7ff8881009b0 2024-09-12T12:31:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8881069c0 0x7ff88819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8881069c0 0x7ff88819a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48558/0 (socket says 172.21.15.137:48558) 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 -- 172.21.15.137:0/2985449456 learned_addr learned my addr 172.21.15.137:0/2985449456 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff887fff640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881009b0 0x7ff8881960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff8877fe640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff888101350 0x7ff888196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 -- 172.21.15.137:0/2985449456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881009b0 msgr2=0x7ff8881960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881009b0 0x7ff8881960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 -- 172.21.15.137:0/2985449456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff888101350 msgr2=0x7ff888196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff888101350 0x7ff888196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 -- 172.21.15.137:0/2985449456 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88819b0c0 con 0x7ff8881069c0 2024-09-12T12:31:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88cc54640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8881069c0 0x7ff88819a990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff880005cc0 tx=0x7ff8800093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff8857fa640 1 -- 172.21.15.137:0/2985449456 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880003e00 con 0x7ff8881069c0 2024-09-12T12:31:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff8857fa640 1 -- 172.21.15.137:0/2985449456 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880004520 con 0x7ff8881069c0 2024-09-12T12:31:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88819b350 con 0x7ff8881069c0 2024-09-12T12:31:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.704+0000 7ff8857fa640 1 -- 172.21.15.137:0/2985449456 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880030db0 con 0x7ff8881069c0 2024-09-12T12:31:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.705+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a2c40 con 0x7ff8881069c0 2024-09-12T12:31:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.707+0000 7ff891452740 1 Processor -- start 2024-09-12T12:31:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.707+0000 7ff8857fa640 1 -- 172.21.15.137:0/2985449456 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff8800046d0 con 0x7ff8881069c0 2024-09-12T12:31:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff891452740 1 -- start start 2024-09-12T12:31:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff891452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610065ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff891452740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e610070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff891452740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610070cb0 0x55e6100730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff891452740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e60ffca2f0 con 0x55e610065af0 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff891452740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e60ff72270 con 0x55e610066410 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff891452740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e60ffc62b0 con 0x55e610070cb0 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e610070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff8877fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610065ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e610070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55264/0 (socket says 172.21.15.137:55264) 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 -- 172.21.15.137:0/3066215966 learned_addr learned my addr 172.21.15.137:0/3066215966 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff8877fe640 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610070cb0 0x55e6100730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 -- 172.21.15.137:0/3066215966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610070cb0 msgr2=0x55e6100730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610070cb0 0x55e6100730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 -- 172.21.15.137:0/3066215966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 msgr2=0x55e610065ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610065ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff887fff640 1 -- 172.21.15.137:0/3066215966 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60ff71340 con 0x55e610066410 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff8877fe640 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610065ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.708+0000 7ff8857fa640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8700a2da0 0x7ff8700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.709+0000 7ff887fff640 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e610070770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff878009c70 tx=0x7ff878011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b380769e9fd82aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.709+0000 7ff8877fe640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8700a2da0 0x7ff8700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.709+0000 7ff8857fa640 1 -- 172.21.15.137:0/2985449456 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8484+0+0 (secure 0 0 0) 0x7ff88002e030 con 0x7ff8881069c0 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.709+0000 7ff86e7fc640 1 -- 172.21.15.137:0/3066215966 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878004230 con 0x55e610066410 2024-09-12T12:31:30.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.709+0000 7ff86e7fc640 1 -- 172.21.15.137:0/3066215966 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8780043d0 con 0x55e610066410 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.709+0000 7ff86e7fc640 1 -- 172.21.15.137:0/3066215966 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780197c0 con 0x55e610066410 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 -- 172.21.15.137:0/3066215966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 msgr2=0x55e610070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e610070770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff878009c70 tx=0x7ff878011810 comp rx=0 tx=0).stop 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff8877fe640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8700a2da0 0x7ff8700a5260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004650 tx=0x7ff87c005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 -- 172.21.15.137:0/3066215966 shutdown_connections 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610070cb0 0x55e6100730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e610070770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 --2- 172.21.15.137:0/3066215966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610065ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 -- 172.21.15.137:0/3066215966 >> 172.21.15.137:0/3066215966 conn(0x55e610062060 msgr2=0x55e610062440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 -- 172.21.15.137:0/3066215966 shutdown_connections 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 -- 172.21.15.137:0/3066215966 wait complete. 2024-09-12T12:31:30.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.710+0000 7ff891452740 1 -- start start 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 1 Processor -- start 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610065af0 0x55e61006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e61006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e61006e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55e610073880 con 0x55e610070cb0 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e60ff53290 con 0x55e610066410 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e610073a00 con 0x55e610065af0 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff891452740 10 client.0 _ll_register_callbacks cb 0x55e60ff84140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff8877fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e61006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.719 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff8877fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e61006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55268/0 (socket says 172.21.15.137:55268) 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff88cc54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e61006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff8877fe640 1 -- 172.21.15.137:0/3430232556 learned_addr learned my addr 172.21.15.137:0/3430232556 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff887fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610065af0 0x55e61006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff8877fe640 1 -- 172.21.15.137:0/3430232556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610065af0 msgr2=0x55e61006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff8877fe640 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610065af0 0x55e61006dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff8877fe640 1 -- 172.21.15.137:0/3430232556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 msgr2=0x55e61006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.711+0000 7ff8877fe640 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e61006e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff8877fe640 1 -- 172.21.15.137:0/3430232556 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60fcdd210 con 0x55e610066410 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff88cc54640 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e61006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff8877fe640 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e61006e000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0043b0 tx=0x7ff87c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a44cf856e272a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:30.720 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff8677fe640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff8677fe640 1 -- 172.21.15.137:0/3430232556 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c01fd20 con 0x55e610066410 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff8677fe640 1 -- 172.21.15.137:0/3430232556 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8484+0+0 (secure 0 0 0) 0x7ff87c0293f0 con 0x55e610066410 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8780043d0 con 0x55e610066410 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff8677fe640 1 -- 172.21.15.137:0/3430232556 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff87c037050 con 0x55e610066410 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff891452740 10 client.26849 fetch_fsmap learned FSMap version 53 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55e60fda9620 con 0x55e610066410 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.712+0000 7ff8677fe640 1 -- 172.21.15.137:0/3430232556 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8780043d0 con 0x55e610066410 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff8677fe640 1 -- 172.21.15.137:0/3430232556 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ff87c01c040 con 0x55e610066410 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff891452740 10 client.26849 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff891452740 10 client.26849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff891452740 20 client.26849 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:30.721 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff891452740 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610083330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff891452740 4 client.26849 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60ffd7a20 con 0x55e610080f40 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.713+0000 7ff887fff640 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610083330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.714+0000 7ff8677fe640 10 client.26849 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.714+0000 7ff887fff640 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610083330 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd96d4443c5beb3 server_cookie=644d39f083012c in_seq=0 out_seq=0 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff8677fe640 1 -- 172.21.15.137:0/3430232556 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e60ffd7a20 con 0x55e610080f40 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff8677fe640 10 client.26849 handle_command_reply: tid=0 2024-09-12T12:31:30.722 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 client.26849 shutdown 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 msgr2=0x55e61006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e61006e000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0043b0 tx=0x7ff87c005f90 comp rx=0 tx=0).stop 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 shutdown_connections 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610083330 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff8677fe640 0 client.26849 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e61006e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610066410 0x55e61006e000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 --2- 172.21.15.137:0/3430232556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610065af0 0x55e61006dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.723 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 >> 172.21.15.137:0/3430232556 conn(0x55e610062060 msgr2=0x55e610067dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.715+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 shutdown_connections 2024-09-12T12:31:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.716+0000 7ff891452740 1 -- 172.21.15.137:0/3430232556 wait complete. 2024-09-12T12:31:30.724 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.716+0000 7ff891452740 20 client.26849 trim_cache size 0 max 16384 2024-09-12T12:31:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 Processor -- start 2024-09-12T12:31:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 -- start start 2024-09-12T12:31:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610068560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e61006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610070cb0 0x55e610088d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881add30 con 0x55e610065af0 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881ab600 con 0x55e610070cb0 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff887fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610068560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff891452740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a9470 con 0x55e610066410 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff8877fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e61006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff887fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610068560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48600/0 (socket says 172.21.15.137:48600) 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff887fff640 1 -- 172.21.15.137:0/1316425716 learned_addr learned my addr 172.21.15.137:0/1316425716 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.733+0000 7ff88cc54640 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610070cb0 0x55e610088d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.734+0000 7ff8877fe640 1 -- 172.21.15.137:0/1316425716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610070cb0 msgr2=0x55e610088d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.734+0000 7ff8877fe640 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610070cb0 0x55e610088d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.734+0000 7ff8877fe640 1 -- 172.21.15.137:0/1316425716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 msgr2=0x55e610068560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.734+0000 7ff8877fe640 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610068560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.734+0000 7ff8877fe640 1 -- 172.21.15.137:0/1316425716 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e610061e90 con 0x55e610066410 2024-09-12T12:31:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.734+0000 7ff8877fe640 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e61006e370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff87c029fd0 tx=0x7ff87c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6111337fdc54cd2a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.734+0000 7ff86dffb640 1 -- 172.21.15.137:0/1316425716 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c040070 con 0x55e610066410 2024-09-12T12:31:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.735+0000 7ff86dffb640 1 -- 172.21.15.137:0/1316425716 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c02e070 con 0x55e610066410 2024-09-12T12:31:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.735+0000 7ff86dffb640 1 -- 172.21.15.137:0/1316425716 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c032c40 con 0x55e610066410 2024-09-12T12:31:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.735+0000 7ff891452740 1 -- 172.21.15.137:0/1316425716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 msgr2=0x55e61006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.735+0000 7ff891452740 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e61006e370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff87c029fd0 tx=0x7ff87c005740 comp rx=0 tx=0).stop 2024-09-12T12:31:30.739 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 -- 172.21.15.137:0/1316425716 shutdown_connections 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610070cb0 0x55e610088d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e61006e370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 --2- 172.21.15.137:0/1316425716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610065af0 0x55e610068560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 -- 172.21.15.137:0/1316425716 >> 172.21.15.137:0/1316425716 conn(0x55e610062060 msgr2=0x55e61006fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 -- 172.21.15.137:0/1316425716 shutdown_connections 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 -- 172.21.15.137:0/1316425716 wait complete. 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.736+0000 7ff891452740 1 -- start start 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff891452740 1 Processor -- start 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff891452740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610065af0 0x55e610079290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff891452740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e6100745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff891452740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e610074af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff891452740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881a9ac0 con 0x55e610070cb0 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff891452740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881aaf30 con 0x55e610065af0 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff891452740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881aa190 con 0x55e610066410 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.737+0000 7ff88cc54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e610074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e610074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48616/0 (socket says 172.21.15.137:48616) 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 -- 172.21.15.137:0/3758902508 learned_addr learned my addr 172.21.15.137:0/3758902508 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff8877fe640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e6100745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff887fff640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610065af0 0x55e610079290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.741 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff891452740 10 client.0 _ll_register_callbacks cb 0x7ff87c0042c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 -- 172.21.15.137:0/3758902508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 msgr2=0x55e6100745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e6100745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 -- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610065af0 msgr2=0x55e610079290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610065af0 0x55e610079290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 -- 172.21.15.137:0/3758902508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e610075190 con 0x55e610070cb0 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff8877fe640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e6100745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff88cc54640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e610074af0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff8800ee6c0 tx=0x7ff880030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303fa50d03dc118c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.738+0000 7ff8677fe640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.740+0000 7ff8677fe640 1 -- 172.21.15.137:0/3758902508 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8800bcfa0 con 0x55e610070cb0 2024-09-12T12:31:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff87c02e070 con 0x55e610070cb0 2024-09-12T12:31:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff8677fe640 1 -- 172.21.15.137:0/3758902508 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8484+0+0 (secure 0 0 0) 0x7ff8800c5780 con 0x55e610070cb0 2024-09-12T12:31:30.743 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff8677fe640 1 -- 172.21.15.137:0/3758902508 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff8800c1350 con 0x55e610070cb0 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 10 client.17619 fetch_fsmap learned FSMap version 53 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55e6100ef810 con 0x55e610070cb0 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff8677fe640 1 -- 172.21.15.137:0/3758902508 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff87c02e070 con 0x55e610070cb0 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff8677fe640 1 -- 172.21.15.137:0/3758902508 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 53) ==== 8346+0+0 (secure 0 0 0) 0x7ff88004e070 con 0x55e610070cb0 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 10 client.17619 fetch_fsmap finished waiting for FSMap version 53 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 10 client.17619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 20 client.17619 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610085c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 4 client.17619 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e60ffd1d70 con 0x55e610080f40 2024-09-12T12:31:30.744 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.741+0000 7ff887fff640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610085c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.742+0000 7ff8677fe640 10 client.17619 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:30.742+0000 7ff887fff640 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610085c50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b31cec779368bc6 server_cookie=2daa3801cddde8b8 in_seq=0 out_seq=0 2024-09-12T12:31:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.032+0000 7ff8677fe640 1 -- 172.21.15.137:0/3758902508 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e60ffd1d70 con 0x55e610080f40 2024-09-12T12:31:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.032+0000 7ff8677fe640 10 client.17619 handle_command_reply: tid=0 2024-09-12T12:31:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.032+0000 7ff891452740 1 client.17619 shutdown 2024-09-12T12:31:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 msgr2=0x55e610074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e610074af0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff8800ee6c0 tx=0x7ff880030fd0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 shutdown_connections 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55e610080f40 0x55e610085c50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e610070cb0 0x55e610074af0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff8677fe640 0 client.17619 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e610066410 0x55e6100745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 --2- 172.21.15.137:0/3758902508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e610065af0 0x55e610079290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 >> 172.21.15.137:0/3758902508 conn(0x55e610062060 msgr2=0x55e6100609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 shutdown_connections 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 1 -- 172.21.15.137:0/3758902508 wait complete. 2024-09-12T12:31:31.035 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.033+0000 7ff891452740 20 client.17619 trim_cache size 0 max 16384 2024-09-12T12:31:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:31:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:31:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:31:31.036 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:31:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.034+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8700a2da0 msgr2=0x7ff8700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:31.037 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.034+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8700a2da0 0x7ff8700a5260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff87c004650 tx=0x7ff87c005850 comp rx=0 tx=0).stop 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.034+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8881069c0 msgr2=0x7ff88819a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8881069c0 0x7ff88819a990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff880005cc0 tx=0x7ff8800093d0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 shutdown_connections 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7ff8700a2da0 0x7ff8700a5260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8881069c0 0x7ff88819a990 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff888101350 0x7ff888196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 --2- 172.21.15.137:0/2985449456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8881009b0 0x7ff8881960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 >> 172.21.15.137:0/2985449456 conn(0x7ff888077990 msgr2=0x7ff8880fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 shutdown_connections 2024-09-12T12:31:31.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:31.035+0000 7ff88e6b7640 1 -- 172.21.15.137:0/2985449456 wait complete. 2024-09-12T12:31:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:30 smithi137 ceph-mon[30950]: pgmap v581: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 733 op/s 2024-09-12T12:31:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:30 smithi189 ceph-mon[35475]: pgmap v581: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 733 op/s 2024-09-12T12:31:31.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:30 smithi171 ceph-mon[35098]: pgmap v581: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 733 op/s 2024-09-12T12:31:31.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 117014 25.14 MB/sec execute 183 sec latency 284.499 ms 2024-09-12T12:31:32.091 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:31:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:31:32.091 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:31:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:31:32.091 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:31:31 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/3754380202 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55732ff5c680 con 0x55732ff31000 2024-09-12T12:31:32.091 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:31:31 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/3754380202 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55732ff53860 con 0x55732ff31000 2024-09-12T12:31:32.091 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:31:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:31:32.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 117751 25.25 MB/sec execute 184 sec latency 75.414 ms 2024-09-12T12:31:33.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:32 smithi171 ceph-mon[35098]: pgmap v582: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 784 op/s 2024-09-12T12:31:33.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:32 smithi171 ceph-mon[35098]: daemon mds.f restarted 2024-09-12T12:31:33.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:32 smithi171 ceph-mon[35098]: osdmap e116: 12 total, 12 up, 12 in 2024-09-12T12:31:33.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:32 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6834/4049352466,v1:172.21.15.189:6835/4049352466] up:boot 2024-09-12T12:31:33.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:32 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:31:33.001 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:32 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:31:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:32 smithi137 ceph-mon[30950]: pgmap v582: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 784 op/s 2024-09-12T12:31:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:32 smithi137 ceph-mon[30950]: daemon mds.f restarted 2024-09-12T12:31:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:32 smithi137 ceph-mon[30950]: osdmap e116: 12 total, 12 up, 12 in 2024-09-12T12:31:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:32 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6834/4049352466,v1:172.21.15.189:6835/4049352466] up:boot 2024-09-12T12:31:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:32 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:31:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:32 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:31:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:32 smithi189 ceph-mon[35475]: pgmap v582: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 784 op/s 2024-09-12T12:31:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:32 smithi189 ceph-mon[35475]: daemon mds.f restarted 2024-09-12T12:31:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:32 smithi189 ceph-mon[35475]: osdmap e116: 12 total, 12 up, 12 in 2024-09-12T12:31:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:32 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6834/4049352466,v1:172.21.15.189:6835/4049352466] up:boot 2024-09-12T12:31:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:32 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:31:33.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:32 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:31:33.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 118349 25.28 MB/sec execute 185 sec latency 112.982 ms 2024-09-12T12:31:34.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 118821 25.29 MB/sec execute 186 sec latency 233.040 ms 2024-09-12T12:31:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:34 smithi137 ceph-mon[30950]: pgmap v584: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-09-12T12:31:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:34 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:31:35.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:34 smithi189 ceph-mon[35475]: pgmap v584: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-09-12T12:31:35.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:34 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:31:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:34 smithi171 ceph-mon[35098]: pgmap v584: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 623 op/s 2024-09-12T12:31:35.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:34 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:31:35.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 119352 25.31 MB/sec execute 187 sec latency 104.421 ms 2024-09-12T12:31:36.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:35 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:31:36.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:35 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:31:36.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:35 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:31:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:35 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:31:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:35 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:31:36.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:35 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:31:36.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:35 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:31:36.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:35 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:31:36.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:35 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:31:36.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 119974 25.32 MB/sec execute 188 sec latency 117.182 ms 2024-09-12T12:31:37.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 120615 25.40 MB/sec execute 189 sec latency 92.094 ms 2024-09-12T12:31:37.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:37 smithi137 ceph-mon[30950]: pgmap v585: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-12T12:31:37.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:37 smithi189 ceph-mon[35475]: pgmap v585: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-12T12:31:37.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:37 smithi171 ceph-mon[35098]: pgmap v585: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-12T12:31:38.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 121221 25.41 MB/sec execute 190 sec latency 106.640 ms 2024-09-12T12:31:39.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 121821 25.45 MB/sec execute 191 sec latency 325.165 ms 2024-09-12T12:31:39.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:39 smithi137 ceph-mon[30950]: pgmap v586: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-12T12:31:39.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:39 smithi189 ceph-mon[35475]: pgmap v586: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-12T12:31:39.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:39 smithi171 ceph-mon[35098]: pgmap v586: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 698 op/s 2024-09-12T12:31:40.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 122493 25.53 MB/sec execute 192 sec latency 91.992 ms 2024-09-12T12:31:41.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 123128 25.57 MB/sec execute 193 sec latency 69.866 ms 2024-09-12T12:31:41.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:41 smithi137 ceph-mon[30950]: pgmap v587: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.8 MiB/s wr, 653 op/s 2024-09-12T12:31:41.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:41 smithi189 ceph-mon[35475]: pgmap v587: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.8 MiB/s wr, 653 op/s 2024-09-12T12:31:41.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:41 smithi171 ceph-mon[35098]: pgmap v587: 97 pgs: 97 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.8 MiB/s wr, 653 op/s 2024-09-12T12:31:42.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 123754 25.63 MB/sec execute 194 sec latency 222.585 ms 2024-09-12T12:31:43.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 124335 25.64 MB/sec execute 195 sec latency 78.724 ms 2024-09-12T12:31:43.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:43 smithi137 ceph-mon[30950]: pgmap v588: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-09-12T12:31:43.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:43 smithi189 ceph-mon[35475]: pgmap v588: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-09-12T12:31:43.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:43 smithi171 ceph-mon[35098]: pgmap v588: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 698 op/s 2024-09-12T12:31:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 -- 172.21.15.137:0/172083778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc1009d0 msgr2=0x7f35fc100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.179 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 --2- 172.21.15.137:0/172083778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc1009d0 0x7f35fc100e30 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009860 tx=0x7f35f002a340 comp rx=0 tx=0).stop 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 -- 172.21.15.137:0/172083778 shutdown_connections 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 --2- 172.21.15.137:0/172083778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35fc101370 0x7f35fc10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 --2- 172.21.15.137:0/172083778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc1009d0 0x7f35fc100e30 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 --2- 172.21.15.137:0/172083778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35fc1069e0 0x7f35fc106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 -- 172.21.15.137:0/172083778 >> 172.21.15.137:0/172083778 conn(0x7f35fc0fc6b0 msgr2=0x7f35fc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 -- 172.21.15.137:0/172083778 shutdown_connections 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.178+0000 7f360154b640 1 -- 172.21.15.137:0/172083778 wait complete. 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 Processor -- start 2024-09-12T12:31:44.180 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 -- start start 2024-09-12T12:31:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35fc1009d0 0x7f35fc1960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc101370 0x7f35fc196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35fc1069e0 0x7f35fc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc10fce0 con 0x7f35fc101370 2024-09-12T12:31:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc10fb60 con 0x7f35fc1069e0 2024-09-12T12:31:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.179+0000 7f360154b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc10fe60 con 0x7f35fc1009d0 2024-09-12T12:31:44.182 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc101370 0x7f35fc196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc101370 0x7f35fc196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41654/0 (socket says 172.21.15.137:41654) 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 -- 172.21.15.137:0/3741350233 learned_addr learned my addr 172.21.15.137:0/3741350233 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35fc1069e0 0x7f35fc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 -- 172.21.15.137:0/3741350233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35fc1009d0 msgr2=0x7f35fc1960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35fc1009d0 0x7f35fc1960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 -- 172.21.15.137:0/3741350233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35fc1069e0 msgr2=0x7f35fc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35fc1069e0 0x7f35fc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 -- 172.21.15.137:0/3741350233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc19b0e0 con 0x7f35fc101370 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35fc1069e0 0x7f35fc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.180+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc101370 0x7f35fc196630 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f35f0035040 tx=0x7f35f0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.181+0000 7f35e3fff640 1 -- 172.21.15.137:0/3741350233 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f003f020 con 0x7f35fc101370 2024-09-12T12:31:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.181+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc19b370 con 0x7f35fc101370 2024-09-12T12:31:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.181+0000 7f35e3fff640 1 -- 172.21.15.137:0/3741350233 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f0004240 con 0x7f35fc101370 2024-09-12T12:31:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.181+0000 7f35e3fff640 1 -- 172.21.15.137:0/3741350233 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0037040 con 0x7f35fc101370 2024-09-12T12:31:44.184 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.181+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc19b7a0 con 0x7f35fc101370 2024-09-12T12:31:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f35e3fff640 1 -- 172.21.15.137:0/3741350233 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f35f0042020 con 0x7f35fc101370 2024-09-12T12:31:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 Processor -- start 2024-09-12T12:31:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 -- start start 2024-09-12T12:31:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c771fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c7720410 0x5613c772a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c772d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5613c76842f0 con 0x5613c772acb0 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613c762c270 con 0x5613c771faf0 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f3603f22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613c76802b0 con 0x5613c7720410 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f35fa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c772d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f35fb7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c771fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f35faffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c7720410 0x5613c772a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f35fa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c772d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41664/0 (socket says 172.21.15.137:41664) 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f35fa7fc640 1 -- 172.21.15.137:0/4202752640 learned_addr learned my addr 172.21.15.137:0/4202752640 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.184+0000 7f35fb7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c771fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:47024/0 (socket says 172.21.15.137:47024) 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35e3fff640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f35e40a2da0 0x7f35e40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fa7fc640 1 -- 172.21.15.137:0/4202752640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c7720410 msgr2=0x5613c772a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.187 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c7720410 0x5613c772a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fa7fc640 1 -- 172.21.15.137:0/4202752640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 msgr2=0x5613c771fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c771fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35e3fff640 1 -- 172.21.15.137:0/3741350233 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f35f00eef60 con 0x7f35fc101370 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fa7fc640 1 -- 172.21.15.137:0/4202752640 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c762b340 con 0x5613c772acb0 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c771fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f35e40a2da0 0x7f35e40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c772d0a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f35f00ec780 tx=0x7f35f0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59297cf8011d7eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35e17fa640 1 -- 172.21.15.137:0/4202752640 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f00bb3b0 con 0x5613c772acb0 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35e17fa640 1 -- 172.21.15.137:0/4202752640 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f00ece50 con 0x5613c772acb0 2024-09-12T12:31:44.188 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.185+0000 7f35e17fa640 1 -- 172.21.15.137:0/4202752640 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f00baf40 con 0x5613c772acb0 2024-09-12T12:31:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 -- 172.21.15.137:0/4202752640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 msgr2=0x5613c772d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f35e40a2da0 0x7f35e40a5260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005fd0 tx=0x7f35f4004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c772d0a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f35f00ec780 tx=0x7f35f0005dc0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 -- 172.21.15.137:0/4202752640 shutdown_connections 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c772d0a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c7720410 0x5613c772a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 --2- 172.21.15.137:0/4202752640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c771fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 -- 172.21.15.137:0/4202752640 >> 172.21.15.137:0/4202752640 conn(0x5613c771c060 msgr2=0x5613c771c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 -- 172.21.15.137:0/4202752640 shutdown_connections 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.186+0000 7f3603f22740 1 -- 172.21.15.137:0/4202752640 wait complete. 2024-09-12T12:31:44.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 -- start start 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 Processor -- start 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c7727b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c7720410 0x5613c7728060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c77285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5613c772d640 con 0x5613c7720410 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5613c760d290 con 0x5613c771faf0 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5613c772d7c0 con 0x5613c772acb0 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f3603f22740 10 client.0 _ll_register_callbacks cb 0x5613c763e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f35faffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c7727b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f35faffd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c7727b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:47040/0 (socket says 172.21.15.137:47040) 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f35fa7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c7720410 0x5613c7728060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.187+0000 7f35faffd640 1 -- 172.21.15.137:0/1354643858 learned_addr learned my addr 172.21.15.137:0/1354643858 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:44.193 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.188+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c77285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.188+0000 7f35faffd640 1 -- 172.21.15.137:0/1354643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 msgr2=0x5613c77285a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.188+0000 7f35faffd640 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c77285a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.188+0000 7f35faffd640 1 -- 172.21.15.137:0/1354643858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c7720410 msgr2=0x5613c7728060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.188+0000 7f35faffd640 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c7720410 0x5613c7728060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.188+0000 7f35faffd640 1 -- 172.21.15.137:0/1354643858 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c7397210 con 0x5613c771faf0 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.188+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c77285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f35faffd640 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c7727b20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d5f0 tx=0x7f35ec00df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b399d6b8a35b403f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f35deff5640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f35deff5640 1 -- 172.21.15.137:0/1354643858 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec016e00 con 0x5613c771faf0 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f00ece50 con 0x5613c771faf0 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f35deff5640 1 -- 172.21.15.137:0/1354643858 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f35ec003d60 con 0x5613c771faf0 2024-09-12T12:31:44.194 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f35deff5640 1 -- 172.21.15.137:0/1354643858 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f35ec005020 con 0x5613c771faf0 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f3603f22740 10 client.26870 fetch_fsmap learned FSMap version 54 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.189+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5613c7463620 con 0x5613c771faf0 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f35deff5640 1 -- 172.21.15.137:0/1354643858 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f00ece50 con 0x5613c771faf0 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f35deff5640 1 -- 172.21.15.137:0/1354643858 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f35ec020480 con 0x5613c771faf0 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f3603f22740 10 client.26870 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f3603f22740 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f3603f22740 20 client.26870 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f3603f22740 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c773aa80 0x5613c773cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f3603f22740 4 client.26870 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613c7691a20 con 0x5613c773aa80 2024-09-12T12:31:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.190+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c773aa80 0x5613c773cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.191+0000 7f35deff5640 10 client.26870 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:44.196 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.191+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c773aa80 0x5613c773cea0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888782a6cf094f37 server_cookie=5f91af90acf55b8b in_seq=0 out_seq=0 2024-09-12T12:31:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.194+0000 7f35deff5640 1 -- 172.21.15.137:0/1354643858 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613c7691a20 con 0x5613c773aa80 2024-09-12T12:31:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.194+0000 7f35deff5640 10 client.26870 handle_command_reply: tid=0 2024-09-12T12:31:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.194+0000 7f3603f22740 1 client.26870 shutdown 2024-09-12T12:31:44.197 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.194+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 msgr2=0x5613c7727b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c7727b20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d5f0 tx=0x7f35ec00df90 comp rx=0 tx=0).stop 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 shutdown_connections 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c77285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c773aa80 0x5613c773cea0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c7720410 0x5613c7728060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f35deff5640 0 client.26870 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 --2- 172.21.15.137:0/1354643858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c771faf0 0x5613c7727b20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 >> 172.21.15.137:0/1354643858 conn(0x5613c771c060 msgr2=0x5613c7721dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:44.198 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 shutdown_connections 2024-09-12T12:31:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 1 -- 172.21.15.137:0/1354643858 wait complete. 2024-09-12T12:31:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.195+0000 7f3603f22740 20 client.26870 trim_cache size 0 max 16384 2024-09-12T12:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.210+0000 7f3603f22740 1 Processor -- start 2024-09-12T12:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.210+0000 7f3603f22740 1 -- start start 2024-09-12T12:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f3603f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c771faf0 0x5613c7728050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f3603f22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c7728590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f3603f22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c77427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f3603f22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1add80 con 0x5613c772acb0 2024-09-12T12:31:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f3603f22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1ab650 con 0x5613c7720410 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f3603f22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1a94a0 con 0x5613c771faf0 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c77427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c77427b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41684/0 (socket says 172.21.15.137:41684) 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 -- 172.21.15.137:0/1109812847 learned_addr learned my addr 172.21.15.137:0/1109812847 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35faffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c771faf0 0x5613c7728050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c7728590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 -- 172.21.15.137:0/1109812847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c771faf0 msgr2=0x5613c7728050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c771faf0 0x5613c7728050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 -- 172.21.15.137:0/1109812847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 msgr2=0x5613c7728590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c7728590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 -- 172.21.15.137:0/1109812847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613c7739510 con 0x5613c772acb0 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c77427b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f35f400f7d0 tx=0x7f35f4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f6b0df2bd39164 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:44.213 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35e0ff9640 1 -- 172.21.15.137:0/1109812847 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f401c430 con 0x5613c772acb0 2024-09-12T12:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35e0ff9640 1 -- 172.21.15.137:0/1109812847 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f401cab0 con 0x5613c772acb0 2024-09-12T12:31:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.211+0000 7f35e0ff9640 1 -- 172.21.15.137:0/1109812847 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f401aa60 con 0x5613c772acb0 2024-09-12T12:31:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 -- 172.21.15.137:0/1109812847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 msgr2=0x5613c77427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c77427b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f35f400f7d0 tx=0x7f35f4015000 comp rx=0 tx=0).stop 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 -- 172.21.15.137:0/1109812847 shutdown_connections 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c772acb0 0x5613c77427b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c7728590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 --2- 172.21.15.137:0/1109812847 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c771faf0 0x5613c7728050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 -- 172.21.15.137:0/1109812847 >> 172.21.15.137:0/1109812847 conn(0x5613c771c060 msgr2=0x5613c7729ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 -- 172.21.15.137:0/1109812847 shutdown_connections 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 -- 172.21.15.137:0/1109812847 wait complete. 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.212+0000 7f3603f22740 1 -- start start 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 1 Processor -- start 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c771faf0 0x5613c7732f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c772e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c772e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c771faf0 0x5613c7732f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1a9b10 con 0x5613c771faf0 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1aaf80 con 0x5613c7720410 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c771faf0 0x5613c7732f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41690/0 (socket says 172.21.15.137:41690) 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc1aa1e0 con 0x5613c772acb0 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 -- 172.21.15.137:0/2695408970 learned_addr learned my addr 172.21.15.137:0/2695408970 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f3603f22740 10 client.0 _ll_register_callbacks cb 0x7f35ec00d440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c772e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35fb7fe640 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c772e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 -- 172.21.15.137:0/2695408970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 msgr2=0x5613c772e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c772e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 -- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 msgr2=0x5613c772e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c772e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.213+0000 7f35faffd640 1 -- 172.21.15.137:0/2695408970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613c772ed30 con 0x5613c771faf0 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.215+0000 7f35faffd640 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c771faf0 0x5613c7732f10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d5f0 tx=0x7f35ec004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b04ad32542b4b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.215+0000 7f35deff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.215+0000 7f35deff5640 1 -- 172.21.15.137:0/2695408970 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec020680 con 0x5613c771faf0 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.215+0000 7f35deff5640 1 -- 172.21.15.137:0/2695408970 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f35ec020ae0 con 0x5613c771faf0 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.215+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f401cab0 con 0x5613c771faf0 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.215+0000 7f35deff5640 1 -- 172.21.15.137:0/2695408970 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f35ec020ce0 con 0x5613c771faf0 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f3603f22740 10 client.17643 fetch_fsmap learned FSMap version 54 2024-09-12T12:31:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5613c77a5520 con 0x5613c771faf0 2024-09-12T12:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f35deff5640 1 -- 172.21.15.137:0/2695408970 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f401cab0 con 0x5613c771faf0 2024-09-12T12:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f35deff5640 1 -- 172.21.15.137:0/2695408970 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f35ec004c30 con 0x5613c771faf0 2024-09-12T12:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f3603f22740 10 client.17643 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f3603f22740 10 client.17643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f3603f22740 20 client.17643 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.216+0000 7f3603f22740 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c77a5bd0 0x5613c77a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.217+0000 7f3603f22740 4 client.17643 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:31:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.217+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613c768bd70 con 0x5613c77a5bd0 2024-09-12T12:31:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.217+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c77a5bd0 0x5613c77a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.218+0000 7f35fa7fc640 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c77a5bd0 0x5613c77a7ff0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c90d7f1606065db server_cookie=868ac1b0e53ba411 in_seq=0 out_seq=0 2024-09-12T12:31:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.218+0000 7f35deff5640 10 client.17643 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:44.229 INFO:tasks.workunit.client.0.smithi137.stdout: 10 124887 25.63 MB/sec execute 196 sec latency 246.765 ms 2024-09-12T12:31:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f35deff5640 1 -- 172.21.15.137:0/2695408970 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613c768bd70 con 0x5613c77a5bd0 2024-09-12T12:31:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f35deff5640 10 client.17643 handle_command_reply: tid=0 2024-09-12T12:31:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 client.17643 shutdown 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c771faf0 msgr2=0x5613c7732f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c771faf0 0x5613c7732f10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00d5f0 tx=0x7f35ec004320 comp rx=0 tx=0).stop 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 shutdown_connections 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613c772acb0 0x5613c772e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5613c7720410 0x5613c772e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5613c77a5bd0 0x5613c77a7ff0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 --2- 172.21.15.137:0/2695408970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613c771faf0 0x5613c7732f10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f35deff5640 0 client.17643 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 >> 172.21.15.137:0/2695408970 conn(0x5613c771c060 msgr2=0x5613c7729380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 shutdown_connections 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 1 -- 172.21.15.137:0/2695408970 wait complete. 2024-09-12T12:31:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.655+0000 7f3603f22740 20 client.17643 trim_cache size 0 max 16384 2024-09-12T12:31:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:31:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:31:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:31:44.658 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f35e40a2da0 msgr2=0x7f35e40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f35e40a2da0 0x7f35e40a5260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35f4005fd0 tx=0x7f35f4004a40 comp rx=0 tx=0).stop 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc101370 msgr2=0x7f35fc196630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc101370 0x7f35fc196630 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f35f0035040 tx=0x7f35f0033d90 comp rx=0 tx=0).stop 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 shutdown_connections 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f35e40a2da0 0x7f35e40a5260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35fc1069e0 0x7f35fc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35fc101370 0x7f35fc196630 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 --2- 172.21.15.137:0/3741350233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35fc1009d0 0x7f35fc1960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 >> 172.21.15.137:0/3741350233 conn(0x7f35fc0fc6b0 msgr2=0x7f35fc0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 shutdown_connections 2024-09-12T12:31:44.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:44.656+0000 7f360154b640 1 -- 172.21.15.137:0/3741350233 wait complete. 2024-09-12T12:31:45.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 125422 25.69 MB/sec execute 197 sec latency 245.925 ms 2024-09-12T12:31:45.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:45 smithi137 ceph-mon[30950]: pgmap v589: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 640 op/s 2024-09-12T12:31:45.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:45 smithi189 ceph-mon[35475]: pgmap v589: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 640 op/s 2024-09-12T12:31:45.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:45 smithi171 ceph-mon[35098]: pgmap v589: 97 pgs: 97 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 640 op/s 2024-09-12T12:31:46.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 126072 25.70 MB/sec execute 198 sec latency 86.036 ms 2024-09-12T12:31:47.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 126767 25.76 MB/sec execute 199 sec latency 68.767 ms 2024-09-12T12:31:47.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:47 smithi137 ceph-mon[30950]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 917 op/s 2024-09-12T12:31:47.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:47 smithi189 ceph-mon[35475]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 917 op/s 2024-09-12T12:31:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:47 smithi171 ceph-mon[35098]: pgmap v590: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 917 op/s 2024-09-12T12:31:48.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 127538 25.88 MB/sec execute 200 sec latency 70.140 ms 2024-09-12T12:31:49.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 128083 25.85 MB/sec execute 201 sec latency 132.816 ms 2024-09-12T12:31:49.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:49 smithi137 ceph-mon[30950]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 590 op/s 2024-09-12T12:31:49.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:49 smithi189 ceph-mon[35475]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 590 op/s 2024-09-12T12:31:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:49 smithi171 ceph-mon[35098]: pgmap v591: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 590 op/s 2024-09-12T12:31:50.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 128668 25.93 MB/sec execute 202 sec latency 284.616 ms 2024-09-12T12:31:51.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 129181 25.92 MB/sec execute 203 sec latency 113.439 ms 2024-09-12T12:31:51.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:51 smithi137 ceph-mon[30950]: pgmap v592: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 647 op/s 2024-09-12T12:31:51.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:51 smithi189 ceph-mon[35475]: pgmap v592: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 647 op/s 2024-09-12T12:31:51.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:51 smithi171 ceph-mon[35098]: pgmap v592: 97 pgs: 97 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 647 op/s 2024-09-12T12:31:52.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 129754 25.90 MB/sec execute 204 sec latency 78.339 ms 2024-09-12T12:31:52.666 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T12:31:52.733 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-09-12T12:31:52.734 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:31:53.021 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:31:53.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 130531 26.01 MB/sec execute 205 sec latency 131.658 ms 2024-09-12T12:31:53.461 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:53 smithi137 ceph-mon[30950]: pgmap v593: 97 pgs: 97 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 -- 172.21.15.137:0/4060225787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054101de0 msgr2=0x7fe0541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/4060225787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054101de0 0x7fe0541021c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe048009900 tx=0x7fe04802a410 comp rx=0 tx=0).stop 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 -- 172.21.15.137:0/4060225787 shutdown_connections 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/4060225787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054106f10 0x7fe05410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/4060225787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe054102790 0x7fe0541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/4060225787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054101de0 0x7fe0541021c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 -- 172.21.15.137:0/4060225787 >> 172.21.15.137:0/4060225787 conn(0x7fe0540fd8e0 msgr2=0x7fe0540ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:53.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 -- 172.21.15.137:0/4060225787 shutdown_connections 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.562+0000 7fe05b1d2640 1 -- 172.21.15.137:0/4060225787 wait complete. 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 Processor -- start 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 -- start start 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054101de0 0x7fe0541984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe054102790 0x7fe054198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054106f10 0x7fe05419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe054111f80 con 0x7fe054106f10 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe054111e00 con 0x7fe054102790 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05b1d2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe054112100 con 0x7fe054101de0 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe058f6e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054101de0 0x7fe0541984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe053fff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe054102790 0x7fe054198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:53.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe058f6e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054101de0 0x7fe0541984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:40582/0 (socket says 172.21.15.137:40582) 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe058f6e640 1 -- 172.21.15.137:0/2854090588 learned_addr learned my addr 172.21.15.137:0/2854090588 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.563+0000 7fe05976f640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054106f10 0x7fe05419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe053fff640 1 -- 172.21.15.137:0/2854090588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054101de0 msgr2=0x7fe0541984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe053fff640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054101de0 0x7fe0541984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe053fff640 1 -- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054106f10 msgr2=0x7fe05419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe053fff640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054106f10 0x7fe05419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe053fff640 1 -- 172.21.15.137:0/2854090588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05419d470 con 0x7fe054102790 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe053fff640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe054102790 0x7fe054198a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe04000e9f0 tx=0x7fe04000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe051ffb640 1 -- 172.21.15.137:0/2854090588 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04000ccc0 con 0x7fe054102790 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05419d760 con 0x7fe054102790 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0541a5040 con 0x7fe054102790 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe051ffb640 1 -- 172.21.15.137:0/2854090588 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe040010910 con 0x7fe054102790 2024-09-12T12:31:53.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.564+0000 7fe051ffb640 1 -- 172.21.15.137:0/2854090588 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040018880 con 0x7fe054102790 2024-09-12T12:31:53.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.565+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe038005290 con 0x7fe054102790 2024-09-12T12:31:53.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.567+0000 7fe051ffb640 1 -- 172.21.15.137:0/2854090588 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe040010430 con 0x7fe054102790 2024-09-12T12:31:53.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.568+0000 7fe051ffb640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe0440a2d70 0x7fe0440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:53.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.568+0000 7fe051ffb640 1 -- 172.21.15.137:0/2854090588 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7fe0400cf5a0 con 0x7fe054102790 2024-09-12T12:31:53.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.568+0000 7fe058f6e640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe0440a2d70 0x7fe0440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:53.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.568+0000 7fe058f6e640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe0440a2d70 0x7fe0440a5230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0480096c0 tx=0x7fe0480046c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:53.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.571+0000 7fe051ffb640 1 -- 172.21.15.137:0/2854090588 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe04009bb60 con 0x7fe054102790 2024-09-12T12:31:53.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:53 smithi189 ceph-mon[35475]: pgmap v593: 97 pgs: 97 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-09-12T12:31:53.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:53 smithi171 ceph-mon[35098]: pgmap v593: 97 pgs: 97 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-09-12T12:31:53.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.826+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe038005580 con 0x7fe054102790 2024-09-12T12:31:53.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.827+0000 7fe051ffb640 1 -- 172.21.15.137:0/2854090588 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7fe04009fe60 con 0x7fe054102790 2024-09-12T12:31:53.829 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:31:53.829 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":54,"btime":"2024-09-12T12:31:32:726301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:31:32.726248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26861":{"gid":26861,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/4049352466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4049352466},{"type":"v1","addr":"172.21.15.189:6835","nonce":4049352466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:31:53.829 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 54 2024-09-12T12:31:53.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.830+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe0440a2d70 msgr2=0x7fe0440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:53.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.830+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe0440a2d70 0x7fe0440a5230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0480096c0 tx=0x7fe0480046c0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.830+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe054102790 msgr2=0x7fe054198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.830+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe054102790 0x7fe054198a00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe04000e9f0 tx=0x7fe04000eeb0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 shutdown_connections 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe0440a2d70 0x7fe0440a5230 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe054106f10 0x7fe05419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe054102790 0x7fe054198a00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 --2- 172.21.15.137:0/2854090588 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe054101de0 0x7fe0541984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 >> 172.21.15.137:0/2854090588 conn(0x7fe0540fd8e0 msgr2=0x7fe0540fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 shutdown_connections 2024-09-12T12:31:53.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:53.831+0000 7fe05b1d2640 1 -- 172.21.15.137:0/2854090588 wait complete. 2024-09-12T12:31:54.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 131330 26.08 MB/sec execute 206 sec latency 82.921 ms 2024-09-12T12:31:54.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:54 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2854090588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:31:54.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:54 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2854090588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:31:54.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:54 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2854090588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:31:54.886 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-09-12T12:31:54.886 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-09-12T12:31:55.188 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:31:55.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 131836 26.12 MB/sec execute 207 sec latency 261.916 ms 2024-09-12T12:31:55.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:55 smithi137 ceph-mon[30950]: pgmap v594: 97 pgs: 97 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 608 op/s 2024-09-12T12:31:55.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:55 smithi189 ceph-mon[35475]: pgmap v594: 97 pgs: 97 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 608 op/s 2024-09-12T12:31:55.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:55 smithi171 ceph-mon[35098]: pgmap v594: 97 pgs: 97 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 608 op/s 2024-09-12T12:31:55.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 -- 172.21.15.137:0/2954140406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c100e20 msgr2=0x7f045c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:55.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 --2- 172.21.15.137:0/2954140406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c100e20 0x7f045c101200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0450009930 tx=0x7f045002a4d0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 -- 172.21.15.137:0/2954140406 shutdown_connections 2024-09-12T12:31:55.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 --2- 172.21.15.137:0/2954140406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f045c102ef0 0x7f045c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 --2- 172.21.15.137:0/2954140406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045c102570 0x7f045c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 --2- 172.21.15.137:0/2954140406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c100e20 0x7f045c101200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 -- 172.21.15.137:0/2954140406 >> 172.21.15.137:0/2954140406 conn(0x7f045c0fc7d0 msgr2=0x7f045c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 -- 172.21.15.137:0/2954140406 shutdown_connections 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.759+0000 7f0462ce3640 1 -- 172.21.15.137:0/2954140406 wait complete. 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.760+0000 7f0462ce3640 1 Processor -- start 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.760+0000 7f0462ce3640 1 -- start start 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.760+0000 7f0462ce3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045c100e20 0x7f045c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.760+0000 7f0462ce3640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c102570 0x7f045c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.760+0000 7f0462ce3640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f045c102ef0 0x7f045c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.760+0000 7f0462ce3640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f045c110f50 con 0x7f045c102ef0 2024-09-12T12:31:55.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f0462ce3640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f045c110dd0 con 0x7f045c102570 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f0462ce3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1110d0 con 0x7f045c100e20 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c102570 0x7f045c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f0461280640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f045c102ef0 0x7f045c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c102570 0x7f045c197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55330/0 (socket says 172.21.15.137:55330) 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 -- 172.21.15.137:0/3227329439 learned_addr learned my addr 172.21.15.137:0/3227329439 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 -- 172.21.15.137:0/3227329439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045c100e20 msgr2=0x7f045c197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045c100e20 0x7f045c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 -- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f045c102ef0 msgr2=0x7f045c19bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f045c102ef0 0x7f045c19bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 -- 172.21.15.137:0/3227329439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c19c3e0 con 0x7f045c102570 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f0461280640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f045c102ef0 0x7f045c19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f045bfff640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c102570 0x7f045c197970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f044800e9f0 tx=0x7f044800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:55.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f0459ffb640 1 -- 172.21.15.137:0/3227329439 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044800cd80 con 0x7f045c102570 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.761+0000 7f0459ffb640 1 -- 172.21.15.137:0/3227329439 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0448010910 con 0x7f045c102570 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.762+0000 7f0459ffb640 1 -- 172.21.15.137:0/3227329439 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448018990 con 0x7f045c102570 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.762+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c19c6d0 con 0x7f045c102570 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.762+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1a3fb0 con 0x7f045c102570 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 Processor -- start 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 -- start start 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f0459ffb640 1 -- 172.21.15.137:0/3227329439 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0448010ac0 con 0x7f045c102570 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794da5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da6160 0x559794d95890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f0460a7f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da6160 0x559794d95890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794dad740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f0460a7f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da6160 0x559794d95890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55340/0 (socket says 172.21.15.137:55340) 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f0461280640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794da5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f0460a7f640 1 -- 172.21.15.137:0/4101800547 learned_addr learned my addr 172.21.15.137:0/4101800547 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559794cfd2a0 con 0x559794d95f00 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559794ca4b10 con 0x559794da6160 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.765+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559794cf9260 con 0x559794da57b0 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f045bfff640 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794dad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f0460a7f640 1 -- 172.21.15.137:0/4101800547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 msgr2=0x559794da5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f0460a7f640 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794da5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f0460a7f640 1 -- 172.21.15.137:0/4101800547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 msgr2=0x559794dad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f0460a7f640 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794dad740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f0460a7f640 1 -- 172.21.15.137:0/4101800547 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559794a6eb90 con 0x559794da6160 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f0460a7f640 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da6160 0x559794d95890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0450031490 tx=0x7f04500314c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b2a83d91bc4d414 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.766+0000 7f0446ffd640 1 -- 172.21.15.137:0/4101800547 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045003e070 con 0x559794da6160 2024-09-12T12:31:55.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.767+0000 7f0446ffd640 1 -- 172.21.15.137:0/4101800547 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04500317b0 con 0x559794da6160 2024-09-12T12:31:55.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.767+0000 7f0446ffd640 1 -- 172.21.15.137:0/4101800547 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450035040 con 0x559794da6160 2024-09-12T12:31:55.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.767+0000 7f0459ffb640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f044c0a2cf0 0x7f044c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.767+0000 7f0459ffb640 1 -- 172.21.15.137:0/3227329439 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f04480cf7c0 con 0x7f045c102570 2024-09-12T12:31:55.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.767+0000 7f0461280640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f044c0a2cf0 0x7f044c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da6160 msgr2=0x559794d95890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da6160 0x559794d95890 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0450031490 tx=0x7f04500314c0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 shutdown_connections 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794dad740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da6160 0x559794d95890 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 --2- 172.21.15.137:0/4101800547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794da5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 >> 172.21.15.137:0/4101800547 conn(0x559794d937e0 msgr2=0x559794d93bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f0461280640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f044c0a2cf0 0x7f044c0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x559794d92580 tx=0x7f0454009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 shutdown_connections 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.768+0000 7f04650bc740 1 -- 172.21.15.137:0/4101800547 wait complete. 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 -- start start 2024-09-12T12:31:55.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 Processor -- start 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794d95f00 0x559794da0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da57b0 0x559794da0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da1140 0x559794dae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x559794d9cd80 con 0x559794da57b0 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559794c856e0 con 0x559794d95f00 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559794d9cf00 con 0x559794da1140 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f04650bc740 10 client.0 _ll_register_callbacks cb 0x559794ca9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f0460a7f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794d95f00 0x559794da0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.769+0000 7f045bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da57b0 0x559794da0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f0460a7f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794d95f00 0x559794da0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:55348/0 (socket says 172.21.15.137:55348) 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f0460a7f640 1 -- 172.21.15.137:0/825681457 learned_addr learned my addr 172.21.15.137:0/825681457 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f0461280640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da1140 0x559794dae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f045bfff640 1 -- 172.21.15.137:0/825681457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da1140 msgr2=0x559794dae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f045bfff640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da1140 0x559794dae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f045bfff640 1 -- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794d95f00 msgr2=0x559794da0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f045bfff640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794d95f00 0x559794da0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f045bfff640 1 -- 172.21.15.137:0/825681457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559794daee20 con 0x559794da57b0 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f0461280640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da1140 0x559794dae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f0460a7f640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794d95f00 0x559794da0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f045bfff640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da57b0 0x559794da0bb0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f04480ccf60 tx=0x7f044809bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9421668dea10f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f043bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f043bfff640 1 -- 172.21.15.137:0/825681457 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480e5070 con 0x559794da57b0 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04500317b0 con 0x559794da57b0 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f043bfff640 1 -- 172.21.15.137:0/825681457 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f0448018d30 con 0x559794da57b0 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f043bfff640 1 -- 172.21.15.137:0/825681457 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f044809bd30 con 0x559794da57b0 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f04650bc740 10 client.17661 fetch_fsmap learned FSMap version 54 2024-09-12T12:31:55.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.770+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559794db17d0 con 0x559794da57b0 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f043bfff640 1 -- 172.21.15.137:0/825681457 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04500317b0 con 0x559794da57b0 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f043bfff640 1 -- 172.21.15.137:0/825681457 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f0448097a80 con 0x559794da57b0 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f04650bc740 10 client.17661 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f04650bc740 10 client.17661 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f04650bc740 20 client.17661 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f04650bc740 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794db2ce0 0x559794db5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f04650bc740 4 client.17661 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559794d0a9d0 con 0x559794db2ce0 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f0460a7f640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794db2ce0 0x559794db5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.771+0000 7f043bfff640 10 client.17661 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.772+0000 7f0460a7f640 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794db2ce0 0x559794db5130 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed8d10bbda5a0b4 server_cookie=631842e80f41bcf4 in_seq=0 out_seq=0 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.773+0000 7f043bfff640 1 -- 172.21.15.137:0/825681457 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559794d0a9d0 con 0x559794db2ce0 2024-09-12T12:31:55.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.773+0000 7f043bfff640 10 client.17661 handle_command_reply: tid=0 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.773+0000 7f04650bc740 1 client.17661 shutdown 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da57b0 msgr2=0x559794da0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da57b0 0x559794da0bb0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f04480ccf60 tx=0x7f044809bc00 comp rx=0 tx=0).stop 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 shutdown_connections 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da1140 0x559794dae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794db2ce0 0x559794db5130 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da57b0 0x559794da0bb0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 --2- 172.21.15.137:0/825681457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794d95f00 0x559794da0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 >> 172.21.15.137:0/825681457 conn(0x559794d937e0 msgr2=0x559794d92650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f043bfff640 0 client.17661 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 shutdown_connections 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 1 -- 172.21.15.137:0/825681457 wait complete. 2024-09-12T12:31:55.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.774+0000 7f04650bc740 20 client.17661 trim_cache size 0 max 16384 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.792+0000 7f04650bc740 1 Processor -- start 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.792+0000 7f04650bc740 1 -- start start 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f04650bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794d92d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f04650bc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da1140 0x559794d95030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f04650bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794d95570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f04650bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1addc0 con 0x559794d95f00 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f04650bc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1ab690 con 0x559794da1140 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f04650bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1a9500 con 0x559794da57b0 2024-09-12T12:31:55.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794d92d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794d92d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37314/0 (socket says 172.21.15.137:37314) 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f045bfff640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da1140 0x559794d95030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 -- 172.21.15.137:0/1356654270 learned_addr learned my addr 172.21.15.137:0/1356654270 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0461280640 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794d95570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 -- 172.21.15.137:0/1356654270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 msgr2=0x559794d95570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794d95570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 -- 172.21.15.137:0/1356654270 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da1140 msgr2=0x559794d95030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da1140 0x559794d95030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 -- 172.21.15.137:0/1356654270 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559794d94e40 con 0x559794d95f00 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.793+0000 7f0460a7f640 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794d92d60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0450031490 tx=0x7f0450005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34efc2a37de8a75c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04467fc640 1 -- 172.21.15.137:0/1356654270 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450035040 con 0x559794d95f00 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04467fc640 1 -- 172.21.15.137:0/1356654270 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0450005850 con 0x559794d95f00 2024-09-12T12:31:55.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04467fc640 1 -- 172.21.15.137:0/1356654270 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045003e040 con 0x559794d95f00 2024-09-12T12:31:55.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 -- 172.21.15.137:0/1356654270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 msgr2=0x559794d92d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:55.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794d92d60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0450031490 tx=0x7f0450005fd0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 -- 172.21.15.137:0/1356654270 shutdown_connections 2024-09-12T12:31:55.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794da57b0 0x559794d95570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da1140 0x559794d95030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 --2- 172.21.15.137:0/1356654270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794d95f00 0x559794d92d60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 -- 172.21.15.137:0/1356654270 >> 172.21.15.137:0/1356654270 conn(0x559794d937e0 msgr2=0x559794d92650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.794+0000 7f04650bc740 1 -- 172.21.15.137:0/1356654270 shutdown_connections 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 -- 172.21.15.137:0/1356654270 wait complete. 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 -- start start 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 Processor -- start 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794d95f00 0x559794dabe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da1140 0x559794d99470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da57b0 0x559794d99a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1a9b50 con 0x559794da1140 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1aafc0 con 0x559794da57b0 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1aa220 con 0x559794d95f00 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f04650bc740 10 client.0 _ll_register_callbacks cb 0x7f04480980a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:55.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f045bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da1140 0x559794d99470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f045bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da1140 0x559794d99470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37318/0 (socket says 172.21.15.137:37318) 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f045bfff640 1 -- 172.21.15.137:0/980407401 learned_addr learned my addr 172.21.15.137:0/980407401 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f0461280640 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da57b0 0x559794d99a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.795+0000 7f0460a7f640 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794d95f00 0x559794dabe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f045bfff640 1 -- 172.21.15.137:0/980407401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794d95f00 msgr2=0x559794dabe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f045bfff640 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794d95f00 0x559794dabe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f045bfff640 1 -- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da57b0 msgr2=0x559794d99a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f045bfff640 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da57b0 0x559794d99a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f045bfff640 1 -- 172.21.15.137:0/980407401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559794d9af10 con 0x559794da1140 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f045bfff640 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da1140 0x559794d99470 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f044809bc70 tx=0x7f04480d3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bf0c1256695b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f043bfff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f043bfff640 1 -- 172.21.15.137:0/980407401 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480e5030 con 0x559794da1140 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f043bfff640 1 -- 172.21.15.137:0/980407401 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f044809b380 con 0x559794da1140 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0450005850 con 0x559794da1140 2024-09-12T12:31:55.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f043bfff640 1 -- 172.21.15.137:0/980407401 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f04480a9050 con 0x559794da1140 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f04650bc740 10 client.17667 fetch_fsmap learned FSMap version 54 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.796+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559794e1d8d0 con 0x559794da1140 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f043bfff640 1 -- 172.21.15.137:0/980407401 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0450005850 con 0x559794da1140 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f043bfff640 1 -- 172.21.15.137:0/980407401 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f04480a1400 con 0x559794da1140 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f04650bc740 10 client.17667 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f04650bc740 10 client.17667 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f04650bc740 20 client.17667 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f04650bc740 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794e1e0c0 0x559794e204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f04650bc740 4 client.17667 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x559794db54c0 con 0x559794e1e0c0 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f0460a7f640 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794e1e0c0 0x559794e204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f0460a7f640 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794e1e0c0 0x559794e204e0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60826dcd23e54f7f server_cookie=dd082b70129a933c in_seq=0 out_seq=0 2024-09-12T12:31:55.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:55.797+0000 7f043bfff640 10 client.17667 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:56.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132130 26.09 MB/sec execute 208 sec latency 427.835 ms 2024-09-12T12:31:56.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f043bfff640 1 -- 172.21.15.137:0/980407401 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559794db54c0 con 0x559794e1e0c0 2024-09-12T12:31:56.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f043bfff640 10 client.17667 handle_command_reply: tid=0 2024-09-12T12:31:56.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 client.17667 shutdown 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da1140 msgr2=0x559794d99470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da1140 0x559794d99470 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f044809bc70 tx=0x7f04480d3000 comp rx=0 tx=0).stop 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 shutdown_connections 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559794da57b0 0x559794d99a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x559794e1e0c0 0x559794e204e0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559794da1140 0x559794d99470 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 --2- 172.21.15.137:0/980407401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559794d95f00 0x559794dabe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f043bfff640 0 client.17667 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 >> 172.21.15.137:0/980407401 conn(0x559794d937e0 msgr2=0x559794d95030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 shutdown_connections 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 1 -- 172.21.15.137:0/980407401 wait complete. 2024-09-12T12:31:56.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.311+0000 7f04650bc740 20 client.17667 trim_cache size 0 max 16384 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 54, 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 66, 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "c318f5ad": { 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 66, 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:31:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 141.8, 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:31:56.315 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:31:56.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f044c0a2cf0 msgr2=0x7f044c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:56.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f044c0a2cf0 0x7f044c0a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x559794d92580 tx=0x7f0454009500 comp rx=0 tx=0).stop 2024-09-12T12:31:56.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c102570 msgr2=0x7f045c197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:56.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c102570 0x7f045c197970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f044800e9f0 tx=0x7f044800eeb0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 shutdown_connections 2024-09-12T12:31:56.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f044c0a2cf0 0x7f044c0a51b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f045c102ef0 0x7f045c19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f045c102570 0x7f045c197970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 --2- 172.21.15.137:0/3227329439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f045c100e20 0x7f045c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:56.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 >> 172.21.15.137:0/3227329439 conn(0x7f045c0fc7d0 msgr2=0x7f045c104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:56.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 shutdown_connections 2024-09-12T12:31:56.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:31:56.312+0000 7f0462ce3640 1 -- 172.21.15.137:0/3227329439 wait complete. 2024-09-12T12:31:57.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.96 MB/sec execute 209 sec latency 1427.894 ms 2024-09-12T12:31:57.256 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c318f5ad', quiesce took 4.5 seconds. Will release after: 17.3 2024-09-12T12:31:57.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:57 smithi137 ceph-mon[30950]: pgmap v595: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 872 op/s 2024-09-12T12:31:57.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:57 smithi189 ceph-mon[35475]: pgmap v595: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 872 op/s 2024-09-12T12:31:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:57 smithi171 ceph-mon[35098]: pgmap v595: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 16 MiB/s wr, 872 op/s 2024-09-12T12:31:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 -- 172.21.15.137:0/1586560467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae8102cc0 msgr2=0x7f9ae81030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/1586560467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae8102cc0 0x7f9ae81030a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009ca0 tx=0x7f9ad802a950 comp rx=0 tx=0).stop 2024-09-12T12:31:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 -- 172.21.15.137:0/1586560467 shutdown_connections 2024-09-12T12:31:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/1586560467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ae810c110 0x7f9ae810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/1586560467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae81035e0 0x7f9ae810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/1586560467 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae8102cc0 0x7f9ae81030a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 -- 172.21.15.137:0/1586560467 >> 172.21.15.137:0/1586560467 conn(0x7f9ae80fc6b0 msgr2=0x7f9ae80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 -- 172.21.15.137:0/1586560467 shutdown_connections 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.755+0000 7f9aef2d9640 1 -- 172.21.15.137:0/1586560467 wait complete. 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 Processor -- start 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 -- start start 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae8102cc0 0x7f9ae81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae81035e0 0x7f9ae8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ae810c110 0x7f9ae819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8111ef0 con 0x7f9ae81035e0 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8111d70 con 0x7f9ae810c110 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aef2d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8112070 con 0x7f9ae8102cc0 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae81035e0 0x7f9ae8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae81035e0 0x7f9ae8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37336/0 (socket says 172.21.15.137:37336) 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 -- 172.21.15.137:0/702473876 learned_addr learned my addr 172.21.15.137:0/702473876 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aed075640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae8102cc0 0x7f9ae81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aed876640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ae810c110 0x7f9ae819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 -- 172.21.15.137:0/702473876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae8102cc0 msgr2=0x7f9ae81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae8102cc0 0x7f9ae81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 -- 172.21.15.137:0/702473876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ae810c110 msgr2=0x7f9ae819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ae810c110 0x7f9ae819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.756+0000 7f9aec874640 1 -- 172.21.15.137:0/702473876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae819d3d0 con 0x7f9ae81035e0 2024-09-12T12:31:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.757+0000 7f9aec874640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae81035e0 0x7f9ae8198920 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00e9f0 tx=0x7f9adc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.757+0000 7f9ae67fc640 1 -- 172.21.15.137:0/702473876 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc00ccc0 con 0x7f9ae81035e0 2024-09-12T12:31:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.757+0000 7f9ae67fc640 1 -- 172.21.15.137:0/702473876 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc00ce60 con 0x7f9ae81035e0 2024-09-12T12:31:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.757+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae819d6c0 con 0x7f9ae81035e0 2024-09-12T12:31:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.757+0000 7f9ae67fc640 1 -- 172.21.15.137:0/702473876 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc010690 con 0x7f9ae81035e0 2024-09-12T12:31:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.757+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae81a4f50 con 0x7f9ae81035e0 2024-09-12T12:31:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.759+0000 7f9ae67fc640 1 -- 172.21.15.137:0/702473876 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9adc0046c0 con 0x7f9ae81035e0 2024-09-12T12:31:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 Processor -- start 2024-09-12T12:31:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 -- start start 2024-09-12T12:31:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff4950ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950e410 0x55ff49518770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff4951b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ff494722f0 con 0x55ff4950daf0 2024-09-12T12:31:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ff4941a270 con 0x55ff49518cb0 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9af206e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ff4946e2b0 con 0x55ff4950e410 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9ae67fc640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ad00a2da0 0x7f9ad00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9aed876640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff4950ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9aed876640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff4950ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37350/0 (socket says 172.21.15.137:37350) 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9aed876640 1 -- 172.21.15.137:0/4263449688 learned_addr learned my addr 172.21.15.137:0/4263449688 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9aed876640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ad00a2da0 0x7f9ad00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9ae67fc640 1 -- 172.21.15.137:0/702473876 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f9adc0cf7f0 con 0x7f9ae81035e0 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9aec874640 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff4951b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.760+0000 7f9aed075640 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950e410 0x55ff49518770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9aed876640 1 -- 172.21.15.137:0/4263449688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950e410 msgr2=0x55ff49518770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9aed876640 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950e410 0x55ff49518770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9aed876640 1 -- 172.21.15.137:0/4263449688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 msgr2=0x55ff4951b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.771 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9aed876640 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff4951b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9aed876640 1 -- 172.21.15.137:0/4263449688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff49419340 con 0x55ff4950daf0 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9aed876640 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff4950ded0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0014be0 tx=0x7f9ae00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a586a39dc1408e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9aed876640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ad00a2da0 0x7f9ad00a5260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0011cb0 tx=0x7f9ae0016000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9acf7fe640 1 -- 172.21.15.137:0/4263449688 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0022070 con 0x55ff4950daf0 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9acf7fe640 1 -- 172.21.15.137:0/4263449688 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae000c960 con 0x55ff4950daf0 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.761+0000 7f9acf7fe640 1 -- 172.21.15.137:0/4263449688 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00049a0 con 0x55ff4950daf0 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 -- 172.21.15.137:0/4263449688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 msgr2=0x55ff4950ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff4950ded0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0014be0 tx=0x7f9ae00117b0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 -- 172.21.15.137:0/4263449688 shutdown_connections 2024-09-12T12:31:57.772 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff4951b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950e410 0x55ff49518770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 --2- 172.21.15.137:0/4263449688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff4950ded0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 -- 172.21.15.137:0/4263449688 >> 172.21.15.137:0/4263449688 conn(0x55ff4950a060 msgr2=0x55ff4950a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 -- 172.21.15.137:0/4263449688 shutdown_connections 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 -- 172.21.15.137:0/4263449688 wait complete. 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 -- start start 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.762+0000 7f9af206e740 1 Processor -- start 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9af206e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49515be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.773 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9af206e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff49516120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9af206e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49516660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9af206e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ff4951b880 con 0x55ff4950daf0 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9af206e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ff493fb290 con 0x55ff4950e410 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9af206e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ff4951ba00 con 0x55ff49518cb0 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9af206e740 10 client.0 _ll_register_callbacks cb 0x55ff4942c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49515be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49515be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37354/0 (socket says 172.21.15.137:37354) 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 -- 172.21.15.137:0/206781248 learned_addr learned my addr 172.21.15.137:0/206781248 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aec874640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff49516120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed876640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49516660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 -- 172.21.15.137:0/206781248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 msgr2=0x55ff49516660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49516660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 -- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 msgr2=0x55ff49516120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff49516120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 -- 172.21.15.137:0/206781248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff49185210 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aec874640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff49516120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9aed075640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49515be0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8035000 tx=0x7f9ad802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3186a8edb95f1340 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9accff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.763+0000 7f9accff9640 1 -- 172.21.15.137:0/206781248 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8049060 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9accff9640 1 -- 172.21.15.137:0/206781248 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f9ad8003b70 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff4947fa20 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9accff9640 1 -- 172.21.15.137:0/206781248 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9ad8034040 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 10 client.17679 fetch_fsmap learned FSMap version 54 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ff49251620 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9accff9640 1 -- 172.21.15.137:0/206781248 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ff4947fa20 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9accff9640 1 -- 172.21.15.137:0/206781248 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f9ad804e4f0 con 0x55ff4950daf0 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 10 client.17679 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:31:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 10 client.17679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 20 client.17679 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49528c40 0x55ff4952b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 4 client.17679 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff49479d70 con 0x55ff49528c40 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.764+0000 7f9aec874640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49528c40 0x55ff4952b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.765+0000 7f9accff9640 10 client.17679 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.765+0000 7f9aec874640 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49528c40 0x55ff4952b030 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d517838ca125b9d server_cookie=9276c1409b3a3fec in_seq=0 out_seq=0 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.765+0000 7f9accff9640 1 -- 172.21.15.137:0/206781248 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff49479d70 con 0x55ff49528c40 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.765+0000 7f9accff9640 10 client.17679 handle_command_reply: tid=0 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.765+0000 7f9af206e740 1 client.17679 shutdown 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 msgr2=0x55ff49515be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49515be0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8035000 tx=0x7f9ad802f3a0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 shutdown_connections 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49516660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff49516120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49528c40 0x55ff4952b030 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 --2- 172.21.15.137:0/206781248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49515be0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 >> 172.21.15.137:0/206781248 conn(0x55ff4950a060 msgr2=0x55ff4950fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9accff9640 0 client.17679 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 shutdown_connections 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 1 -- 172.21.15.137:0/206781248 wait complete. 2024-09-12T12:31:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.766+0000 7f9af206e740 20 client.17679 trim_cache size 0 max 16384 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.780+0000 7f9af206e740 1 Processor -- start 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.780+0000 7f9af206e740 1 -- start start 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.780+0000 7f9af206e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950daf0 0x55ff49510560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9af206e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950e410 0x55ff49516490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9af206e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff49530920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9af206e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81afff0 con 0x55ff4950e410 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9af206e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81ad8c0 con 0x55ff49518cb0 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9af206e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81ab710 con 0x55ff4950daf0 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950e410 0x55ff49516490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950e410 0x55ff49516490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37360/0 (socket says 172.21.15.137:37360) 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 -- 172.21.15.137:0/2164063138 learned_addr learned my addr 172.21.15.137:0/2164063138 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aed876640 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff49530920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aed075640 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950daf0 0x55ff49510560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 -- 172.21.15.137:0/2164063138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950daf0 msgr2=0x55ff49510560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950daf0 0x55ff49510560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 -- 172.21.15.137:0/2164063138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 msgr2=0x55ff49530920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff49530920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 -- 172.21.15.137:0/2164063138 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff49509e90 con 0x55ff4950e410 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aed876640 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff49530920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aed075640 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950daf0 0x55ff49510560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aec874640 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950e410 0x55ff49516490 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c6a0 tx=0x7f9adc004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f42a29a027a2d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aceffd640 1 -- 172.21.15.137:0/2164063138 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0a0110 con 0x55ff4950e410 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aceffd640 1 -- 172.21.15.137:0/2164063138 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc0a49d0 con 0x55ff4950e410 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.781+0000 7f9aceffd640 1 -- 172.21.15.137:0/2164063138 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0a4540 con 0x55ff4950e410 2024-09-12T12:31:57.787 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 -- 172.21.15.137:0/2164063138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950e410 msgr2=0x55ff49516490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950e410 0x55ff49516490 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c6a0 tx=0x7f9adc004a00 comp rx=0 tx=0).stop 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 -- 172.21.15.137:0/2164063138 shutdown_connections 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff49518cb0 0x55ff49530920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950e410 0x55ff49516490 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 --2- 172.21.15.137:0/2164063138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff4950daf0 0x55ff49510560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 -- 172.21.15.137:0/2164063138 >> 172.21.15.137:0/2164063138 conn(0x55ff4950a060 msgr2=0x55ff49517c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 -- 172.21.15.137:0/2164063138 shutdown_connections 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 -- 172.21.15.137:0/2164063138 wait complete. 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 -- start start 2024-09-12T12:31:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.782+0000 7f9af206e740 1 Processor -- start 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9af206e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49521350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9af206e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff495128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9af206e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49512e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9af206e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81abd80 con 0x55ff4950daf0 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9af206e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81ad1f0 con 0x55ff4950e410 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9af206e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81ac450 con 0x55ff49518cb0 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9af206e740 10 client.0 _ll_register_callbacks cb 0x7f9ad802fe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49521350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49521350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37372/0 (socket says 172.21.15.137:37372) 2024-09-12T12:31:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 -- 172.21.15.137:0/799011428 learned_addr learned my addr 172.21.15.137:0/799011428 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aec874640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff495128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed876640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49512e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 -- 172.21.15.137:0/799011428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 msgr2=0x55ff49512e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49512e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 -- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 msgr2=0x55ff495128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff495128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 -- 172.21.15.137:0/799011428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4950b110 con 0x55ff4950daf0 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed876640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49512e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9aed075640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49521350 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802f490 tx=0x7f9ad804e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37e2f3774a6d871 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9accff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.783+0000 7f9accff9640 1 -- 172.21.15.137:0/799011428 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8049070 con 0x55ff4950daf0 2024-09-12T12:31:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9accff9640 1 -- 172.21.15.137:0/799011428 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8440+0+0 (secure 0 0 0) 0x7f9ad8003d80 con 0x55ff4950daf0 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9adc0a49d0 con 0x55ff4950daf0 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9accff9640 1 -- 172.21.15.137:0/799011428 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9ad802fec0 con 0x55ff4950daf0 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 10 client.17685 fetch_fsmap learned FSMap version 54 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ff49593660 con 0x55ff4950daf0 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9accff9640 1 -- 172.21.15.137:0/799011428 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9adc0a49d0 con 0x55ff4950daf0 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9accff9640 1 -- 172.21.15.137:0/799011428 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f9ad803a430 con 0x55ff4950daf0 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 10 client.17685 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 20 client.17685 populate_metadata read hostname 'smithi137' 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49593e30 0x55ff49596250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 4 client.17685 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff493f6900 con 0x55ff49593e30 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.784+0000 7f9aec874640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49593e30 0x55ff49596250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.785+0000 7f9accff9640 10 client.17685 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:57.785+0000 7f9aec874640 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49593e30 0x55ff49596250 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9398699728192d96 server_cookie=c6cabf21b66cc095 in_seq=0 out_seq=0 2024-09-12T12:31:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9accff9640 1 -- 172.21.15.137:0/799011428 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff493f6900 con 0x55ff49593e30 2024-09-12T12:31:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9accff9640 10 client.17685 handle_command_reply: tid=0 2024-09-12T12:31:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 client.17685 shutdown 2024-09-12T12:31:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 msgr2=0x55ff49521350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49521350 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802f490 tx=0x7f9ad804e690 comp rx=0 tx=0).stop 2024-09-12T12:31:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 shutdown_connections 2024-09-12T12:31:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ff49518cb0 0x55ff49512e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ff4950e410 0x55ff495128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ff49593e30 0x55ff49596250 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 --2- 172.21.15.137:0/799011428 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff4950daf0 0x55ff49521350 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9accff9640 0 client.17685 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:31:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 >> 172.21.15.137:0/799011428 conn(0x55ff4950a060 msgr2=0x55ff495089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 shutdown_connections 2024-09-12T12:31:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 1 -- 172.21.15.137:0/799011428 wait complete. 2024-09-12T12:31:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.036+0000 7f9af206e740 20 client.17685 trim_cache size 0 max 16384 2024-09-12T12:31:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:31:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:31:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:31:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:31:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ad00a2da0 msgr2=0x7f9ad00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ad00a2da0 0x7f9ad00a5260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0011cb0 tx=0x7f9ae0016000 comp rx=0 tx=0).stop 2024-09-12T12:31:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae81035e0 msgr2=0x7f9ae8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:31:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae81035e0 0x7f9ae8198920 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00e9f0 tx=0x7f9adc00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 shutdown_connections 2024-09-12T12:31:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f9ad00a2da0 0x7f9ad00a5260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9ae810c110 0x7f9ae819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ae81035e0 0x7f9ae8198920 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 --2- 172.21.15.137:0/702473876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ae8102cc0 0x7f9ae81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:31:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 >> 172.21.15.137:0/702473876 conn(0x7f9ae80fc6b0 msgr2=0x7f9ae80fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:31:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 shutdown_connections 2024-09-12T12:31:58.045 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:31:58.037+0000 7f9aef2d9640 1 -- 172.21.15.137:0/702473876 wait complete. 2024-09-12T12:31:58.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.84 MB/sec execute 210 sec latency 2427.982 ms 2024-09-12T12:31:59.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.72 MB/sec execute 211 sec latency 3428.074 ms 2024-09-12T12:31:59.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:31:59 smithi137 ceph-mon[30950]: pgmap v596: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-09-12T12:31:59.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:31:59 smithi189 ceph-mon[35475]: pgmap v596: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-09-12T12:31:59.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:31:59 smithi171 ceph-mon[35098]: pgmap v596: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 537 op/s 2024-09-12T12:32:00.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.59 MB/sec execute 212 sec latency 4428.189 ms 2024-09-12T12:32:01.230 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.47 MB/sec execute 213 sec latency 5428.285 ms 2024-09-12T12:32:01.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:01 smithi137 ceph-mon[30950]: pgmap v597: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-09-12T12:32:01.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:01 smithi189 ceph-mon[35475]: pgmap v597: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-09-12T12:32:01.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:01 smithi171 ceph-mon[35098]: pgmap v597: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 607 op/s 2024-09-12T12:32:02.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.36 MB/sec execute 214 sec latency 6428.377 ms 2024-09-12T12:32:03.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.24 MB/sec execute 215 sec latency 7428.505 ms 2024-09-12T12:32:03.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:03 smithi137 ceph-mon[30950]: pgmap v598: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 739 op/s 2024-09-12T12:32:03.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:03 smithi189 ceph-mon[35475]: pgmap v598: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 739 op/s 2024-09-12T12:32:03.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:03 smithi171 ceph-mon[35098]: pgmap v598: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 739 op/s 2024-09-12T12:32:04.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.12 MB/sec execute 216 sec latency 8428.623 ms 2024-09-12T12:32:05.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 25.01 MB/sec execute 217 sec latency 9428.748 ms 2024-09-12T12:32:05.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:05 smithi137 ceph-mon[30950]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 523 op/s 2024-09-12T12:32:05.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:05 smithi189 ceph-mon[35475]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 523 op/s 2024-09-12T12:32:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:05 smithi171 ceph-mon[35098]: pgmap v599: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.3 MiB/s wr, 523 op/s 2024-09-12T12:32:06.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.89 MB/sec execute 218 sec latency 10428.863 ms 2024-09-12T12:32:07.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.78 MB/sec execute 219 sec latency 11428.971 ms 2024-09-12T12:32:07.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:07 smithi189 ceph-mon[35475]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-09-12T12:32:07.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:07 smithi171 ceph-mon[35098]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-09-12T12:32:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:07 smithi137 ceph-mon[30950]: pgmap v600: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 11 MiB/s wr, 621 op/s 2024-09-12T12:32:08.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.66 MB/sec execute 220 sec latency 12429.025 ms 2024-09-12T12:32:08.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:08 smithi189 ceph-mon[35475]: osdmap e117: 12 total, 12 up, 12 in 2024-09-12T12:32:08.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:08 smithi171 ceph-mon[35098]: osdmap e117: 12 total, 12 up, 12 in 2024-09-12T12:32:08.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:08 smithi137 ceph-mon[30950]: osdmap e117: 12 total, 12 up, 12 in 2024-09-12T12:32:09.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.55 MB/sec execute 221 sec latency 13429.070 ms 2024-09-12T12:32:09.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:09 smithi189 ceph-mon[35475]: pgmap v602: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 428 op/s 2024-09-12T12:32:09.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:09 smithi171 ceph-mon[35098]: pgmap v602: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 428 op/s 2024-09-12T12:32:09.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:09 smithi137 ceph-mon[30950]: pgmap v602: 97 pgs: 97 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 428 op/s 2024-09-12T12:32:10.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.44 MB/sec execute 222 sec latency 14429.195 ms 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 -- 172.21.15.137:0/489600158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c101ad0 msgr2=0x7f7e0c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 --2- 172.21.15.137:0/489600158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c101ad0 0x7f7e0c101f30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009930 tx=0x7f7e0002a420 comp rx=0 tx=0).stop 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 -- 172.21.15.137:0/489600158 shutdown_connections 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 --2- 172.21.15.137:0/489600158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e0c102470 0x7f7e0c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 --2- 172.21.15.137:0/489600158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c101ad0 0x7f7e0c101f30 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 --2- 172.21.15.137:0/489600158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e0c107b70 0x7f7e0c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 -- 172.21.15.137:0/489600158 >> 172.21.15.137:0/489600158 conn(0x7f7e0c0fd7d0 msgr2=0x7f7e0c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 -- 172.21.15.137:0/489600158 shutdown_connections 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 -- 172.21.15.137:0/489600158 wait complete. 2024-09-12T12:32:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 Processor -- start 2024-09-12T12:32:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.136+0000 7f7e1361e640 1 -- start start 2024-09-12T12:32:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e1361e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e0c101ad0 0x7f7e0c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.138 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e1361e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c102470 0x7f7e0c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e1361e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e0c107b70 0x7f7e0c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e1361e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c111ee0 con 0x7f7e0c102470 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e1361e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c111d60 con 0x7f7e0c101ad0 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e1361e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c112060 con 0x7f7e0c107b70 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c102470 0x7f7e0c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c102470 0x7f7e0c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53132/0 (socket says 172.21.15.137:53132) 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3336877496 learned_addr learned my addr 172.21.15.137:0/3336877496 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e113ba640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e0c101ad0 0x7f7e0c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e0c107b70 0x7f7e0c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3336877496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e0c107b70 msgr2=0x7f7e0c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e0c107b70 0x7f7e0c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.139 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3336877496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e0c101ad0 msgr2=0x7f7e0c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e0c101ad0 0x7f7e0c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3336877496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c19d360 con 0x7f7e0c102470 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e113ba640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e0c101ad0 0x7f7e0c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c102470 0x7f7e0c1988b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009230 tx=0x7f7e00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e0a7fc640 1 -- 172.21.15.137:0/3336877496 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0003f030 con 0x7f7e0c102470 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e0a7fc640 1 -- 172.21.15.137:0/3336877496 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e00003ad0 con 0x7f7e0c102470 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c19d5f0 con 0x7f7e0c102470 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.137+0000 7f7e0a7fc640 1 -- 172.21.15.137:0/3336877496 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00035070 con 0x7f7e0c102470 2024-09-12T12:32:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.138+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c1a4ee0 con 0x7f7e0c102470 2024-09-12T12:32:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.140+0000 7f7e0a7fc640 1 -- 172.21.15.137:0/3336877496 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7e00042030 con 0x7f7e0c102470 2024-09-12T12:32:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.140+0000 7f7e15ffd740 1 Processor -- start 2024-09-12T12:32:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.140+0000 7f7e15ffd740 1 -- start start 2024-09-12T12:32:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.140+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.147 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4d9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4da24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556ab4cf52f0 con 0x556ab4d909b0 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556ab4c9d270 con 0x556ab4da00c0 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556ab4cf12b0 con 0x556ab4d912d0 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e0a7fc640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7df40a2da0 0x7f7df40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d90d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53146/0 (socket says 172.21.15.137:53146) 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 -- 172.21.15.137:0/1227938841 learned_addr learned my addr 172.21.15.137:0/1227938841 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7df40a2da0 0x7f7df40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e0a7fc640 1 -- 172.21.15.137:0/3336877496 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8528+0+0 (secure 0 0 0) 0x7f7e000efa00 con 0x7f7e0c102470 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4da24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e113ba640 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4d9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 -- 172.21.15.137:0/1227938841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 msgr2=0x556ab4d9fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4d9fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 -- 172.21.15.137:0/1227938841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 msgr2=0x556ab4da24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4da24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 -- 172.21.15.137:0/1227938841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab4c9c340 con 0x556ab4d909b0 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d90d90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7e04013aa0 tx=0x7f7e04013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6368b3275392fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7df40a2da0 0x7f7df40a5260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e04005d00 tx=0x7f7e0401b000 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.141+0000 7f7df37fe640 1 -- 172.21.15.137:0/1227938841 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04023070 con 0x556ab4d909b0 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7df37fe640 1 -- 172.21.15.137:0/1227938841 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0400be50 con 0x556ab4d909b0 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7df37fe640 1 -- 172.21.15.137:0/1227938841 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04004d80 con 0x556ab4d909b0 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 -- 172.21.15.137:0/1227938841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 msgr2=0x556ab4d90d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d90d90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7e04013aa0 tx=0x7f7e04013f60 comp rx=0 tx=0).stop 2024-09-12T12:32:11.149 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 -- 172.21.15.137:0/1227938841 shutdown_connections 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4da24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4d9fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/1227938841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d90d90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 -- 172.21.15.137:0/1227938841 >> 172.21.15.137:0/1227938841 conn(0x556ab4d8d060 msgr2=0x556ab4d8d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 -- 172.21.15.137:0/1227938841 shutdown_connections 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.142+0000 7f7e15ffd740 1 -- 172.21.15.137:0/1227938841 wait complete. 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 -- start start 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 Processor -- start 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d9d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d912d0 0x556ab4d9d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4da00c0 0x556ab4d9dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x556ab4da2c30 con 0x556ab4d909b0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556ab4c7e290 con 0x556ab4d912d0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556ab4da2db0 con 0x556ab4da00c0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e15ffd740 10 client.0 _ll_register_callbacks cb 0x556ab4caf140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e113ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d9d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e113ba640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d9d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53162/0 (socket says 172.21.15.137:53162) 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.143+0000 7f7e113ba640 1 -- 172.21.15.137:0/23893262 learned_addr learned my addr 172.21.15.137:0/23893262 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4da00c0 0x556ab4d9dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d912d0 0x556ab4d9d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e113ba640 1 -- 172.21.15.137:0/23893262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4da00c0 msgr2=0x556ab4d9dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e113ba640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4da00c0 0x556ab4d9dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e113ba640 1 -- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d912d0 msgr2=0x556ab4d9d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e113ba640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d912d0 0x556ab4d9d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e113ba640 1 -- 172.21.15.137:0/23893262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab4a08210 con 0x556ab4d909b0 2024-09-12T12:32:11.151 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d912d0 0x556ab4d9d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e113ba640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d9d040 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0139f0 tx=0x7f7dfc013eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf4e2447a95d5f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7df0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7df0ff9640 1 -- 172.21.15.137:0/23893262 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003970 con 0x556ab4d909b0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7df0ff9640 1 -- 172.21.15.137:0/23893262 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8528+0+0 (secure 0 0 0) 0x7f7dfc003b10 con 0x556ab4d909b0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0400be50 con 0x556ab4d909b0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7df0ff9640 1 -- 172.21.15.137:0/23893262 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7dfc003d10 con 0x556ab4d909b0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e15ffd740 10 client.17697 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556ab4ad4620 con 0x556ab4d909b0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.144+0000 7f7df0ff9640 1 -- 172.21.15.137:0/23893262 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0400be50 con 0x556ab4d909b0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7df0ff9640 1 -- 172.21.15.137:0/23893262 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f7dfc01a030 con 0x556ab4d909b0 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e15ffd740 10 client.17697 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e15ffd740 10 client.17697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e15ffd740 20 client.17697 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:11.152 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4db0080 0x556ab4db24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e15ffd740 4 client.17697 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ab4d02a20 con 0x556ab4db0080 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4db0080 0x556ab4db24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7df0ff9640 10 client.17697 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.145+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4db0080 0x556ab4db24b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9554242d395f06 server_cookie=a504b74a1d2a8fd9 in_seq=0 out_seq=0 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7df0ff9640 1 -- 172.21.15.137:0/23893262 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ab4d02a20 con 0x556ab4db0080 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7df0ff9640 10 client.17697 handle_command_reply: tid=0 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 client.17697 shutdown 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 msgr2=0x556ab4d9d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d9d040 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0139f0 tx=0x7f7dfc013eb0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 shutdown_connections 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4da00c0 0x556ab4d9dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d912d0 0x556ab4d9d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4db0080 0x556ab4db24b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/23893262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d909b0 0x556ab4d9d040 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7df0ff9640 0 client.17697 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.146+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 >> 172.21.15.137:0/23893262 conn(0x556ab4d8d060 msgr2=0x556ab4d8c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.147+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 shutdown_connections 2024-09-12T12:32:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.147+0000 7f7e15ffd740 1 -- 172.21.15.137:0/23893262 wait complete. 2024-09-12T12:32:11.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.147+0000 7f7e15ffd740 20 client.17697 trim_cache size 0 max 16384 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 Processor -- start 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 -- start start 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d909b0 0x556ab4d9efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d912d0 0x556ab4db7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4db8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1ade80 con 0x556ab4d912d0 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1ab750 con 0x556ab4da00c0 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1a95c0 con 0x556ab4d909b0 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e10bb9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d912d0 0x556ab4db7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e10bb9640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d912d0 0x556ab4db7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53164/0 (socket says 172.21.15.137:53164) 2024-09-12T12:32:11.165 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3050693163 learned_addr learned my addr 172.21.15.137:0/3050693163 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.161+0000 7f7e113ba640 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d909b0 0x556ab4d9efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3050693163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d909b0 msgr2=0x556ab4d9efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d909b0 0x556ab4d9efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4db8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3050693163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 msgr2=0x556ab4db8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4db8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e10bb9640 1 -- 172.21.15.137:0/3050693163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab4d91100 con 0x556ab4d912d0 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d912d0 0x556ab4db7e10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7e00005c40 tx=0x7f7e000dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971712ac7b3c11f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7df2ffd640 1 -- 172.21.15.137:0/3050693163 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00049040 con 0x556ab4d912d0 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7df2ffd640 1 -- 172.21.15.137:0/3050693163 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e000bc180 con 0x556ab4d912d0 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7df2ffd640 1 -- 172.21.15.137:0/3050693163 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e000b8590 con 0x556ab4d912d0 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3050693163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d912d0 msgr2=0x556ab4db7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d912d0 0x556ab4db7e10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f7e00005c40 tx=0x7f7e000dc040 comp rx=0 tx=0).stop 2024-09-12T12:32:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3050693163 shutdown_connections 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4da00c0 0x556ab4db8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4d912d0 0x556ab4db7e10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3050693163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d909b0 0x556ab4d9efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.162+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3050693163 >> 172.21.15.137:0/3050693163 conn(0x556ab4d8d060 msgr2=0x556ab4d9e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3050693163 shutdown_connections 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3050693163 wait complete. 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 -- start start 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 Processor -- start 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d909b0 0x556ab4da8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4da3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4da00c0 0x556ab4da3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.167 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1a9c10 con 0x556ab4da00c0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1ab080 con 0x556ab4d909b0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c1aa2e0 con 0x556ab4d912d0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e15ffd740 10 client.0 _ll_register_callbacks cb 0x7f7dfc009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.163+0000 7f7e11bbb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4da00c0 0x556ab4da3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4da00c0 0x556ab4da3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53168/0 (socket says 172.21.15.137:53168) 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e113ba640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d909b0 0x556ab4da8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 -- 172.21.15.137:0/3797450126 learned_addr learned my addr 172.21.15.137:0/3797450126 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e10bb9640 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4da3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 -- 172.21.15.137:0/3797450126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 msgr2=0x556ab4da3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4da3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 -- 172.21.15.137:0/3797450126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d909b0 msgr2=0x556ab4da8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d909b0 0x556ab4da8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 -- 172.21.15.137:0/3797450126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab4da4210 con 0x556ab4da00c0 2024-09-12T12:32:11.168 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e11bbb640 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4da00c0 0x556ab4da3cd0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7e040048b0 tx=0x7f7e04010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7114f7006d337f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7df0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7df0ff9640 1 -- 172.21.15.137:0/3797450126 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04023040 con 0x556ab4da00c0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e000bc180 con 0x556ab4da00c0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7df0ff9640 1 -- 172.21.15.137:0/3797450126 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8528+0+0 (secure 0 0 0) 0x7f7e0401a8e0 con 0x556ab4da00c0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7df0ff9640 1 -- 172.21.15.137:0/3797450126 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7e0400be50 con 0x556ab4da00c0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e15ffd740 10 client.17703 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.164+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556ab4e1aaa0 con 0x556ab4da00c0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7df0ff9640 1 -- 172.21.15.137:0/3797450126 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e000bc180 con 0x556ab4da00c0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7df0ff9640 1 -- 172.21.15.137:0/3797450126 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f7e0401c030 con 0x556ab4da00c0 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e15ffd740 10 client.17703 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e15ffd740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e15ffd740 20 client.17703 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:11.169 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4e1aee0 0x556ab4e1d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e15ffd740 4 client.17703 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:32:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ab4cfcd70 con 0x556ab4e1aee0 2024-09-12T12:32:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e113ba640 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4e1aee0 0x556ab4e1d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7e113ba640 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4e1aee0 0x556ab4e1d2d0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2cd048138bba33 server_cookie=919a1e28f3fe13bf in_seq=0 out_seq=0 2024-09-12T12:32:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.165+0000 7f7df0ff9640 10 client.17703 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:11.231 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.33 MB/sec execute 223 sec latency 15429.243 ms 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.258+0000 7f7df0ff9640 1 -- 172.21.15.137:0/3797450126 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ab4cfcd70 con 0x556ab4e1aee0 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.258+0000 7f7df0ff9640 10 client.17703 handle_command_reply: tid=0 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.258+0000 7f7e15ffd740 1 client.17703 shutdown 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4da00c0 msgr2=0x556ab4da3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4da00c0 0x556ab4da3cd0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f7e040048b0 tx=0x7f7e04010fd0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 shutdown_connections 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x556ab4e1aee0 0x556ab4e1d2d0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556ab4da00c0 0x556ab4da3cd0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7df0ff9640 0 client.17703 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556ab4d912d0 0x556ab4da3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 --2- 172.21.15.137:0/3797450126 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556ab4d909b0 0x556ab4da8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 >> 172.21.15.137:0/3797450126 conn(0x556ab4d8d060 msgr2=0x556ab4d8c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 shutdown_connections 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 1 -- 172.21.15.137:0/3797450126 wait complete. 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e15ffd740 20 client.17703 trim_cache size 0 max 16384 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:32:11.260 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7df40a2da0 msgr2=0x7f7df40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e1361e640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7df40a2da0 0x7f7df40a5260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e04005d00 tx=0x7f7e0401b000 comp rx=0 tx=0).stop 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c102470 msgr2=0x7f7e0c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e1361e640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c102470 0x7f7e0c1988b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009230 tx=0x7f7e00004270 comp rx=0 tx=0).stop 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 shutdown_connections 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e1361e640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f7df40a2da0 0x7f7df40a5260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.259+0000 7f7e1361e640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e0c107b70 0x7f7e0c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.261 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.260+0000 7f7e1361e640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e0c102470 0x7f7e0c1988b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.260+0000 7f7e1361e640 1 --2- 172.21.15.137:0/3336877496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e0c101ad0 0x7f7e0c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.260+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 >> 172.21.15.137:0/3336877496 conn(0x7f7e0c0fd7d0 msgr2=0x7f7e0c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.260+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 shutdown_connections 2024-09-12T12:32:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:11.260+0000 7f7e1361e640 1 -- 172.21.15.137:0/3336877496 wait complete. 2024-09-12T12:32:11.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:11 smithi189 ceph-mon[35475]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 347 op/s 2024-09-12T12:32:11.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:11 smithi171 ceph-mon[35098]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 347 op/s 2024-09-12T12:32:11.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:11 smithi137 ceph-mon[30950]: pgmap v603: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 347 op/s 2024-09-12T12:32:12.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.22 MB/sec execute 224 sec latency 16429.334 ms 2024-09-12T12:32:13.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.12 MB/sec execute 225 sec latency 17429.439 ms 2024-09-12T12:32:13.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:13 smithi189 ceph-mon[35475]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 5.5 MiB/s wr, 209 op/s 2024-09-12T12:32:13.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:13 smithi189 ceph-mon[35475]: osdmap e118: 12 total, 12 up, 12 in 2024-09-12T12:32:13.601 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:13 smithi137 ceph-mon[30950]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 5.5 MiB/s wr, 209 op/s 2024-09-12T12:32:13.601 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:13 smithi137 ceph-mon[30950]: osdmap e118: 12 total, 12 up, 12 in 2024-09-12T12:32:13.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:13 smithi171 ceph-mon[35098]: pgmap v604: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 5.5 MiB/s wr, 209 op/s 2024-09-12T12:32:13.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:13 smithi171 ceph-mon[35098]: osdmap e118: 12 total, 12 up, 12 in 2024-09-12T12:32:14.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 24.01 MB/sec execute 226 sec latency 18429.546 ms 2024-09-12T12:32:14.557 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:32:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:32:15.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 23.90 MB/sec execute 227 sec latency 19429.613 ms 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 -- 172.21.15.137:0/2938390370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370101c60 msgr2=0x7f0370105580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 --2- 172.21.15.137:0/2938390370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370101c60 0x7f0370105580 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0364009900 tx=0x7f036402a400 comp rx=0 tx=0).stop 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 -- 172.21.15.137:0/2938390370 shutdown_connections 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 --2- 172.21.15.137:0/2938390370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0370105cb0 0x7f037010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 --2- 172.21.15.137:0/2938390370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370101c60 0x7f0370105580 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 --2- 172.21.15.137:0/2938390370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 0x7f0370101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 -- 172.21.15.137:0/2938390370 >> 172.21.15.137:0/2938390370 conn(0x7f0370077070 msgr2=0x7f03700fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:15.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 -- 172.21.15.137:0/2938390370 shutdown_connections 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.449+0000 7f036f59e640 1 -- 172.21.15.137:0/2938390370 wait complete. 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.450+0000 7f036f59e640 1 Processor -- start 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.450+0000 7f036f59e640 1 -- start start 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.450+0000 7f036f59e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 0x7f037010e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.450+0000 7f036f59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0370101c60 0x7f037010e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.450+0000 7f036f59e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370105cb0 0x7f037010ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.450+0000 7f036f59e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0370111e20 con 0x7f0370105cb0 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.450+0000 7f036f59e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0370111ca0 con 0x7f03701012b0 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036f59e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0370111fa0 con 0x7f0370101c60 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 0x7f037010e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 0x7f037010e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:40352/0 (socket says 172.21.15.137:40352) 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 -- 172.21.15.137:0/3608112666 learned_addr learned my addr 172.21.15.137:0/3608112666 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:15.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036ed9d640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370105cb0 0x7f037010ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:15.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 -- 172.21.15.137:0/3608112666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0370101c60 msgr2=0x7f037010e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:15.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0370101c60 0x7f037010e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 -- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370105cb0 msgr2=0x7f037010ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:15.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370105cb0 0x7f037010ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 -- 172.21.15.137:0/3608112666 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03701aacd0 con 0x7f03701012b0 2024-09-12T12:32:15.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036ed9d640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370105cb0 0x7f037010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:15.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f036e59c640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 0x7f037010e000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f036800ebc0 tx=0x7f036800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:15.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.451+0000 7f035f7fe640 1 -- 172.21.15.137:0/3608112666 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368010630 con 0x7f03701012b0 2024-09-12T12:32:15.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.452+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03701aaf00 con 0x7f03701012b0 2024-09-12T12:32:15.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.452+0000 7f035f7fe640 1 -- 172.21.15.137:0/3608112666 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0368010cb0 con 0x7f03701012b0 2024-09-12T12:32:15.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.452+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03701ab3b0 con 0x7f03701012b0 2024-09-12T12:32:15.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.452+0000 7f035f7fe640 1 -- 172.21.15.137:0/3608112666 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368018c40 con 0x7f03701012b0 2024-09-12T12:32:15.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.453+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0350005290 con 0x7f03701012b0 2024-09-12T12:32:15.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.454+0000 7f035f7fe640 1 -- 172.21.15.137:0/3608112666 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f03680107d0 con 0x7f03701012b0 2024-09-12T12:32:15.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.455+0000 7f035f7fe640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03580a2d70 0x7f03580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:15.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.455+0000 7f035f7fe640 1 -- 172.21.15.137:0/3608112666 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f03680cfe10 con 0x7f03701012b0 2024-09-12T12:32:15.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.455+0000 7f036dd9b640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03580a2d70 0x7f03580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:15.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.456+0000 7f036dd9b640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03580a2d70 0x7f03580a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f036402a920 tx=0x7f0364000bc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:15.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.461+0000 7f035f7fe640 1 -- 172.21.15.137:0/3608112666 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f036809c9f0 con 0x7f03701012b0 2024-09-12T12:32:15.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:15 smithi189 ceph-mon[35475]: pgmap v606: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 2.9 MiB/s wr, 113 op/s 2024-09-12T12:32:15.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.663+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0350005580 con 0x7f03701012b0 2024-09-12T12:32:15.663 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:15 smithi137 ceph-mon[30950]: pgmap v606: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 2.9 MiB/s wr, 113 op/s 2024-09-12T12:32:15.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.664+0000 7f035f7fe640 1 -- 172.21.15.137:0/3608112666 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7f036809c2f0 con 0x7f03701012b0 2024-09-12T12:32:15.664 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:32:15.665 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":54,"btime":"2024-09-12T12:31:32:726301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:31:32.726248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26861":{"gid":26861,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/4049352466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4049352466},{"type":"v1","addr":"172.21.15.189:6835","nonce":4049352466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:32:15.665 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 54 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03580a2d70 msgr2=0x7f03580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03580a2d70 0x7f03580a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f036402a920 tx=0x7f0364000bc0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 msgr2=0x7f037010e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 0x7f037010e000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f036800ebc0 tx=0x7f036800c6a0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 shutdown_connections 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f03580a2d70 0x7f03580a5230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0370105cb0 0x7f037010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0370101c60 0x7f037010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 --2- 172.21.15.137:0/3608112666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03701012b0 0x7f037010e000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 >> 172.21.15.137:0/3608112666 conn(0x7f0370077070 msgr2=0x7f03700fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 shutdown_connections 2024-09-12T12:32:15.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:15.666+0000 7f036f59e640 1 -- 172.21.15.137:0/3608112666 wait complete. 2024-09-12T12:32:15.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:15 smithi171 ceph-mon[35098]: pgmap v606: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 2.9 MiB/s wr, 113 op/s 2024-09-12T12:32:16.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 23.80 MB/sec execute 228 sec latency 20429.692 ms 2024-09-12T12:32:16.508 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id c318f5ad --release --await' 2024-09-12T12:32:16.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id c318f5ad --release --await 2024-09-12T12:32:16.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:16 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3608112666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:16.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:16 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3608112666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:16.757 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:32:16.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:16 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3608112666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:17.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132132 23.69 MB/sec execute 229 sec latency 21429.756 ms 2024-09-12T12:32:17.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 -- 172.21.15.137:0/2121869993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948101790 msgr2=0x7f2948101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 --2- 172.21.15.137:0/2121869993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948101790 0x7f2948101bd0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f293c009900 tx=0x7f293c02a480 comp rx=0 tx=0).stop 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 -- 172.21.15.137:0/2121869993 shutdown_connections 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 --2- 172.21.15.137:0/2121869993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2948102110 0x7f294810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 --2- 172.21.15.137:0/2121869993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948101790 0x7f2948101bd0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 --2- 172.21.15.137:0/2121869993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948105ba0 0x7f2948105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 -- 172.21.15.137:0/2121869993 >> 172.21.15.137:0/2121869993 conn(0x7f2948077210 msgr2=0x7f2948077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 -- 172.21.15.137:0/2121869993 shutdown_connections 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.236+0000 7f294d2b2640 1 -- 172.21.15.137:0/2121869993 wait complete. 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 Processor -- start 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 -- start start 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2948101790 0x7f2948199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948102110 0x7f29481996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948105ba0 0x7f29481a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2948110ea0 con 0x7f2948105ba0 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2948110d20 con 0x7f2948102110 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.237+0000 7f294d2b2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2948111020 con 0x7f2948101790 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948105ba0 0x7f29481a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948105ba0 0x7f29481a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53202/0 (socket says 172.21.15.137:53202) 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 -- 172.21.15.137:0/821751951 learned_addr learned my addr 172.21.15.137:0/821751951 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f2946ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2948101790 0x7f2948199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29467fc640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948102110 0x7f29481996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 -- 172.21.15.137:0/821751951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2948101790 msgr2=0x7f2948199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2948101790 0x7f2948199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 -- 172.21.15.137:0/821751951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948102110 msgr2=0x7f29481996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948102110 0x7f29481996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 -- 172.21.15.137:0/821751951 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29481a4750 con 0x7f2948105ba0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29467fc640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948102110 0x7f29481996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f29477fe640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948105ba0 0x7f29481a4210 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f294000d750 tx=0x7f294000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f292ffff640 1 -- 172.21.15.137:0/821751951 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940017070 con 0x7f2948105ba0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29481a4920 con 0x7f2948105ba0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f292ffff640 1 -- 172.21.15.137:0/821751951 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2940005500 con 0x7f2948105ba0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f292ffff640 1 -- 172.21.15.137:0/821751951 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940004d90 con 0x7f2948105ba0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.238+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29481a4e10 con 0x7f2948105ba0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.240+0000 7f292ffff640 1 -- 172.21.15.137:0/821751951 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2940004f30 con 0x7f2948105ba0 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 Processor -- start 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 -- start start 2024-09-12T12:32:17.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff85f00 0x56079ff862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x56079ff97a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56079feeb620 con 0x56079ff95650 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56079fe92e50 con 0x56079ff86820 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f294f68b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56079fee75e0 con 0x56079ff85f00 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f292ffff640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29300a2cf0 0x7f29300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x56079ff97a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.241+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x56079ff97a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53204/0 (socket says 172.21.15.137:53204) 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 -- 172.21.15.137:0/2295413607 learned_addr learned my addr 172.21.15.137:0/2295413607 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff85f00 0x56079ff862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29477fe640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29300a2cf0 0x7f29300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f2946ffd640 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f292ffff640 1 -- 172.21.15.137:0/821751951 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f29400cee40 con 0x7f2948105ba0 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 -- 172.21.15.137:0/2295413607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff85f00 msgr2=0x56079ff862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff85f00 0x56079ff862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 -- 172.21.15.137:0/2295413607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 msgr2=0x56079ff95110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff95110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 -- 172.21.15.137:0/2295413607 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079fe83d40 con 0x56079ff95650 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f2946ffd640 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff95110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29467fc640 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x56079ff97a40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f293c0314e0 tx=0x7f293c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f61edc62f38b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f29477fe640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29300a2cf0 0x7f29300a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2940097780 tx=0x7f2940000bc0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f292d7fa640 1 -- 172.21.15.137:0/2295413607 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c035040 con 0x56079ff95650 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f292d7fa640 1 -- 172.21.15.137:0/2295413607 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c031540 con 0x56079ff95650 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.242+0000 7f292d7fa640 1 -- 172.21.15.137:0/2295413607 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c03d680 con 0x56079ff95650 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 -- 172.21.15.137:0/2295413607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 msgr2=0x56079ff97a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x56079ff97a40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f293c0314e0 tx=0x7f293c0056d0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 -- 172.21.15.137:0/2295413607 shutdown_connections 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x56079ff97a40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff95110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 --2- 172.21.15.137:0/2295413607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff85f00 0x56079ff862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 -- 172.21.15.137:0/2295413607 >> 172.21.15.137:0/2295413607 conn(0x56079ff825f0 msgr2=0x56079ff829d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 -- 172.21.15.137:0/2295413607 shutdown_connections 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 -- 172.21.15.137:0/2295413607 wait complete. 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 -- start start 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.243+0000 7f294f68b740 1 Processor -- start 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff925e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff92b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff95650 0x56079ff93060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x56079ff98220 con 0x56079ff85f00 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56079fe73f10 con 0x56079ff86820 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56079ff983a0 con 0x56079ff95650 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff925e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 10 client.0 _ll_register_callbacks cb 0x56079fe97db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff925e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53220/0 (socket says 172.21.15.137:53220) 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 -- 172.21.15.137:0/556741775 learned_addr learned my addr 172.21.15.137:0/556741775 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f2946ffd640 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff92b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 -- 172.21.15.137:0/556741775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff95650 msgr2=0x56079ff93060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff95650 0x56079ff93060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 -- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 msgr2=0x56079ff92b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff92b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 -- 172.21.15.137:0/556741775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079fbfa580 con 0x56079ff85f00 2024-09-12T12:32:17.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f2946ffd640 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff92b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f29467fc640 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff925e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f293c005ab0 tx=0x7f293c004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db606c29c169da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f292aff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f292aff5640 1 -- 172.21.15.137:0/556741775 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c000e60 con 0x56079ff85f00 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f292aff5640 1 -- 172.21.15.137:0/556741775 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f293c031b00 con 0x56079ff85f00 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.244+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293c031540 con 0x56079ff85f00 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f292aff5640 1 -- 172.21.15.137:0/556741775 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f293c035040 con 0x56079ff85f00 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 10 client.17721 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56079fcc4960 con 0x56079ff85f00 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f292aff5640 1 -- 172.21.15.137:0/556741775 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c031540 con 0x56079ff85f00 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f292aff5640 1 -- 172.21.15.137:0/556741775 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f293c05b050 con 0x56079ff85f00 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 10 client.17721 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:17.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 10 client.17721 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 20 client.17721 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56079ffa5130 0x56079ffa7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 4 client.17721 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56079fef8b90 con 0x56079ffa5130 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.245+0000 7f2946ffd640 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56079ffa5130 0x56079ffa7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.246+0000 7f2946ffd640 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56079ffa5130 0x56079ffa7520 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b5c6afc13a456e server_cookie=67bd7f4c74eca445 in_seq=0 out_seq=0 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.246+0000 7f292aff5640 10 client.17721 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.246+0000 7f292aff5640 1 -- 172.21.15.137:0/556741775 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56079fef8b90 con 0x56079ffa5130 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.246+0000 7f292aff5640 10 client.17721 handle_command_reply: tid=0 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.246+0000 7f294f68b740 1 client.17721 shutdown 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.246+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 msgr2=0x56079ff925e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.246+0000 7f294f68b740 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff925e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f293c005ab0 tx=0x7f293c004580 comp rx=0 tx=0).stop 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 shutdown_connections 2024-09-12T12:32:17.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff95650 0x56079ff93060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff86820 0x56079ff92b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x56079ffa5130 0x56079ffa7520 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 --2- 172.21.15.137:0/556741775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff925e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 >> 172.21.15.137:0/556741775 conn(0x56079ff825f0 msgr2=0x56079ff81750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f292aff5640 0 client.17721 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 shutdown_connections 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 1 -- 172.21.15.137:0/556741775 wait complete. 2024-09-12T12:32:17.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.247+0000 7f294f68b740 20 client.17721 trim_cache size 0 max 16384 2024-09-12T12:32:17.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 Processor -- start 2024-09-12T12:32:17.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 -- start start 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff81640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ff8f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff95650 0x56079ff8fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f29481ade50 con 0x56079ff85f00 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f29481ab720 con 0x56079ff95650 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f294f68b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29481a95b0 con 0x56079ff86820 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff81640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff81640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53222/0 (socket says 172.21.15.137:53222) 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 -- 172.21.15.137:0/4175201850 learned_addr learned my addr 172.21.15.137:0/4175201850 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff95650 0x56079ff8fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f2946ffd640 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ff8f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 -- 172.21.15.137:0/4175201850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 msgr2=0x56079ff8f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ff8f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 -- 172.21.15.137:0/4175201850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff95650 msgr2=0x56079ff8fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff95650 0x56079ff8fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.264+0000 7f29467fc640 1 -- 172.21.15.137:0/4175201850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56079ff86610 con 0x56079ff85f00 2024-09-12T12:32:17.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f29467fc640 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff81640 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f293c03dcd0 tx=0x7f293c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1af96df7712c5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:17.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f292cff9640 1 -- 172.21.15.137:0/4175201850 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c005d50 con 0x56079ff85f00 2024-09-12T12:32:17.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f292cff9640 1 -- 172.21.15.137:0/4175201850 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c031540 con 0x56079ff85f00 2024-09-12T12:32:17.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f292cff9640 1 -- 172.21.15.137:0/4175201850 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0052a0 con 0x56079ff85f00 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f294f68b740 1 -- 172.21.15.137:0/4175201850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 msgr2=0x56079ff81640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f294f68b740 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff81640 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f293c03dcd0 tx=0x7f293c005a20 comp rx=0 tx=0).stop 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f294f68b740 1 -- 172.21.15.137:0/4175201850 shutdown_connections 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f294f68b740 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff95650 0x56079ff8fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f294f68b740 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ff8f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f294f68b740 1 --2- 172.21.15.137:0/4175201850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff85f00 0x56079ff81640 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.265+0000 7f294f68b740 1 -- 172.21.15.137:0/4175201850 >> 172.21.15.137:0/4175201850 conn(0x56079ff825f0 msgr2=0x56079ff80f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 -- 172.21.15.137:0/4175201850 shutdown_connections 2024-09-12T12:32:17.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 -- 172.21.15.137:0/4175201850 wait complete. 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 -- start start 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 Processor -- start 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff85f00 0x56079ffa82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ffa8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x5607a000ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f29481a9c00 con 0x56079ff95650 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f29481ab050 con 0x56079ff85f00 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29481aa2b0 con 0x56079ff86820 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f294f68b740 10 client.0 _ll_register_callbacks cb 0x7f293c003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x5607a000ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f29467fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x5607a000ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53224/0 (socket says 172.21.15.137:53224) 2024-09-12T12:32:17.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f29467fc640 1 -- 172.21.15.137:0/1641684592 learned_addr learned my addr 172.21.15.137:0/1641684592 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f29467fc640 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff85f00 0x56079ffa82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.266+0000 7f2946ffd640 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ffa8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f29467fc640 1 -- 172.21.15.137:0/1641684592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 msgr2=0x56079ffa8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f29467fc640 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ffa8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f29467fc640 1 -- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff85f00 msgr2=0x56079ffa82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f29467fc640 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff85f00 0x56079ffa82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f29467fc640 1 -- 172.21.15.137:0/1641684592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56079ffa9120 con 0x56079ff95650 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f29467fc640 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x5607a000ea70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f293c040040 tx=0x7f293c0562a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7444500d90b7cbe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f292aff5640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f292aff5640 1 -- 172.21.15.137:0/1641684592 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c005d50 con 0x56079ff95650 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f292aff5640 1 -- 172.21.15.137:0/1641684592 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f293c005040 con 0x56079ff95650 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293c031540 con 0x56079ff95650 2024-09-12T12:32:17.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f292aff5640 1 -- 172.21.15.137:0/1641684592 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f293c0040d0 con 0x56079ff95650 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f294f68b740 10 client.17727 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56079ffa9120 con 0x56079ff95650 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f292aff5640 1 -- 172.21.15.137:0/1641684592 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f293c031540 con 0x56079ff95650 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.267+0000 7f292aff5640 1 -- 172.21.15.137:0/1641684592 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f293c05b030 con 0x56079ff95650 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f294f68b740 10 client.17727 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f294f68b740 10 client.17727 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f294f68b740 20 client.17727 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f294f68b740 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5607a00103c0 0x5607a00127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f294f68b740 4 client.17727 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c318f5ad", "release": true, "await": true}] 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c318f5ad", "release": true, "await": true}) -- 0x56079fef30a0 con 0x5607a00103c0 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f2946ffd640 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5607a00103c0 0x5607a00127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f292aff5640 10 client.17727 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:17.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.268+0000 7f2946ffd640 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5607a00103c0 0x5607a00127b0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f901f5757b2fe605 server_cookie=3105fa743ae4771d in_seq=0 out_seq=0 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.313+0000 7f292aff5640 1 -- 172.21.15.137:0/1641684592 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56079fef30a0 con 0x5607a00103c0 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.313+0000 7f292aff5640 10 client.17727 handle_command_reply: tid=0 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.313+0000 7f294f68b740 1 client.17727 shutdown 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 msgr2=0x5607a000ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x5607a000ea70 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f293c040040 tx=0x7f293c0562a0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 shutdown_connections 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5607a00103c0 0x5607a00127b0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56079ff95650 0x5607a000ea70 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56079ff86820 0x56079ffa8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f292aff5640 0 client.17727 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 --2- 172.21.15.137:0/1641684592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56079ff85f00 0x56079ffa82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 >> 172.21.15.137:0/1641684592 conn(0x56079ff825f0 msgr2=0x56079ff92dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:17.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 shutdown_connections 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 1 -- 172.21.15.137:0/1641684592 wait complete. 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294f68b740 20 client.17727 trim_cache size 0 max 16384 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 54, 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 68, 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stdout: "c318f5ad": { 2024-09-12T12:32:17.316 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 68, 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 141.8, 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:32:17.317 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:32:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:17.318 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:17.318 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:32:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29300a2cf0 msgr2=0x7f29300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294d2b2640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29300a2cf0 0x7f29300a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2940097780 tx=0x7f2940000bc0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.314+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948105ba0 msgr2=0x7f29481a4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948105ba0 0x7f29481a4210 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f294000d750 tx=0x7f294000dc10 comp rx=0 tx=0).stop 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 shutdown_connections 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f29300a2cf0 0x7f29300a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2948105ba0 0x7f29481a4210 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2948102110 0x7f29481996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 --2- 172.21.15.137:0/821751951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2948101790 0x7f2948199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 >> 172.21.15.137:0/821751951 conn(0x7f2948077210 msgr2=0x7f29480ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 shutdown_connections 2024-09-12T12:32:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:17.315+0000 7f294d2b2640 1 -- 172.21.15.137:0/821751951 wait complete. 2024-09-12T12:32:17.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:17 smithi137 ceph-mon[30950]: pgmap v607: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 6.3 MiB/s wr, 258 op/s 2024-09-12T12:32:17.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:17 smithi189 ceph-mon[35475]: pgmap v607: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 6.3 MiB/s wr, 258 op/s 2024-09-12T12:32:17.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:17 smithi171 ceph-mon[35098]: pgmap v607: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 6.3 MiB/s wr, 258 op/s 2024-09-12T12:32:18.117 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c318f5ad', total seconds elapsed: 25.4 2024-09-12T12:32:18.117 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.8 2024-09-12T12:32:18.117 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:32:18.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 132629 23.70 MB/sec execute 230 sec latency 21517.178 ms 2024-09-12T12:32:18.447 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:32:19.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 -- 172.21.15.137:0/3384027765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c105f50 msgr2=0x7f0a0c1063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:19.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 --2- 172.21.15.137:0/3384027765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c105f50 0x7f0a0c1063e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009ca0 tx=0x7f0a0402a940 comp rx=0 tx=0).stop 2024-09-12T12:32:19.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 -- 172.21.15.137:0/3384027765 shutdown_connections 2024-09-12T12:32:19.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 --2- 172.21.15.137:0/3384027765 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c105f50 0x7f0a0c1063e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 --2- 172.21.15.137:0/3384027765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a0c101460 0x7f0a0c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 --2- 172.21.15.137:0/3384027765 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 0x7f0a0c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 -- 172.21.15.137:0/3384027765 >> 172.21.15.137:0/3384027765 conn(0x7f0a0c0fc7d0 msgr2=0x7f0a0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 -- 172.21.15.137:0/3384027765 shutdown_connections 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.990+0000 7f0a114d4640 1 -- 172.21.15.137:0/3384027765 wait complete. 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 Processor -- start 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 -- start start 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 0x7f0a0c1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c101460 0x7f0a0c1a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a0c105f50 0x7f0a0c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c118570 con 0x7f0a0c101460 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1183f0 con 0x7f0a0c100b40 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.991+0000 7f0a114d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c1186f0 con 0x7f0a0c105f50 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c101460 0x7f0a0c1a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 0x7f0a0c1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:19.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 0x7f0a0c1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:40422/0 (socket says 172.21.15.137:40422) 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 -- 172.21.15.137:0/1433105817 learned_addr learned my addr 172.21.15.137:0/1433105817 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0a7fc640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c101460 0x7f0a0c1a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53236/0 (socket says 172.21.15.137:53236) 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0b7fe640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a0c105f50 0x7f0a0c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 -- 172.21.15.137:0/1433105817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a0c105f50 msgr2=0x7f0a0c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a0c105f50 0x7f0a0c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 -- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c101460 msgr2=0x7f0a0c1a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c101460 0x7f0a0c1a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 -- 172.21.15.137:0/1433105817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c19f050 con 0x7f0a0c100b40 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0a7fc640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c101460 0x7f0a0c1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:19.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a0affd640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 0x7f0a0c1a4760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f09f3fff640 1 -- 172.21.15.137:0/1433105817 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00cd80 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c19f340 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f09f3fff640 1 -- 172.21.15.137:0/1433105817 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09fc010910 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f09f3fff640 1 -- 172.21.15.137:0/1433105817 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0188f0 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.992+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c1ab630 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.993+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0c105930 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.995+0000 7f09f3fff640 1 -- 172.21.15.137:0/1433105817 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f09fc010ac0 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.995+0000 7f09f3fff640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09f40a2cf0 0x7f09f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.995+0000 7f0a0a7fc640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09f40a2cf0 0x7f09f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.995+0000 7f09f3fff640 1 -- 172.21.15.137:0/1433105817 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f09fc0cfbc0 con 0x7f0a0c100b40 2024-09-12T12:32:19.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:18.996+0000 7f0a0a7fc640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09f40a2cf0 0x7f09f40a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c106690 tx=0x7f0a00005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:19.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.000+0000 7f09f3fff640 1 -- 172.21.15.137:0/1433105817 <== mon.2 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09fc09c1b0 con 0x7f0a0c100b40 2024-09-12T12:32:19.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 133211 23.70 MB/sec execute 231 sec latency 136.326 ms 2024-09-12T12:32:19.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.263+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a0c1ab860 con 0x7f0a0c100b40 2024-09-12T12:32:19.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.264+0000 7f09f3fff640 1 -- 172.21.15.137:0/1433105817 <== mon.2 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7f09fc0a04b0 con 0x7f0a0c100b40 2024-09-12T12:32:19.266 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:32:19.266 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":54,"btime":"2024-09-12T12:31:32:726301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:31:32.726248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26861":{"gid":26861,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/4049352466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4049352466},{"type":"v1","addr":"172.21.15.189:6835","nonce":4049352466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:32:19.266 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 54 2024-09-12T12:32:19.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09f40a2cf0 msgr2=0x7f09f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:19.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09f40a2cf0 0x7f09f40a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c106690 tx=0x7f0a00005f90 comp rx=0 tx=0).stop 2024-09-12T12:32:19.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 msgr2=0x7f0a0c1a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:19.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 0x7f0a0c1a4760 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00e9f0 tx=0x7f09fc00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 shutdown_connections 2024-09-12T12:32:19.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f09f40a2cf0 0x7f09f40a51b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0a0c105f50 0x7f0a0c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a0c101460 0x7f0a0c1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 --2- 172.21.15.137:0/1433105817 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a0c100b40 0x7f0a0c1a4760 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:19.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.268+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 >> 172.21.15.137:0/1433105817 conn(0x7f0a0c0fc7d0 msgr2=0x7f0a0c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:19.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.269+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 shutdown_connections 2024-09-12T12:32:19.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:19.269+0000 7f0a114d4640 1 -- 172.21.15.137:0/1433105817 wait complete. 2024-09-12T12:32:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:19 smithi137 ceph-mon[30950]: pgmap v608: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 5.3 MiB/s wr, 215 op/s 2024-09-12T12:32:19.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:19 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1433105817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:19.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:19 smithi189 ceph-mon[35475]: pgmap v608: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 5.3 MiB/s wr, 215 op/s 2024-09-12T12:32:19.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:19 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1433105817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:19.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:19 smithi171 ceph-mon[35098]: pgmap v608: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 5.3 MiB/s wr, 215 op/s 2024-09-12T12:32:19.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:19 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1433105817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:20.058 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0' 2024-09-12T12:32:20.059 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db / --timeout 90 --expiration 141.8 --await-for 45.0 2024-09-12T12:32:20.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 133661 23.69 MB/sec execute 232 sec latency 288.243 ms 2024-09-12T12:32:20.365 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:32:21.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 -- 172.21.15.137:0/970676616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 msgr2=0x7faf8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 --2- 172.21.15.137:0/970676616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 0x7faf8410da40 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009ca0 tx=0x7faf7c02a950 comp rx=0 tx=0).stop 2024-09-12T12:32:21.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 -- 172.21.15.137:0/970676616 shutdown_connections 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 --2- 172.21.15.137:0/970676616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 0x7faf8410da40 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 --2- 172.21.15.137:0/970676616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf84102570 0x7faf841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 --2- 172.21.15.137:0/970676616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 0x7faf84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 -- 172.21.15.137:0/970676616 >> 172.21.15.137:0/970676616 conn(0x7faf840fc7d0 msgr2=0x7faf840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 -- 172.21.15.137:0/970676616 shutdown_connections 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.104+0000 7faf8881d640 1 -- 172.21.15.137:0/970676616 wait complete. 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.105+0000 7faf8881d640 1 Processor -- start 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.105+0000 7faf8881d640 1 -- start start 2024-09-12T12:32:21.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8881d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 0x7faf841993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8881d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf84102570 0x7faf84199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8881d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 0x7faf84199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8881d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf84110f80 con 0x7faf84102ef0 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8881d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf84110e00 con 0x7faf84102570 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8881d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf84111100 con 0x7faf84100e20 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 0x7faf841993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 0x7faf841993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54870/0 (socket says 172.21.15.137:54870) 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8259c640 1 -- 172.21.15.137:0/712981502 learned_addr learned my addr 172.21.15.137:0/712981502 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf82d9d640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 0x7faf84199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf79d9b640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf84102570 0x7faf84199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf79d9b640 1 -- 172.21.15.137:0/712981502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 msgr2=0x7faf841993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf79d9b640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 0x7faf841993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf79d9b640 1 -- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 msgr2=0x7faf84199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf79d9b640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 0x7faf84199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf79d9b640 1 -- 172.21.15.137:0/712981502 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf841a47d0 con 0x7faf84102570 2024-09-12T12:32:21.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.106+0000 7faf8259c640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 0x7faf841993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:21.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.107+0000 7faf79d9b640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf84102570 0x7faf84199910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00e9f0 tx=0x7faf6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:21.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.107+0000 7faf7bfff640 1 -- 172.21.15.137:0/712981502 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00ccc0 con 0x7faf84102570 2024-09-12T12:32:21.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.107+0000 7faf7bfff640 1 -- 172.21.15.137:0/712981502 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf6c010910 con 0x7faf84102570 2024-09-12T12:32:21.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.107+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf841a4ac0 con 0x7faf84102570 2024-09-12T12:32:21.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.107+0000 7faf7bfff640 1 -- 172.21.15.137:0/712981502 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c018960 con 0x7faf84102570 2024-09-12T12:32:21.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.107+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf841a5000 con 0x7faf84102570 2024-09-12T12:32:21.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 Processor -- start 2024-09-12T12:32:21.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 -- start start 2024-09-12T12:32:21.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf7bfff640 1 -- 172.21.15.137:0/712981502 <== mon.2 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7faf6c010ac0 con 0x7faf84102570 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562eff73e2a0 con 0x562eff7e3940 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562eff6e5b10 con 0x562eff7d9080 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8abf6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562eff73a260 con 0x562eff7d8760 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7d8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54874/0 (socket says 172.21.15.137:54874) 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf8259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf82d9d640 1 -- 172.21.15.137:0/3743646483 learned_addr learned my addr 172.21.15.137:0/3743646483 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.110+0000 7faf79d9b640 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf82d9d640 1 -- 172.21.15.137:0/3743646483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 msgr2=0x562eff7e3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf82d9d640 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7e3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf82d9d640 1 -- 172.21.15.137:0/3743646483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 msgr2=0x562eff7e5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf82d9d640 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7e5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf82d9d640 1 -- 172.21.15.137:0/3743646483 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eff4afb90 con 0x562eff7d8760 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf8259c640 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf79d9b640 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf7bfff640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faf700a2cf0 0x7faf700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf82d9d640 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7d8b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009ca0 tx=0x7faf7c0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17e97f9f660540e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf8259c640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faf700a2cf0 0x7faf700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.111+0000 7faf7bfff640 1 -- 172.21.15.137:0/712981502 <== mon.2 v2:172.21.15.171:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7faf6c0cf740 con 0x7faf84102570 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.112+0000 7faf78d99640 1 -- 172.21.15.137:0/3743646483 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c048070 con 0x562eff7d8760 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.112+0000 7faf78d99640 1 -- 172.21.15.137:0/3743646483 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c02f930 con 0x562eff7d8760 2024-09-12T12:32:21.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.112+0000 7faf78d99640 1 -- 172.21.15.137:0/3743646483 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c034040 con 0x562eff7d8760 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 -- 172.21.15.137:0/3743646483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 msgr2=0x562eff7d8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7d8b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009ca0 tx=0x7faf7c0093e0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8259c640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faf700a2cf0 0x7faf700a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x562eff7d5b50 tx=0x7faf74005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 -- 172.21.15.137:0/3743646483 shutdown_connections 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 --2- 172.21.15.137:0/3743646483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7d8b40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 -- 172.21.15.137:0/3743646483 >> 172.21.15.137:0/3743646483 conn(0x562eff7d4cf0 msgr2=0x562eff7d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 -- 172.21.15.137:0/3743646483 shutdown_connections 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 -- 172.21.15.137:0/3743646483 wait complete. 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 -- start start 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.113+0000 7faf8abf6740 1 Processor -- start 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d9080 0x562eff7e0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7e3940 0x562eff7e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8abf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562eff7e6510 con 0x562eff7d8760 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8abf6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562eff6c66e0 con 0x562eff7e3940 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8abf6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562eff7e6690 con 0x562eff7d9080 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8abf6740 10 client.0 _ll_register_callbacks cb 0x562eff6eaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf79d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d9080 0x562eff7e0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf79d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d9080 0x562eff7e0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:54890/0 (socket says 172.21.15.137:54890) 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf79d9b640 1 -- 172.21.15.137:0/4007897405 learned_addr learned my addr 172.21.15.137:0/4007897405 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7e3940 0x562eff7e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf82d9d640 1 -- 172.21.15.137:0/4007897405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d9080 msgr2=0x562eff7e0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d9080 0x562eff7e0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf82d9d640 1 -- 172.21.15.137:0/4007897405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 msgr2=0x562eff7e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7e0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf82d9d640 1 -- 172.21.15.137:0/4007897405 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eff7efa00 con 0x562eff7e3940 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf79d9b640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d9080 0x562eff7e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.114+0000 7faf8259c640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7e3940 0x562eff7e12a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf7c030040 tx=0x7faf7c009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dafa9d078ae2e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf69ffb640 1 -- 172.21.15.137:0/4007897405 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c048030 con 0x562eff7e3940 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c02f930 con 0x562eff7e3940 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf69ffb640 1 -- 172.21.15.137:0/4007897405 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7faf7c040040 con 0x562eff7e3940 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf69ffb640 1 -- 172.21.15.137:0/4007897405 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7faf7c034050 con 0x562eff7e3940 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf8abf6740 10 client.26966 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562eff7f2390 con 0x562eff7e3940 2024-09-12T12:32:21.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf69ffb640 1 -- 172.21.15.137:0/4007897405 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c02f930 con 0x562eff7e3940 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.115+0000 7faf69ffb640 1 -- 172.21.15.137:0/4007897405 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7faf7c055070 con 0x562eff7e3940 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8abf6740 10 client.26966 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8abf6740 10 client.26966 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8abf6740 20 client.26966 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff7f36e0 0x562eff7f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8abf6740 4 client.26966 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eff74b9d0 con 0x562eff7f36e0 2024-09-12T12:32:21.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8259c640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff7f36e0 0x562eff7f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf8259c640 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff7f36e0 0x562eff7f5ad0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9b33794e36df5a server_cookie=528fa163dd1c2e31 in_seq=0 out_seq=0 2024-09-12T12:32:21.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.116+0000 7faf69ffb640 10 client.26966 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:21.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.118+0000 7faf69ffb640 1 -- 172.21.15.137:0/4007897405 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eff74b9d0 con 0x562eff7f36e0 2024-09-12T12:32:21.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.118+0000 7faf69ffb640 10 client.26966 handle_command_reply: tid=0 2024-09-12T12:32:21.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.118+0000 7faf8abf6740 1 client.26966 shutdown 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7e3940 msgr2=0x562eff7e12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7e3940 0x562eff7e12a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faf7c030040 tx=0x7faf7c009960 comp rx=0 tx=0).stop 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 shutdown_connections 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7e3940 0x562eff7e12a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d9080 0x562eff7e0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff7f36e0 0x562eff7f5ad0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4007897405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 >> 172.21.15.137:0/4007897405 conn(0x562eff7d4cf0 msgr2=0x562eff7d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf69ffb640 0 client.26966 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 shutdown_connections 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 1 -- 172.21.15.137:0/4007897405 wait complete. 2024-09-12T12:32:21.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.119+0000 7faf8abf6740 20 client.26966 trim_cache size 0 max 16384 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.136+0000 7faf8abf6740 1 Processor -- start 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8abf6740 1 -- start start 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8abf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf841add80 con 0x562eff7e3940 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8abf6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf841ab650 con 0x562eff7d9080 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8abf6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf841a94e0 con 0x562eff7d8760 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7f8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51364/0 (socket says 172.21.15.137:51364) 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf8259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 -- 172.21.15.137:0/4262368499 learned_addr learned my addr 172.21.15.137:0/4262368499 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf79d9b640 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 -- 172.21.15.137:0/4262368499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 msgr2=0x562eff7eeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7eeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 -- 172.21.15.137:0/4262368499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 msgr2=0x562eff7f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf82d9d640 1 -- 172.21.15.137:0/4262368499 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eff7d0260 con 0x562eff7e3940 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.137+0000 7faf79d9b640 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.138+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7f8c50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7faf7c002230 tx=0x7faf7c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4c5ef68cac381b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.138+0000 7faf6bfff640 1 -- 172.21.15.137:0/4262368499 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c05b020 con 0x562eff7e3940 2024-09-12T12:32:21.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.138+0000 7faf6bfff640 1 -- 172.21.15.137:0/4262368499 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c04a070 con 0x562eff7e3940 2024-09-12T12:32:21.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf6bfff640 1 -- 172.21.15.137:0/4262368499 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7c048030 con 0x562eff7e3940 2024-09-12T12:32:21.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 -- 172.21.15.137:0/4262368499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 msgr2=0x562eff7f8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7f8c50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7faf7c002230 tx=0x7faf7c0093e0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 -- 172.21.15.137:0/4262368499 shutdown_connections 2024-09-12T12:32:21.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7e3940 0x562eff7f8c50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4262368499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7d8760 0x562eff7eeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 -- 172.21.15.137:0/4262368499 >> 172.21.15.137:0/4262368499 conn(0x562eff7d4cf0 msgr2=0x562eff7ee8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:21.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 -- 172.21.15.137:0/4262368499 shutdown_connections 2024-09-12T12:32:21.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.139+0000 7faf8abf6740 1 -- 172.21.15.137:0/4262368499 wait complete. 2024-09-12T12:32:21.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.140+0000 7faf8abf6740 1 -- start start 2024-09-12T12:32:21.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.140+0000 7faf8abf6740 1 Processor -- start 2024-09-12T12:32:21.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.140+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.140+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.140+0000 7faf8abf6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7e3940 0x562eff7f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf79d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51372/0 (socket says 172.21.15.137:51372) 2024-09-12T12:32:21.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf79d9b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:59154/0 (socket says 172.21.15.137:59154) 2024-09-12T12:32:21.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 -- 172.21.15.137:0/4152355695 learned_addr learned my addr 172.21.15.137:0/4152355695 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:21.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8abf6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf841a9b30 con 0x562eff7d8760 2024-09-12T12:32:21.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faf841aaf80 con 0x562eff7d9080 2024-09-12T12:32:21.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf841aa1e0 con 0x562eff7e3940 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8abf6740 10 client.0 _ll_register_callbacks cb 0x7faf74009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf82d9d640 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7e3940 0x562eff7f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 -- 172.21.15.137:0/4152355695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7e3940 msgr2=0x562eff7f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7e3940 0x562eff7f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 -- 172.21.15.137:0/4152355695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 msgr2=0x562eff7f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 -- 172.21.15.137:0/4152355695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eff7db5d0 con 0x562eff7d8760 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf8259c640 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7f65c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7faf74009de0 tx=0x7faf740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5b52536b873a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:21.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.141+0000 7faf69ffb640 1 -- 172.21.15.137:0/4152355695 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7401dad0 con 0x562eff7d8760 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf7c04a070 con 0x562eff7d8760 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf69ffb640 1 -- 172.21.15.137:0/4152355695 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7faf7401dc70 con 0x562eff7d8760 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf69ffb640 1 -- 172.21.15.137:0/4152355695 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7faf74033430 con 0x562eff7d8760 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 10 client.17751 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562eff866a50 con 0x562eff7d8760 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf69ffb640 1 -- 172.21.15.137:0/4152355695 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf7c04a070 con 0x562eff7d8760 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf69ffb640 1 -- 172.21.15.137:0/4152355695 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7faf7403a070 con 0x562eff7d8760 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 10 client.17751 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 10 client.17751 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 20 client.17751 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff867290 0x562eff8696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 4 client.17751 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}] 2024-09-12T12:32:21.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.142+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.8, "await_for": 45.0}) -- 0x562eff745d20 con 0x562eff867290 2024-09-12T12:32:21.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.143+0000 7faf79d9b640 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff867290 0x562eff8696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:21.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.143+0000 7faf69ffb640 10 client.17751 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:21.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.143+0000 7faf79d9b640 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff867290 0x562eff8696b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8dcec3b7cc18683 server_cookie=9a7c8953e248184a in_seq=0 out_seq=0 2024-09-12T12:32:21.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134164 23.74 MB/sec execute 233 sec latency 153.184 ms 2024-09-12T12:32:21.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:21 smithi137 ceph-mon[30950]: pgmap v609: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 5.5 MiB/s wr, 229 op/s 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.696+0000 7faf69ffb640 1 -- 172.21.15.137:0/4152355695 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562eff745d20 con 0x562eff867290 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.696+0000 7faf69ffb640 10 client.17751 handle_command_reply: tid=0 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.696+0000 7faf8abf6740 1 client.17751 shutdown 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 msgr2=0x562eff7f65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7f65c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7faf74009de0 tx=0x7faf740046c0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 shutdown_connections 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562eff7e3940 0x562eff7f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562eff7d9080 0x562eff7f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x562eff867290 0x562eff8696b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 --2- 172.21.15.137:0/4152355695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562eff7d8760 0x562eff7f65c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 >> 172.21.15.137:0/4152355695 conn(0x562eff7d4cf0 msgr2=0x562eff7e0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf69ffb640 0 client.17751 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:21.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 shutdown_connections 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 1 -- 172.21.15.137:0/4152355695 wait complete. 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.697+0000 7faf8abf6740 20 client.17751 trim_cache size 0 max 16384 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 54, 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 70, 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "6a68da80": { 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 70, 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:32:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 141.8, 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "QUIESCED", 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:21.701 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:32:21.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faf700a2cf0 msgr2=0x7faf700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faf700a2cf0 0x7faf700a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x562eff7d5b50 tx=0x7faf74005f90 comp rx=0 tx=0).stop 2024-09-12T12:32:21.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf84102570 msgr2=0x7faf84199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:21.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf84102570 0x7faf84199910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00e9f0 tx=0x7faf6c00eeb0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 shutdown_connections 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7faf700a2cf0 0x7faf700a51b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf84102ef0 0x7faf84199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faf84102570 0x7faf84199910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 --2- 172.21.15.137:0/712981502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf84100e20 0x7faf841993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 >> 172.21.15.137:0/712981502 conn(0x7faf840fc7d0 msgr2=0x7faf841068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 shutdown_connections 2024-09-12T12:32:21.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:21.698+0000 7faf8881d640 1 -- 172.21.15.137:0/712981502 wait complete. 2024-09-12T12:32:21.702 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:21 smithi171 ceph-mon[35098]: pgmap v609: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 5.5 MiB/s wr, 229 op/s 2024-09-12T12:32:21.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:21 smithi189 ceph-mon[35475]: pgmap v609: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 5.5 MiB/s wr, 229 op/s 2024-09-12T12:32:22.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 23.64 MB/sec execute 234 sec latency 881.116 ms 2024-09-12T12:32:22.723 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a68da80', quiesce took 4.6 seconds. Will release after: 17.200000000000003 2024-09-12T12:32:23.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 23.54 MB/sec execute 235 sec latency 1881.222 ms 2024-09-12T12:32:23.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:23 smithi171 ceph-mon[35098]: pgmap v610: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.3 MiB/s wr, 344 op/s 2024-09-12T12:32:23.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:23 smithi137 ceph-mon[30950]: pgmap v610: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.3 MiB/s wr, 344 op/s 2024-09-12T12:32:23.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:23 smithi189 ceph-mon[35475]: pgmap v610: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 7.3 MiB/s wr, 344 op/s 2024-09-12T12:32:24.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 23.44 MB/sec execute 236 sec latency 2881.342 ms 2024-09-12T12:32:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 -- 172.21.15.137:0/1767690701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4106a00 msgr2=0x7f0eb4106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 --2- 172.21.15.137:0/1767690701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4106a00 0x7f0eb4106de0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009930 tx=0x7f0eac02a420 comp rx=0 tx=0).stop 2024-09-12T12:32:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 -- 172.21.15.137:0/1767690701 shutdown_connections 2024-09-12T12:32:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 --2- 172.21.15.137:0/1767690701 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0eb4101390 0x7f0eb410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 --2- 172.21.15.137:0/1767690701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0eb41009f0 0x7f0eb4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.360 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 --2- 172.21.15.137:0/1767690701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4106a00 0x7f0eb4106de0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 -- 172.21.15.137:0/1767690701 >> 172.21.15.137:0/1767690701 conn(0x7f0eb40fc6b0 msgr2=0x7f0eb40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 -- 172.21.15.137:0/1767690701 shutdown_connections 2024-09-12T12:32:24.361 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.359+0000 7f0eb882e640 1 -- 172.21.15.137:0/1767690701 wait complete. 2024-09-12T12:32:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 Processor -- start 2024-09-12T12:32:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 -- start start 2024-09-12T12:32:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0eb41009f0 0x7f0eb410e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4101390 0x7f0eb410e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0eb4106a00 0x7f0eb410edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.363 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4111f80 con 0x7f0eb4101390 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4111e00 con 0x7f0eb4106a00 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb882e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4112100 con 0x7f0eb41009f0 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4101390 0x7f0eb410e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4101390 0x7f0eb410e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51382/0 (socket says 172.21.15.137:51382) 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.360+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/2344611123 learned_addr learned my addr 172.21.15.137:0/2344611123 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb259c640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0eb41009f0 0x7f0eb410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0eb4106a00 0x7f0eb410edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/2344611123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0eb41009f0 msgr2=0x7f0eb410e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0eb41009f0 0x7f0eb410e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/2344611123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0eb4106a00 msgr2=0x7f0eb410edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0eb4106a00 0x7f0eb410edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/2344611123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb41a68f0 con 0x7f0eb4101390 2024-09-12T12:32:24.364 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb259c640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0eb41009f0 0x7f0eb410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0eb4106a00 0x7f0eb410edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4101390 0x7f0eb410e890 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00ea80 tx=0x7f0e9c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eab7fe640 1 -- 172.21.15.137:0/2344611123 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00ce40 con 0x7f0eb4101390 2024-09-12T12:32:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eab7fe640 1 -- 172.21.15.137:0/2344611123 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c005020 con 0x7f0eb4101390 2024-09-12T12:32:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb41a6be0 con 0x7f0eb4101390 2024-09-12T12:32:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eab7fe640 1 -- 172.21.15.137:0/2344611123 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c011760 con 0x7f0eb4101390 2024-09-12T12:32:24.365 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.361+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb41a70e0 con 0x7f0eb4101390 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.363+0000 7f0eab7fe640 1 -- 172.21.15.137:0/2344611123 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0e9c0051d0 con 0x7f0eb4101390 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 Processor -- start 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 -- start start 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d569af0 0x55fe0d569ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d574770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d5770a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe0d4ce2f0 con 0x55fe0d574cb0 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fe0d476270 con 0x55fe0d56a410 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe0d4ca2b0 con 0x55fe0d569af0 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0eb1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d5770a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.372 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0eab7fe640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ea00a2da0 0x7f0ea00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0eb2d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d569af0 0x55fe0d569ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0eb1d9b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d5770a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51396/0 (socket says 172.21.15.137:51396) 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0eb259c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d574770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.364+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/3253603620 learned_addr learned my addr 172.21.15.137:0/3253603620 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ea00a2da0 0x7f0ea00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eab7fe640 1 -- 172.21.15.137:0/2344611123 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f0e9c0cfb00 con 0x7f0eb4101390 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/3253603620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d569af0 msgr2=0x55fe0d569ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d569af0 0x55fe0d569ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/3253603620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 msgr2=0x55fe0d574770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d574770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb1d9b640 1 -- 172.21.15.137:0/3253603620 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0d475340 con 0x55fe0d574cb0 2024-09-12T12:32:24.373 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb259c640 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d574770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d5770a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c01c000 tx=0x7f0e9c09bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dee907e5b18b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0ea8ff9640 1 -- 172.21.15.137:0/3253603620 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c09c040 con 0x55fe0d574cb0 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ea00a2da0 0x7f0ea00a5260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4005fd0 tx=0x7f0ea4004730 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0ea8ff9640 1 -- 172.21.15.137:0/3253603620 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0a03f0 con 0x55fe0d574cb0 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.365+0000 7f0ea8ff9640 1 -- 172.21.15.137:0/3253603620 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0986b0 con 0x55fe0d574cb0 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/3253603620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 msgr2=0x55fe0d5770a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d5770a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c01c000 tx=0x7f0e9c09bc40 comp rx=0 tx=0).stop 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/3253603620 shutdown_connections 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d5770a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d574770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/3253603620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d569af0 0x55fe0d569ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/3253603620 >> 172.21.15.137:0/3253603620 conn(0x55fe0d566060 msgr2=0x55fe0d566440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/3253603620 shutdown_connections 2024-09-12T12:32:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/3253603620 wait complete. 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.366+0000 7f0ebb5cb740 1 -- start start 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 1 Processor -- start 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d571bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d572110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d572650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55fe0d577820 con 0x55fe0d574cb0 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fe0d457290 con 0x55fe0d569af0 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe0d5779a0 con 0x55fe0d56a410 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0ebb5cb740 10 client.0 _ll_register_callbacks cb 0x55fe0d488140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d572650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.375 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d572650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51400/0 (socket says 172.21.15.137:51400) 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/635202470 learned_addr learned my addr 172.21.15.137:0/635202470 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb259c640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d571bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d572110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/635202470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 msgr2=0x55fe0d572110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d572110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 msgr2=0x55fe0d571bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d571bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/635202470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0d1e1210 con 0x55fe0d574cb0 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d572110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb259c640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d571bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d572650 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4011800 tx=0x7f0ea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15317fc874b2e0d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.367+0000 7f0e99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0e99ffb640 1 -- 172.21.15.137:0/635202470 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4019070 con 0x55fe0d574cb0 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0e99ffb640 1 -- 172.21.15.137:0/635202470 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f0ea4010b50 con 0x55fe0d574cb0 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c0a03f0 con 0x55fe0d574cb0 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0e99ffb640 1 -- 172.21.15.137:0/635202470 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0ea4011e40 con 0x55fe0d574cb0 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 10 client.17763 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55fe0d2ad620 con 0x55fe0d574cb0 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0e99ffb640 1 -- 172.21.15.137:0/635202470 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0a03f0 con 0x55fe0d574cb0 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0e99ffb640 1 -- 172.21.15.137:0/635202470 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f0ea401f420 con 0x55fe0d574cb0 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 10 client.17763 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 10 client.17763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 20 client.17763 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d584c90 0x55fe0d587080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 4 client.17763 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:32:24.377 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.368+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe0d4dba20 con 0x55fe0d584c90 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.369+0000 7f0eb259c640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d584c90 0x55fe0d587080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.369+0000 7f0eb259c640 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d584c90 0x55fe0d587080 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4b6b63d1c3b121 server_cookie=121ecad6feb2fb63 in_seq=0 out_seq=0 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.369+0000 7f0e99ffb640 10 client.17763 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0e99ffb640 1 -- 172.21.15.137:0/635202470 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe0d4dba20 con 0x55fe0d584c90 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0e99ffb640 10 client.17763 handle_command_reply: tid=0 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 client.17763 shutdown 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 msgr2=0x55fe0d572650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d572650 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4011800 tx=0x7f0ea40046c0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 shutdown_connections 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d584c90 0x55fe0d587080 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d572650 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0e99ffb640 0 client.17763 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:24.378 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d572110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/635202470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d571bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 >> 172.21.15.137:0/635202470 conn(0x55fe0d566060 msgr2=0x55fe0d56bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 shutdown_connections 2024-09-12T12:32:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/635202470 wait complete. 2024-09-12T12:32:24.379 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.370+0000 7f0ebb5cb740 20 client.17763 trim_cache size 0 max 16384 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 Processor -- start 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 -- start start 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d572480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d58c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d58ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41b2360 con 0x55fe0d574cb0 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41afc30 con 0x55fe0d569af0 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41adaa0 con 0x55fe0d56a410 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d58ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d58ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51414/0 (socket says 172.21.15.137:51414) 2024-09-12T12:32:24.390 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/1581868598 learned_addr learned my addr 172.21.15.137:0/1581868598 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb259c640 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d572480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/1581868598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 msgr2=0x55fe0d58c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d58c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/1581868598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 msgr2=0x55fe0d572480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d572480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 -- 172.21.15.137:0/1581868598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0d56a240 con 0x55fe0d574cb0 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb259c640 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d572480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d58ceb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0ea401f890 tx=0x7f0ea401fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d6f585c5c7443c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.385+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1581868598 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4034070 con 0x55fe0d574cb0 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1581868598 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea401f620 con 0x55fe0d574cb0 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0e9bfff640 1 -- 172.21.15.137:0/1581868598 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4019070 con 0x55fe0d574cb0 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/1581868598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 msgr2=0x55fe0d58ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.391 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d58ceb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0ea401f890 tx=0x7f0ea401fd40 comp rx=0 tx=0).stop 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/1581868598 shutdown_connections 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d574cb0 0x55fe0d58ceb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d56a410 0x55fe0d58c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/1581868598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d569af0 0x55fe0d572480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/1581868598 >> 172.21.15.137:0/1581868598 conn(0x55fe0d566060 msgr2=0x55fe0d573c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/1581868598 shutdown_connections 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/1581868598 wait complete. 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.386+0000 7f0ebb5cb740 1 -- start start 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 Processor -- start 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d569af0 0x55fe0d5788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d578de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d574cb0 0x55fe0d579320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.392 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ae0f0 con 0x55fe0d569af0 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41af560 con 0x55fe0d56a410 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb41ae7c0 con 0x55fe0d574cb0 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 10 client.0 _ll_register_callbacks cb 0x7f0ea4013f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d569af0 0x55fe0d5788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d569af0 0x55fe0d5788a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51430/0 (socket says 172.21.15.137:51430) 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 -- 172.21.15.137:0/704193529 learned_addr learned my addr 172.21.15.137:0/704193529 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d578de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d574cb0 0x55fe0d579320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 -- 172.21.15.137:0/704193529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d574cb0 msgr2=0x55fe0d579320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d574cb0 0x55fe0d579320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 -- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 msgr2=0x55fe0d578de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d578de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.393 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 -- 172.21.15.137:0/704193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0d5ee940 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb2d9d640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d574cb0 0x55fe0d579320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d578de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0eb259c640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d569af0 0x55fe0d5788a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0eac006010 tx=0x7f0eac0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cc189185934353 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0e99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0e99ffb640 1 -- 172.21.15.137:0/704193529 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac048020 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0e99ffb640 1 -- 172.21.15.137:0/704193529 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f0eac0046c0 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.387+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea401f620 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0e99ffb640 1 -- 172.21.15.137:0/704193529 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f0eac053050 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 10 client.17769 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55fe0d5efb20 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0e99ffb640 1 -- 172.21.15.137:0/704193529 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea401f620 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0e99ffb640 1 -- 172.21.15.137:0/704193529 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f0eac031ae0 con 0x55fe0d569af0 2024-09-12T12:32:24.394 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 10 client.17769 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 20 client.17769 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d5f0240 0x55fe0d5f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 4 client.17769 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe0d4d5d70 con 0x55fe0d5f0240 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.388+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d5f0240 0x55fe0d5f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.389+0000 7f0e99ffb640 10 client.17769 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:24.395 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.389+0000 7f0eb1d9b640 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d5f0240 0x55fe0d5f2660 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb642338492e1e9 server_cookie=6a20534837f79fea in_seq=0 out_seq=0 2024-09-12T12:32:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0e99ffb640 1 -- 172.21.15.137:0/704193529 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe0d4d5d70 con 0x55fe0d5f0240 2024-09-12T12:32:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0e99ffb640 10 client.17769 handle_command_reply: tid=0 2024-09-12T12:32:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 client.17769 shutdown 2024-09-12T12:32:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d569af0 msgr2=0x55fe0d5788a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d569af0 0x55fe0d5788a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0eac006010 tx=0x7f0eac0050d0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.577 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 shutdown_connections 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe0d574cb0 0x55fe0d579320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fe0d56a410 0x55fe0d578de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55fe0d5f0240 0x55fe0d5f2660 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 --2- 172.21.15.137:0/704193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fe0d569af0 0x55fe0d5788a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0e99ffb640 0 client.17769 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 >> 172.21.15.137:0/704193529 conn(0x55fe0d566060 msgr2=0x55fe0d573c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 shutdown_connections 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 1 -- 172.21.15.137:0/704193529 wait complete. 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.576+0000 7f0ebb5cb740 20 client.17769 trim_cache size 0 max 16384 2024-09-12T12:32:24.578 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:32:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:32:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:32:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:32:24.579 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ea00a2da0 msgr2=0x7f0ea00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ea00a2da0 0x7f0ea00a5260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4005fd0 tx=0x7f0ea4004730 comp rx=0 tx=0).stop 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4101390 msgr2=0x7f0eb410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4101390 0x7f0eb410e890 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00ea80 tx=0x7f0e9c00ef40 comp rx=0 tx=0).stop 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 shutdown_connections 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f0ea00a2da0 0x7f0ea00a5260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0eb4106a00 0x7f0eb410edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4101390 0x7f0eb410e890 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 --2- 172.21.15.137:0/2344611123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0eb41009f0 0x7f0eb410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 >> 172.21.15.137:0/2344611123 conn(0x7f0eb40fc6b0 msgr2=0x7f0eb40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 shutdown_connections 2024-09-12T12:32:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:24.577+0000 7f0eb882e640 1 -- 172.21.15.137:0/2344611123 wait complete. 2024-09-12T12:32:25.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 23.34 MB/sec execute 237 sec latency 3881.384 ms 2024-09-12T12:32:25.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:25 smithi171 ceph-mon[35098]: pgmap v611: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 316 op/s 2024-09-12T12:32:25.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:25 smithi137 ceph-mon[30950]: pgmap v611: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 316 op/s 2024-09-12T12:32:25.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:25 smithi189 ceph-mon[35475]: pgmap v611: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.7 MiB/s wr, 316 op/s 2024-09-12T12:32:26.232 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 23.24 MB/sec execute 238 sec latency 4881.434 ms 2024-09-12T12:32:27.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 23.14 MB/sec execute 239 sec latency 5881.519 ms 2024-09-12T12:32:27.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:27 smithi171 ceph-mon[35098]: pgmap v612: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.7 MiB/s wr, 507 op/s 2024-09-12T12:32:27.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:27 smithi137 ceph-mon[30950]: pgmap v612: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.7 MiB/s wr, 507 op/s 2024-09-12T12:32:27.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:27 smithi189 ceph-mon[35475]: pgmap v612: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.7 MiB/s wr, 507 op/s 2024-09-12T12:32:28.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 23.05 MB/sec execute 240 sec latency 6881.572 ms 2024-09-12T12:32:29.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.95 MB/sec execute 241 sec latency 7881.685 ms 2024-09-12T12:32:29.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:29 smithi171 ceph-mon[35098]: pgmap v613: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 403 op/s 2024-09-12T12:32:29.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:29 smithi137 ceph-mon[30950]: pgmap v613: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 403 op/s 2024-09-12T12:32:29.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:29 smithi189 ceph-mon[35475]: pgmap v613: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 403 op/s 2024-09-12T12:32:30.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.85 MB/sec execute 242 sec latency 8881.807 ms 2024-09-12T12:32:31.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.76 MB/sec execute 243 sec latency 9881.857 ms 2024-09-12T12:32:31.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:31 smithi171 ceph-mon[35098]: pgmap v614: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 407 op/s 2024-09-12T12:32:31.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:31 smithi137 ceph-mon[30950]: pgmap v614: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 407 op/s 2024-09-12T12:32:31.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:31 smithi189 ceph-mon[35475]: pgmap v614: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.2 MiB/s wr, 407 op/s 2024-09-12T12:32:32.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.67 MB/sec execute 244 sec latency 10881.946 ms 2024-09-12T12:32:33.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.57 MB/sec execute 245 sec latency 11882.084 ms 2024-09-12T12:32:33.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:33 smithi171 ceph-mon[35098]: pgmap v615: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.7 MiB/s wr, 459 op/s 2024-09-12T12:32:33.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:33 smithi137 ceph-mon[30950]: pgmap v615: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.7 MiB/s wr, 459 op/s 2024-09-12T12:32:33.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:33 smithi189 ceph-mon[35475]: pgmap v615: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.7 MiB/s wr, 459 op/s 2024-09-12T12:32:34.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.48 MB/sec execute 246 sec latency 12882.223 ms 2024-09-12T12:32:35.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.39 MB/sec execute 247 sec latency 13882.320 ms 2024-09-12T12:32:35.642 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:35 smithi189 ceph-mon[35475]: pgmap v616: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-09-12T12:32:35.642 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:35 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:32:35.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:35 smithi171 ceph-mon[35098]: pgmap v616: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-09-12T12:32:35.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:35 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:32:35.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:35 smithi137 ceph-mon[30950]: pgmap v616: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 289 op/s 2024-09-12T12:32:35.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:35 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:32:36.233 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.30 MB/sec execute 248 sec latency 14882.370 ms 2024-09-12T12:32:36.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:36 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:32:36.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:36 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:32:36.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:36 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:32:36.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:36 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:32:36.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:36 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:32:36.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:36 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:32:36.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:36 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:32:36.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:36 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:32:36.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:36 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:32:37.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.21 MB/sec execute 249 sec latency 15882.454 ms 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.674+0000 7fc911f0a640 1 -- 172.21.15.137:0/184522972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c106b70 msgr2=0x7fc90c106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.674+0000 7fc911f0a640 1 --2- 172.21.15.137:0/184522972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c106b70 0x7fc90c106f50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009900 tx=0x7fc8fc02a410 comp rx=0 tx=0).stop 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 -- 172.21.15.137:0/184522972 shutdown_connections 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 --2- 172.21.15.137:0/184522972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc90c1020c0 0x7fc90c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 --2- 172.21.15.137:0/184522972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c101740 0x7fc90c101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 --2- 172.21.15.137:0/184522972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c106b70 0x7fc90c106f50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 -- 172.21.15.137:0/184522972 >> 172.21.15.137:0/184522972 conn(0x7fc90c075ef0 msgr2=0x7fc90c076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 -- 172.21.15.137:0/184522972 shutdown_connections 2024-09-12T12:32:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 -- 172.21.15.137:0/184522972 wait complete. 2024-09-12T12:32:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 Processor -- start 2024-09-12T12:32:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.675+0000 7fc911f0a640 1 -- start start 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc911f0a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc90c101740 0x7fc90c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc911f0a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1020c0 0x7fc90c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc911f0a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c106b70 0x7fc90c1a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc911f0a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c110ed0 con 0x7fc90c1020c0 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc911f0a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c110d50 con 0x7fc90c106b70 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc911f0a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c111050 con 0x7fc90c101740 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1020c0 0x7fc90c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1020c0 0x7fc90c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47764/0 (socket says 172.21.15.137:47764) 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 -- 172.21.15.137:0/3940459901 learned_addr learned my addr 172.21.15.137:0/3940459901 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90bfff640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c106b70 0x7fc90c1a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc90c101740 0x7fc90c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 -- 172.21.15.137:0/3940459901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc90c101740 msgr2=0x7fc90c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc90c101740 0x7fc90c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 -- 172.21.15.137:0/3940459901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c106b70 msgr2=0x7fc90c1a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c106b70 0x7fc90c1a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 -- 172.21.15.137:0/3940459901 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c1a47a0 con 0x7fc90c1020c0 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90bfff640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c106b70 0x7fc90c1a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc90affd640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1020c0 0x7fc90c199720 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d950 tx=0x7fc8f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc908ff9640 1 -- 172.21.15.137:0/3940459901 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4017070 con 0x7fc90c1020c0 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc908ff9640 1 -- 172.21.15.137:0/3940459901 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4009c30 con 0x7fc90c1020c0 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c1a4970 con 0x7fc90c1020c0 2024-09-12T12:32:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.676+0000 7fc908ff9640 1 -- 172.21.15.137:0/3940459901 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400f6b0 con 0x7fc90c1020c0 2024-09-12T12:32:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.677+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c1a4e90 con 0x7fc90c1020c0 2024-09-12T12:32:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.679+0000 7fc908ff9640 1 -- 172.21.15.137:0/3940459901 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc8f40109d0 con 0x7fc90c1020c0 2024-09-12T12:32:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 Processor -- start 2024-09-12T12:32:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 -- start start 2024-09-12T12:32:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564eabdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ebab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564ebb0c0 0x55b564ebd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b564e102f0 con 0x55b564ebb0c0 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b564db8270 con 0x55b564eab9d0 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc914ca0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b564e0c2b0 con 0x55b564eac2f0 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564ebb0c0 0x55b564ebd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564ebb0c0 0x55b564ebd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47778/0 (socket says 172.21.15.137:47778) 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc90affd640 1 -- 172.21.15.137:0/334498315 learned_addr learned my addr 172.21.15.137:0/334498315 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc90bfff640 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564eabdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ebab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.680+0000 7fc908ff9640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8f80a2da0 0x7fc8f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90affd640 1 -- 172.21.15.137:0/334498315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 msgr2=0x55b564ebab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90affd640 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ebab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90affd640 1 -- 172.21.15.137:0/334498315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 msgr2=0x55b564eabdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90affd640 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564eabdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90affd640 1 -- 172.21.15.137:0/334498315 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b564db7340 con 0x55b564ebb0c0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90bfff640 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564eabdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90bfff640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8f80a2da0 0x7fc8f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc908ff9640 1 -- 172.21.15.137:0/3940459901 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7fc8f40cf130 con 0x7fc90c1020c0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90affd640 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564ebb0c0 0x55b564ebd4b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40cc810 tx=0x7fc8f401e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58242727fa7e2a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc901ffb640 1 -- 172.21.15.137:0/334498315 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f409b1b0 con 0x55b564ebb0c0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc90bfff640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8f80a2da0 0x7fc8f80a5260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc904005fd0 tx=0x7fc904004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc901ffb640 1 -- 172.21.15.137:0/334498315 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4009de0 con 0x55b564ebb0c0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.681+0000 7fc901ffb640 1 -- 172.21.15.137:0/334498315 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40d2990 con 0x55b564ebb0c0 2024-09-12T12:32:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 -- 172.21.15.137:0/334498315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564ebb0c0 msgr2=0x55b564ebd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564ebb0c0 0x55b564ebd4b0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40cc810 tx=0x7fc8f401e770 comp rx=0 tx=0).stop 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 -- 172.21.15.137:0/334498315 shutdown_connections 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564ebb0c0 0x55b564ebd4b0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ebab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 --2- 172.21.15.137:0/334498315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564eabdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 -- 172.21.15.137:0/334498315 >> 172.21.15.137:0/334498315 conn(0x55b564ea8060 msgr2=0x55b564ea8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 -- 172.21.15.137:0/334498315 shutdown_connections 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 -- 172.21.15.137:0/334498315 wait complete. 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.682+0000 7fc914ca0740 1 -- start start 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 1 Processor -- start 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eab9d0 0x55b564eb7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564eb8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564eb8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55b564ebdc90 con 0x55b564eac2f0 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b564d99290 con 0x55b564ebb0c0 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b564ebde10 con 0x55b564eab9d0 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc914ca0740 10 client.0 _ll_register_callbacks cb 0x55b564dca140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564eb8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564eb8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47782/0 (socket says 172.21.15.137:47782) 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 -- 172.21.15.137:0/3271084667 learned_addr learned my addr 172.21.15.137:0/3271084667 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90bfff640 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564eb8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eab9d0 0x55b564eb7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 -- 172.21.15.137:0/3271084667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eab9d0 msgr2=0x55b564eb7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eab9d0 0x55b564eb7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 -- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 msgr2=0x55b564eb8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564eb8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 -- 172.21.15.137:0/3271084667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b564b23210 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc90affd640 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564eb8500 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40058c0 tx=0x7fc8f409b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe0fdb2f0dfc33e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.683+0000 7fc8eaffd640 1 -- 172.21.15.137:0/3271084667 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f409afd0 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc8eaffd640 1 -- 172.21.15.137:0/3271084667 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7fc8f4099970 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f4009de0 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc8eaffd640 1 -- 172.21.15.137:0/3271084667 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc8f409b1b0 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 10 client.17781 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b564bef620 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc8eaffd640 1 -- 172.21.15.137:0/3271084667 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4009de0 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc8eaffd640 1 -- 172.21.15.137:0/3271084667 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fc8f40ccb20 con 0x55b564eac2f0 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 10 client.17781 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:37.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 10 client.17781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 20 client.17781 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564ecaec0 0x55b564ecd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 4 client.17781 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b564e1da20 con 0x55b564ecaec0 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.684+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564ecaec0 0x55b564ecd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.685+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564ecaec0 0x55b564ecd2b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff29854c015a51f server_cookie=77ceb021e5e212f7 in_seq=0 out_seq=0 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.685+0000 7fc8eaffd640 10 client.17781 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.685+0000 7fc8eaffd640 1 -- 172.21.15.137:0/3271084667 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b564e1da20 con 0x55b564ecaec0 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.685+0000 7fc8eaffd640 10 client.17781 handle_command_reply: tid=0 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 client.17781 shutdown 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 msgr2=0x55b564eb8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564eb8500 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40058c0 tx=0x7fc8f409b4c0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.694 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 shutdown_connections 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564eb8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564ecaec0 0x55b564ecd2b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564eb8500 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3271084667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eab9d0 0x55b564eb7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc8eaffd640 0 client.17781 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 >> 172.21.15.137:0/3271084667 conn(0x55b564ea8060 msgr2=0x55b564eaddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 shutdown_connections 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 1 -- 172.21.15.137:0/3271084667 wait complete. 2024-09-12T12:32:37.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.686+0000 7fc914ca0740 20 client.17781 trim_cache size 0 max 16384 2024-09-12T12:32:37.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:37 smithi171 ceph-mon[35098]: pgmap v617: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 310 op/s 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.700+0000 7fc914ca0740 1 Processor -- start 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.700+0000 7fc914ca0740 1 -- start start 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564ea70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564ed02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564ebb0c0 0x55b564ed0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc914ca0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1adec0 con 0x55b564eac2f0 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc914ca0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1ab790 con 0x55b564eab9d0 2024-09-12T12:32:37.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc914ca0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1a9620 con 0x55b564ebb0c0 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564ed02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564ed02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47794/0 (socket says 172.21.15.137:47794) 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 -- 172.21.15.137:0/3631234650 learned_addr learned my addr 172.21.15.137:0/3631234650 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90bfff640 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564ebb0c0 0x55b564ed0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564ea70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 -- 172.21.15.137:0/3631234650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564ebb0c0 msgr2=0x55b564ed0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564ebb0c0 0x55b564ed0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 -- 172.21.15.137:0/3631234650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 msgr2=0x55b564ea70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564ea70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 -- 172.21.15.137:0/3631234650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b564ea7e90 con 0x55b564eac2f0 2024-09-12T12:32:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc90affd640 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564ed02d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40a3db0 tx=0x7fc8f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774f09b1eed0a614 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc9017fa640 1 -- 172.21.15.137:0/3631234650 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4009de0 con 0x55b564eac2f0 2024-09-12T12:32:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc9017fa640 1 -- 172.21.15.137:0/3631234650 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f409bd70 con 0x55b564eac2f0 2024-09-12T12:32:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.701+0000 7fc9017fa640 1 -- 172.21.15.137:0/3631234650 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4097ce0 con 0x55b564eac2f0 2024-09-12T12:32:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 -- 172.21.15.137:0/3631234650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 msgr2=0x55b564ed02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564ed02d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40a3db0 tx=0x7fc8f4000bc0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 -- 172.21.15.137:0/3631234650 shutdown_connections 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564ebb0c0 0x55b564ed0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eac2f0 0x55b564ed02d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 --2- 172.21.15.137:0/3631234650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564eab9d0 0x55b564ea70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 -- 172.21.15.137:0/3631234650 >> 172.21.15.137:0/3631234650 conn(0x55b564ea8060 msgr2=0x55b564ea69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 -- 172.21.15.137:0/3631234650 shutdown_connections 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 -- 172.21.15.137:0/3631234650 wait complete. 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.702+0000 7fc914ca0740 1 -- start start 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 1 Processor -- start 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eab9d0 0x55b564ece340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ece880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564ecedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1a9c70 con 0x55b564eab9d0 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1ab0c0 con 0x55b564ebb0c0 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c1aa320 con 0x55b564eac2f0 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc914ca0740 10 client.0 _ll_register_callbacks cb 0x7fc904009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eab9d0 0x55b564ece340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eab9d0 0x55b564ece340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47796/0 (socket says 172.21.15.137:47796) 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 -- 172.21.15.137:0/740750617 learned_addr learned my addr 172.21.15.137:0/740750617 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90bfff640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564ecedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90affd640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ece880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 -- 172.21.15.137:0/740750617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 msgr2=0x55b564ece880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ece880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 -- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 msgr2=0x55b564ecedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564ecedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 -- 172.21.15.137:0/740750617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b564eb2d50 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90bfff640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564ecedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc90b7fe640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eab9d0 0x55b564ece340 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0096c0 tx=0x7fc8fc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd6bb77b803051c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.703+0000 7fc8eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc8eaffd640 1 -- 172.21.15.137:0/740750617 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc035040 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc8eaffd640 1 -- 172.21.15.137:0/740750617 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7fc8fc004250 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f409bd70 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc8eaffd640 1 -- 172.21.15.137:0/740750617 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fc8fc004450 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 10 client.17787 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b564f35bc0 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc8eaffd640 1 -- 172.21.15.137:0/740750617 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f409bd70 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc8eaffd640 1 -- 172.21.15.137:0/740750617 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7fc8fc04e070 con 0x55b564eab9d0 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 10 client.17787 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 10 client.17787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 20 client.17787 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564f36310 0x55b564f38730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 4 client.17787 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:32:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b564e17d70 con 0x55b564f36310 2024-09-12T12:32:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.704+0000 7fc90affd640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564f36310 0x55b564f38730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.705+0000 7fc8eaffd640 10 client.17787 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.705+0000 7fc90affd640 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564f36310 0x55b564f38730 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b27b5da4bc50b6 server_cookie=bfb06079aa5d2315 in_seq=0 out_seq=0 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.774+0000 7fc8eaffd640 1 -- 172.21.15.137:0/740750617 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b564e17d70 con 0x55b564f36310 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.774+0000 7fc8eaffd640 10 client.17787 handle_command_reply: tid=0 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.774+0000 7fc914ca0740 1 client.17787 shutdown 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eab9d0 msgr2=0x55b564ece340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eab9d0 0x55b564ece340 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0096c0 tx=0x7fc8fc005dc0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 shutdown_connections 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b564ebb0c0 0x55b564ecedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b564eac2f0 0x55b564ece880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55b564f36310 0x55b564f38730 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 --2- 172.21.15.137:0/740750617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b564eab9d0 0x55b564ece340 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc8eaffd640 0 client.17787 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 >> 172.21.15.137:0/740750617 conn(0x55b564ea8060 msgr2=0x55b564ec68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 shutdown_connections 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 1 -- 172.21.15.137:0/740750617 wait complete. 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc914ca0740 20 client.17787 trim_cache size 0 max 16384 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:32:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:32:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8f80a2da0 msgr2=0x7fc8f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8f80a2da0 0x7fc8f80a5260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc904005fd0 tx=0x7fc904004a40 comp rx=0 tx=0).stop 2024-09-12T12:32:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1020c0 msgr2=0x7fc90c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1020c0 0x7fc90c199720 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d950 tx=0x7fc8f400de10 comp rx=0 tx=0).stop 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 shutdown_connections 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fc8f80a2da0 0x7fc8f80a5260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc90c106b70 0x7fc90c1a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc90c1020c0 0x7fc90c199720 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 --2- 172.21.15.137:0/3940459901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc90c101740 0x7fc90c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 >> 172.21.15.137:0/3940459901 conn(0x7fc90c075ef0 msgr2=0x7fc90c0ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.775+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 shutdown_connections 2024-09-12T12:32:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:37.776+0000 7fc911f0a640 1 -- 172.21.15.137:0/3940459901 wait complete. 2024-09-12T12:32:37.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:37 smithi137 ceph-mon[30950]: pgmap v617: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 310 op/s 2024-09-12T12:32:37.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:37 smithi189 ceph-mon[35475]: pgmap v617: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 310 op/s 2024-09-12T12:32:38.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.12 MB/sec execute 250 sec latency 16882.554 ms 2024-09-12T12:32:39.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 22.04 MB/sec execute 251 sec latency 17882.669 ms 2024-09-12T12:32:39.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:39 smithi171 ceph-mon[35098]: pgmap v618: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.4 MiB/s wr, 90 op/s 2024-09-12T12:32:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:39 smithi137 ceph-mon[30950]: pgmap v618: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.4 MiB/s wr, 90 op/s 2024-09-12T12:32:39.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:39 smithi189 ceph-mon[35475]: pgmap v618: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.4 MiB/s wr, 90 op/s 2024-09-12T12:32:39.925 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:32:40.185 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:32:40.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 21.95 MB/sec execute 252 sec latency 18882.761 ms 2024-09-12T12:32:40.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 -- 172.21.15.137:0/113440107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14410c190 msgr2=0x7fd14410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:40.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 --2- 172.21.15.137:0/113440107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14410c190 0x7fd14410e610 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ca0 tx=0x7fd13c02a950 comp rx=0 tx=0).stop 2024-09-12T12:32:40.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 -- 172.21.15.137:0/113440107 shutdown_connections 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 --2- 172.21.15.137:0/113440107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd14410c190 0x7fd14410e610 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 --2- 172.21.15.137:0/113440107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd1441036d0 0x7fd14410bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 --2- 172.21.15.137:0/113440107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd144102db0 0x7fd144103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 -- 172.21.15.137:0/113440107 >> 172.21.15.137:0/113440107 conn(0x7fd1440fc7d0 msgr2=0x7fd1440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 -- 172.21.15.137:0/113440107 shutdown_connections 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.673+0000 7fd14a55a640 1 -- 172.21.15.137:0/113440107 wait complete. 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 Processor -- start 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 -- start start 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144102db0 0x7fd144100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1441036d0 0x7fd144100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:40.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd14410c190 0x7fd144101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111fb0 con 0x7fd144102db0 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd144111e30 con 0x7fd1441036d0 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd14a55a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd144112130 con 0x7fd14410c190 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd143fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144102db0 0x7fd144100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd143fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144102db0 0x7fd144100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46304/0 (socket says 172.21.15.137:46304) 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd143fff640 1 -- 172.21.15.137:0/1619835530 learned_addr learned my addr 172.21.15.137:0/1619835530 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd148af7640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd14410c190 0x7fd144101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.674+0000 7fd13bfff640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1441036d0 0x7fd144100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd143fff640 1 -- 172.21.15.137:0/1619835530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd14410c190 msgr2=0x7fd144101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd143fff640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd14410c190 0x7fd144101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd143fff640 1 -- 172.21.15.137:0/1619835530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1441036d0 msgr2=0x7fd144100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd143fff640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1441036d0 0x7fd144100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd143fff640 1 -- 172.21.15.137:0/1619835530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1441a6900 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd143fff640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144102db0 0x7fd144100a10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd13400b4a0 tx=0x7fd13400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd141ffb640 1 -- 172.21.15.137:0/1619835530 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134014070 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd141ffb640 1 -- 172.21.15.137:0/1619835530 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd134010af0 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1441a6bf0 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd141ffb640 1 -- 172.21.15.137:0/1619835530 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134017d10 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.675+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1441a7130 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.676+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd144103190 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.678+0000 7fd141ffb640 1 -- 172.21.15.137:0/1619835530 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd134017070 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.678+0000 7fd141ffb640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1300a2cf0 0x7fd1300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.679+0000 7fd141ffb640 1 -- 172.21.15.137:0/1619835530 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7fd1340ceb30 con 0x7fd144102db0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.679+0000 7fd13bfff640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1300a2cf0 0x7fd1300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:40.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.679+0000 7fd13bfff640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1300a2cf0 0x7fd1300a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005fd0 tx=0x7fd12c005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:40.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.682+0000 7fd141ffb640 1 -- 172.21.15.137:0/1619835530 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd13409b880 con 0x7fd144102db0 2024-09-12T12:32:40.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.895+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd144101f40 con 0x7fd144102db0 2024-09-12T12:32:40.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.896+0000 7fd141ffb640 1 -- 172.21.15.137:0/1619835530 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7fd13409b180 con 0x7fd144102db0 2024-09-12T12:32:40.897 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:32:40.897 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":54,"btime":"2024-09-12T12:31:32:726301+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:31:32.726248+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26861":{"gid":26861,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/4049352466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4049352466},{"type":"v1","addr":"172.21.15.189:6835","nonce":4049352466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24454,24475,24476]},"id":1}]} 2024-09-12T12:32:40.897 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 54 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1300a2cf0 msgr2=0x7fd1300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1300a2cf0 0x7fd1300a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005fd0 tx=0x7fd12c005850 comp rx=0 tx=0).stop 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144102db0 msgr2=0x7fd144100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144102db0 0x7fd144100a10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fd13400b4a0 tx=0x7fd13400b960 comp rx=0 tx=0).stop 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 shutdown_connections 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd1300a2cf0 0x7fd1300a51b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd14410c190 0x7fd144101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1441036d0 0x7fd144100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 --2- 172.21.15.137:0/1619835530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144102db0 0x7fd144100a10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 >> 172.21.15.137:0/1619835530 conn(0x7fd1440fc7d0 msgr2=0x7fd1440fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 shutdown_connections 2024-09-12T12:32:40.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:40.898+0000 7fd14a55a640 1 -- 172.21.15.137:0/1619835530 wait complete. 2024-09-12T12:32:41.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 21.86 MB/sec execute 253 sec latency 19882.831 ms 2024-09-12T12:32:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:41 smithi171 ceph-mon[35098]: pgmap v619: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.4 MiB/s wr, 92 op/s 2024-09-12T12:32:41.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:41 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1619835530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:41 smithi137 ceph-mon[30950]: pgmap v619: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.4 MiB/s wr, 92 op/s 2024-09-12T12:32:41.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:41 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1619835530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:41.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:41 smithi189 ceph-mon[35475]: pgmap v619: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 2.4 MiB/s wr, 92 op/s 2024-09-12T12:32:41.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:41 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1619835530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:32:41.853 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --set-id 6a68da80 --release --await' 2024-09-12T12:32:41.853 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --set-id 6a68da80 --release --await 2024-09-12T12:32:42.101 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:32:42.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134182 21.77 MB/sec execute 254 sec latency 20882.891 ms 2024-09-12T12:32:42.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 -- 172.21.15.137:0/723458491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0102570 msgr2=0x7f5df01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/723458491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0102570 0x7f5df01029b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009980 tx=0x7f5de402c6b0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 -- 172.21.15.137:0/723458491 shutdown_connections 2024-09-12T12:32:42.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/723458491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5df0102ef0 0x7f5df010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/723458491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0102570 0x7f5df01029b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/723458491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5df0100e20 0x7f5df0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 -- 172.21.15.137:0/723458491 >> 172.21.15.137:0/723458491 conn(0x7f5df00fc7d0 msgr2=0x7f5df00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 -- 172.21.15.137:0/723458491 shutdown_connections 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 -- 172.21.15.137:0/723458491 wait complete. 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 Processor -- start 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.605+0000 7f5df6e4b640 1 -- start start 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df6e4b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0100e20 0x7f5df01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df6e4b640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5df0102570 0x7f5df0196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df6e4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5df0102ef0 0x7f5df019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df6e4b640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5df010fea0 con 0x7f5df0100e20 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df6e4b640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5df010fd20 con 0x7f5df0102570 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df6e4b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5df0110020 con 0x7f5df0102ef0 2024-09-12T12:32:42.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0100e20 0x7f5df01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0100e20 0x7f5df01963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46316/0 (socket says 172.21.15.137:46316) 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 -- 172.21.15.137:0/3910531625 learned_addr learned my addr 172.21.15.137:0/3910531625 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5deffff640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5df0102570 0x7f5df0196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df53e8640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5df0102ef0 0x7f5df019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 -- 172.21.15.137:0/3910531625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5df0102ef0 msgr2=0x7f5df019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5df0102ef0 0x7f5df019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 -- 172.21.15.137:0/3910531625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5df0102570 msgr2=0x7f5df0196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5df0102570 0x7f5df0196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 -- 172.21.15.137:0/3910531625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df019b370 con 0x7f5df0100e20 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df4be7640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0100e20 0x7f5df01963c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b4a0 tx=0x7f5de000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5dedffb640 1 -- 172.21.15.137:0/3910531625 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0014070 con 0x7f5df0100e20 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5dedffb640 1 -- 172.21.15.137:0/3910531625 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de0010af0 con 0x7f5df0100e20 2024-09-12T12:32:42.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df019b660 con 0x7f5df0100e20 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.606+0000 7f5dedffb640 1 -- 172.21.15.137:0/3910531625 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de0017d10 con 0x7f5df0100e20 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.607+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df01a2f40 con 0x7f5df0100e20 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5dedffb640 1 -- 172.21.15.137:0/3910531625 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5de0017070 con 0x7f5df0100e20 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 Processor -- start 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 -- start start 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5dedffb640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5dd80a2cf0 0x7f5dd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f4a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401f54d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d401eaf620 con 0x55d401f4a980 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d401e56e50 con 0x55d401f55240 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.609+0000 7f5df9224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d401eab5e0 con 0x55d401f4a060 2024-09-12T12:32:42.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5df53e8640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5dd80a2cf0 0x7f5dd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5dedffb640 1 -- 172.21.15.137:0/3910531625 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f5de00ce7c0 con 0x7f5df0100e20 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401f54d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401f54d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46320/0 (socket says 172.21.15.137:46320) 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 -- 172.21.15.137:0/3009228160 learned_addr learned my addr 172.21.15.137:0/3009228160 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5df4be7640 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f4a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5df53e8640 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 -- 172.21.15.137:0/3009228160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 msgr2=0x55d401f4a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f4a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 -- 172.21.15.137:0/3009228160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 msgr2=0x55d401f57630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f57630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 -- 172.21.15.137:0/3009228160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d401e47d40 con 0x55d401f4a980 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5df53e8640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5dd80a2cf0 0x7f5dd80a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5de8005fd0 tx=0x7f5de8010740 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:42.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5deffff640 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401f54d00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009980 tx=0x7f5de4004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3d471f431d057c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5dd6ffd640 1 -- 172.21.15.137:0/3009228160 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4037030 con 0x55d401f4a980 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5dd6ffd640 1 -- 172.21.15.137:0/3009228160 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4042070 con 0x55d401f4a980 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.610+0000 7f5dd6ffd640 1 -- 172.21.15.137:0/3009228160 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4033af0 con 0x55d401f4a980 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 -- 172.21.15.137:0/3009228160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 msgr2=0x55d401f54d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401f54d00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009980 tx=0x7f5de4004170 comp rx=0 tx=0).stop 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 -- 172.21.15.137:0/3009228160 shutdown_connections 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f57630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401f54d00 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 --2- 172.21.15.137:0/3009228160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f4a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 -- 172.21.15.137:0/3009228160 >> 172.21.15.137:0/3009228160 conn(0x55d401f465f0 msgr2=0x55d401f469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 -- 172.21.15.137:0/3009228160 shutdown_connections 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 -- 172.21.15.137:0/3009228160 wait complete. 2024-09-12T12:32:42.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.611+0000 7f5df9224740 1 -- start start 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 1 Processor -- start 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a060 0x55d401f52150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a980 0x55d401f52690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f52bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55d401f57e10 con 0x55d401f4a060 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d401e37f10 con 0x55d401f55240 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d401f57f90 con 0x55d401f4a980 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df9224740 10 client.0 _ll_register_callbacks cb 0x55d401e5bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a060 0x55d401f52150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a060 0x55d401f52150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46336/0 (socket says 172.21.15.137:46336) 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 -- 172.21.15.137:0/2803500914 learned_addr learned my addr 172.21.15.137:0/2803500914 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:42.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5deffff640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a980 0x55d401f52690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df53e8640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f52bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 -- 172.21.15.137:0/2803500914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a980 msgr2=0x55d401f52690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a980 0x55d401f52690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 -- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 msgr2=0x55d401f52bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f52bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df4be7640 1 -- 172.21.15.137:0/2803500914 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d401bbe580 con 0x55d401f4a060 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.612+0000 7f5df53e8640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f52bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:32:42.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df4be7640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a060 0x55d401f52150 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5de009ad40 tx=0x7f5de009a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582dcce9ed1a5ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5dcffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5dcffff640 1 -- 172.21.15.137:0/2803500914 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de001dc50 con 0x55d401f4a060 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5dcffff640 1 -- 172.21.15.137:0/2803500914 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f5de0017360 con 0x55d401f4a060 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de4042070 con 0x55d401f4a060 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5dcffff640 1 -- 172.21.15.137:0/2803500914 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5de009adb0 con 0x55d401f4a060 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 10 client.17805 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d401c88960 con 0x55d401f4a060 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5dcffff640 1 -- 172.21.15.137:0/2803500914 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de4042070 con 0x55d401f4a060 2024-09-12T12:32:42.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5dcffff640 1 -- 172.21.15.137:0/2803500914 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5de0085040 con 0x55d401f4a060 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 10 client.17805 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 10 client.17805 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 20 client.17805 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401f64d90 0x55d401f671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 4 client.17805 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.613+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d401ebcb90 con 0x55d401f64d90 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.614+0000 7f5deffff640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401f64d90 0x55d401f671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.614+0000 7f5deffff640 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401f64d90 0x55d401f671e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b8df757dc9d4e6b server_cookie=483fcc2f07ee634b in_seq=0 out_seq=0 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.614+0000 7f5dcffff640 10 client.17805 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5dcffff640 1 -- 172.21.15.137:0/2803500914 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d401ebcb90 con 0x55d401f64d90 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5dcffff640 10 client.17805 handle_command_reply: tid=0 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 client.17805 shutdown 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a060 msgr2=0x55d401f52150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a060 0x55d401f52150 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5de009ad40 tx=0x7f5de009a9e0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 shutdown_connections 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f55240 0x55d401f52bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a980 0x55d401f52690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401f64d90 0x55d401f671e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 --2- 172.21.15.137:0/2803500914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a060 0x55d401f52150 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5dcffff640 0 client.17805 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 >> 172.21.15.137:0/2803500914 conn(0x55d401f465f0 msgr2=0x55d401f45750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 shutdown_connections 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 1 -- 172.21.15.137:0/2803500914 wait complete. 2024-09-12T12:32:42.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.615+0000 7f5df9224740 20 client.17805 trim_cache size 0 max 16384 2024-09-12T12:32:42.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 Processor -- start 2024-09-12T12:32:42.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 -- start start 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f54020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a980 0x55d401f4cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f55240 0x55d401f5d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01adf30 con 0x55d401f55240 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01ab800 con 0x55d401f4a980 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df9224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01a9670 con 0x55d401f4a060 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df53e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f55240 0x55d401f5d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df53e8640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f55240 0x55d401f5d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46338/0 (socket says 172.21.15.137:46338) 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df53e8640 1 -- 172.21.15.137:0/2080346725 learned_addr learned my addr 172.21.15.137:0/2080346725 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5deffff640 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a980 0x55d401f4cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.632+0000 7f5df4be7640 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f54020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5df53e8640 1 -- 172.21.15.137:0/2080346725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 msgr2=0x55d401f54020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5df53e8640 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f54020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5df53e8640 1 -- 172.21.15.137:0/2080346725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a980 msgr2=0x55d401f4cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5df53e8640 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a980 0x55d401f4cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5df53e8640 1 -- 172.21.15.137:0/2080346725 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d401f49e70 con 0x55d401f55240 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5df53e8640 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f55240 0x55d401f5d5c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5de801f7e0 tx=0x7f5de8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edadbf869d5273a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5dd67fc640 1 -- 172.21.15.137:0/2080346725 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de802c070 con 0x55d401f55240 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5dd67fc640 1 -- 172.21.15.137:0/2080346725 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de8013b80 con 0x55d401f55240 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.633+0000 7f5dd67fc640 1 -- 172.21.15.137:0/2080346725 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8011640 con 0x55d401f55240 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 -- 172.21.15.137:0/2080346725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f55240 msgr2=0x55d401f5d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f55240 0x55d401f5d5c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5de801f7e0 tx=0x7f5de8015000 comp rx=0 tx=0).stop 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 -- 172.21.15.137:0/2080346725 shutdown_connections 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f55240 0x55d401f5d5c0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a980 0x55d401f4cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 --2- 172.21.15.137:0/2080346725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f4a060 0x55d401f54020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 -- 172.21.15.137:0/2080346725 >> 172.21.15.137:0/2080346725 conn(0x55d401f465f0 msgr2=0x55d401f45750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 -- 172.21.15.137:0/2080346725 shutdown_connections 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 -- 172.21.15.137:0/2080346725 wait complete. 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 -- start start 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.634+0000 7f5df9224740 1 Processor -- start 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a060 0x55d401fc7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401fc81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f55240 0x55d401fcbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01a9cc0 con 0x55d401f4a980 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01ab130 con 0x55d401f4a060 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5df01aa390 con 0x55d401f55240 2024-09-12T12:32:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 10 client.0 _ll_register_callbacks cb 0x7f5de009ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401fc81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401fc81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46340/0 (socket says 172.21.15.137:46340) 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 -- 172.21.15.137:0/4004958143 learned_addr learned my addr 172.21.15.137:0/4004958143 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df4be7640 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a060 0x55d401fc7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df53e8640 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f55240 0x55d401fcbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 -- 172.21.15.137:0/4004958143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f55240 msgr2=0x55d401fcbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f55240 0x55d401fcbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 -- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a060 msgr2=0x55d401fc7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a060 0x55d401fc7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 -- 172.21.15.137:0/4004958143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d401fcc510 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5deffff640 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401fc81b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5de402c6f0 tx=0x7f5de4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8938a5bfa7670d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5dcffff640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5dcffff640 1 -- 172.21.15.137:0/4004958143 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4056020 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5dcffff640 1 -- 172.21.15.137:0/4004958143 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f5de4003da0 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.635+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5de8013b80 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5dcffff640 1 -- 172.21.15.137:0/4004958143 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5de4037040 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 10 client.17811 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d401fd4ae0 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5dcffff640 1 -- 172.21.15.137:0/4004958143 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5de8013b80 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5dcffff640 1 -- 172.21.15.137:0/4004958143 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f5de4004c80 con 0x55d401f4a980 2024-09-12T12:32:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 10 client.17811 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 10 client.17811 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 20 client.17811 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401fd4ed0 0x55d401fd72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 4 client.17811 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a68da80", "release": true, "await": true}] 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a68da80", "release": true, "await": true}) -- 0x55d401eb70a0 con 0x55d401fd4ed0 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.636+0000 7f5df4be7640 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401fd4ed0 0x55d401fd72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.637+0000 7f5df4be7640 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401fd4ed0 0x55d401fd72c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f6e58f3ad785a9 server_cookie=28e387734015ad7b in_seq=0 out_seq=0 2024-09-12T12:32:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.637+0000 7f5dcffff640 10 client.17811 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:42.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5dcffff640 1 -- 172.21.15.137:0/4004958143 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d401eb70a0 con 0x55d401fd4ed0 2024-09-12T12:32:42.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5dcffff640 10 client.17811 handle_command_reply: tid=0 2024-09-12T12:32:42.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 client.17811 shutdown 2024-09-12T12:32:42.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 msgr2=0x55d401fc81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401fc81b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5de402c6f0 tx=0x7f5de4003820 comp rx=0 tx=0).stop 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 shutdown_connections 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d401f55240 0x55d401fcbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55d401fd4ed0 0x55d401fd72c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d401f4a980 0x55d401fc81b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 --2- 172.21.15.137:0/4004958143 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d401f4a060 0x55d401fc7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5dcffff640 0 client.17811 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 >> 172.21.15.137:0/4004958143 conn(0x55d401f465f0 msgr2=0x55d401f44f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 shutdown_connections 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.667+0000 7f5df9224740 1 -- 172.21.15.137:0/4004958143 wait complete. 2024-09-12T12:32:42.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df9224740 20 client.17811 trim_cache size 0 max 16384 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 54, 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 72, 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": { 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "6a68da80": { 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "version": 72, 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "age_ref": 0.0, 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:32:42.670 INFO:teuthology.orchestra.run.smithi137.stdout: "timeout": 90.0, 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: "expiration": 141.8, 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: "members": { 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: "file:/": { 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: "excluded": false, 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: "state": { 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RELEASED", 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.0 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:32:42.671 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:32:42.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5dd80a2cf0 msgr2=0x7f5dd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5dd80a2cf0 0x7f5dd80a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5de8005fd0 tx=0x7f5de8010740 comp rx=0 tx=0).stop 2024-09-12T12:32:42.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0100e20 msgr2=0x7f5df01963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:42.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0100e20 0x7f5df01963c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5de000b4a0 tx=0x7f5de000b960 comp rx=0 tx=0).stop 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 shutdown_connections 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f5dd80a2cf0 0x7f5dd80a51b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5df0102ef0 0x7f5df019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5df0102570 0x7f5df0196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 --2- 172.21.15.137:0/3910531625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5df0100e20 0x7f5df01963c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.668+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 >> 172.21.15.137:0/3910531625 conn(0x7f5df00fc7d0 msgr2=0x7f5df00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.669+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 shutdown_connections 2024-09-12T12:32:42.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:32:42.669+0000 7f5df6e4b640 1 -- 172.21.15.137:0/3910531625 wait complete. 2024-09-12T12:32:43.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 134595 21.75 MB/sec execute 255 sec latency 21319.997 ms 2024-09-12T12:32:43.465 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:43 smithi137 ceph-mon[30950]: pgmap v620: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.4 MiB/s wr, 109 op/s 2024-09-12T12:32:43.502 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a68da80', total seconds elapsed: 25.4 2024-09-12T12:32:43.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:43 smithi171 ceph-mon[35098]: pgmap v620: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.4 MiB/s wr, 109 op/s 2024-09-12T12:32:43.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:43 smithi189 ceph-mon[35475]: pgmap v620: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 2.4 MiB/s wr, 109 op/s 2024-09-12T12:32:44.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 135280 21.85 MB/sec execute 256 sec latency 72.984 ms 2024-09-12T12:32:45.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 135907 21.89 MB/sec execute 257 sec latency 85.176 ms 2024-09-12T12:32:45.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:45 smithi171 ceph-mon[35098]: pgmap v621: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 780 KiB/s wr, 44 op/s 2024-09-12T12:32:45.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:45 smithi137 ceph-mon[30950]: pgmap v621: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 780 KiB/s wr, 44 op/s 2024-09-12T12:32:45.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:45 smithi189 ceph-mon[35475]: pgmap v621: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 780 KiB/s wr, 44 op/s 2024-09-12T12:32:46.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 136460 21.94 MB/sec execute 258 sec latency 80.049 ms 2024-09-12T12:32:47.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 137098 21.97 MB/sec execute 259 sec latency 106.379 ms 2024-09-12T12:32:47.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:47 smithi171 ceph-mon[35098]: pgmap v622: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-09-12T12:32:47.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:47 smithi137 ceph-mon[30950]: pgmap v622: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-09-12T12:32:47.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:47 smithi189 ceph-mon[35475]: pgmap v622: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-09-12T12:32:48.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 137725 22.00 MB/sec execute 260 sec latency 93.524 ms 2024-09-12T12:32:49.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 138450 22.06 MB/sec execute 261 sec latency 77.454 ms 2024-09-12T12:32:49.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:49 smithi171 ceph-mon[35098]: pgmap v623: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-09-12T12:32:49.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:49 smithi137 ceph-mon[30950]: pgmap v623: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-09-12T12:32:49.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:49 smithi189 ceph-mon[35475]: pgmap v623: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-09-12T12:32:50.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 139173 22.14 MB/sec execute 262 sec latency 69.872 ms 2024-09-12T12:32:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 -- 172.21.15.137:0/1835139802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341069c0 msgr2=0x7f4234106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 --2- 172.21.15.137:0/1835139802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341069c0 0x7f4234106da0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4230009930 tx=0x7f423002a420 comp rx=0 tx=0).stop 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 -- 172.21.15.137:0/1835139802 shutdown_connections 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 --2- 172.21.15.137:0/1835139802 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4234101350 0x7f423410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 --2- 172.21.15.137:0/1835139802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f42341009b0 0x7f4234100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 --2- 172.21.15.137:0/1835139802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341069c0 0x7f4234106da0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 -- 172.21.15.137:0/1835139802 >> 172.21.15.137:0/1835139802 conn(0x7f4234077990 msgr2=0x7f42340fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 -- 172.21.15.137:0/1835139802 shutdown_connections 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.879+0000 7f423c642640 1 -- 172.21.15.137:0/1835139802 wait complete. 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 Processor -- start 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 -- start start 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341009b0 0x7f42341991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4234101350 0x7f4234199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42341069c0 0x7f42341a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4234110dc0 con 0x7f42341009b0 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4234110c40 con 0x7f42341069c0 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423c642640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4234110f40 con 0x7f4234101350 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423a3de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341009b0 0x7f42341991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423a3de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341009b0 0x7f42341991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46836/0 (socket says 172.21.15.137:46836) 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f423a3de640 1 -- 172.21.15.137:0/158703828 learned_addr learned my addr 172.21.15.137:0/158703828 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.880+0000 7f4239bdd640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4234101350 0x7f4234199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423abdf640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42341069c0 0x7f42341a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423a3de640 1 -- 172.21.15.137:0/158703828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4234101350 msgr2=0x7f4234199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423a3de640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4234101350 0x7f4234199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423a3de640 1 -- 172.21.15.137:0/158703828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42341069c0 msgr2=0x7f42341a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423a3de640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42341069c0 0x7f42341a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423a3de640 1 -- 172.21.15.137:0/158703828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42341a4780 con 0x7f42341009b0 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f4239bdd640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4234101350 0x7f4234199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423a3de640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341009b0 0x7f42341991c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4230009230 tx=0x7f4230004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f422b7fe640 1 -- 172.21.15.137:0/158703828 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423003f030 con 0x7f42341009b0 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42341a4950 con 0x7f42341009b0 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f422b7fe640 1 -- 172.21.15.137:0/158703828 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4230004440 con 0x7f42341009b0 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f422b7fe640 1 -- 172.21.15.137:0/158703828 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230035070 con 0x7f42341009b0 2024-09-12T12:32:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.881+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42341a4e10 con 0x7f42341009b0 2024-09-12T12:32:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.885+0000 7f423f3d3740 1 Processor -- start 2024-09-12T12:32:50.887 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.885+0000 7f423f3d3740 1 -- start start 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.885+0000 7f422b7fe640 1 -- 172.21.15.137:0/158703828 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4230042030 con 0x7f42341009b0 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.885+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c049d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.885+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c058b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c05b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423f3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae6bfae2f0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423f3d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ae6bf56270 con 0x55ae6c0590c0 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423f3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae6bfaa2b0 con 0x55ae6c0499b0 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423a3de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c058b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423abdf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c049d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423abdf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c049d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:47100/0 (socket says 172.21.15.137:47100) 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423abdf640 1 -- 172.21.15.137:0/3520502995 learned_addr learned my addr 172.21.15.137:0/3520502995 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423a3de640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c058b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46840/0 (socket says 172.21.15.137:46840) 2024-09-12T12:32:50.888 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f4239bdd640 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c05b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423a3de640 1 -- 172.21.15.137:0/3520502995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 msgr2=0x55ae6c049d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423a3de640 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c049d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423a3de640 1 -- 172.21.15.137:0/3520502995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 msgr2=0x55ae6c05b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423a3de640 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c05b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423a3de640 1 -- 172.21.15.137:0/3520502995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6bf55340 con 0x55ae6c04a2d0 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f4239bdd640 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c05b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f422b7fe640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f42200a2da0 0x7f42200a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.886+0000 7f423abdf640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f42200a2da0 0x7f42200a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.887+0000 7f422b7fe640 1 -- 172.21.15.137:0/158703828 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f42300ef700 con 0x7f42341009b0 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.887+0000 7f423a3de640 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c058b80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f42300ece70 tx=0x7f42300f5cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e5eaabc38f5a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.887+0000 7f4228bfe640 1 -- 172.21.15.137:0/3520502995 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230049020 con 0x55ae6c04a2d0 2024-09-12T12:32:50.889 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.887+0000 7f4228bfe640 1 -- 172.21.15.137:0/3520502995 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42300bbcf0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.890 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.887+0000 7f4228bfe640 1 -- 172.21.15.137:0/3520502995 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300b7f90 con 0x55ae6c04a2d0 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 -- 172.21.15.137:0/3520502995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 msgr2=0x55ae6c058b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423abdf640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f42200a2da0 0x7f42200a5260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f422c011630 tx=0x7f422c009500 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c058b80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f42300ece70 tx=0x7f42300f5cd0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 -- 172.21.15.137:0/3520502995 shutdown_connections 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c05b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c058b80 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 --2- 172.21.15.137:0/3520502995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c049d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 -- 172.21.15.137:0/3520502995 >> 172.21.15.137:0/3520502995 conn(0x55ae6c046060 msgr2=0x55ae6c046440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 -- 172.21.15.137:0/3520502995 shutdown_connections 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 -- 172.21.15.137:0/3520502995 wait complete. 2024-09-12T12:32:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 -- start start 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.888+0000 7f423f3d3740 1 Processor -- start 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c0590c0 0x55ae6c056af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423f3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae6c05ba50 con 0x55ae6c0590c0 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423f3d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ae6bf37290 con 0x55ae6c04a2d0 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423f3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae6c05bbd0 con 0x55ae6c0499b0 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423f3d3740 10 client.0 _ll_register_callbacks cb 0x55ae6bf68140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f4239bdd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:47108/0 (socket says 172.21.15.137:47108) 2024-09-12T12:32:50.895 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 -- 172.21.15.137:0/4200283100 learned_addr learned my addr 172.21.15.137:0/4200283100 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423abdf640 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c0590c0 0x55ae6c056af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 -- 172.21.15.137:0/4200283100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c04a2d0 msgr2=0x55ae6c0565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 -- 172.21.15.137:0/4200283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c0590c0 msgr2=0x55ae6c056af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c0590c0 0x55ae6c056af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 -- 172.21.15.137:0/4200283100 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6bcc1210 con 0x55ae6c0499b0 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423abdf640 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c0590c0 0x55ae6c056af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f423a3de640 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4230000c00 tx=0x7f42300f5960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d6e62c7de8abfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.889+0000 7f421dffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f421dffb640 1 -- 172.21.15.137:0/4200283100 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230049070 con 0x55ae6c0499b0 2024-09-12T12:32:50.896 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42300bbcf0 con 0x55ae6c0499b0 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f421dffb640 1 -- 172.21.15.137:0/4200283100 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f42300ed0f0 con 0x55ae6c0499b0 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f421dffb640 1 -- 172.21.15.137:0/4200283100 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f42300bb650 con 0x55ae6c0499b0 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 10 client.27076 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ae6bd8d620 con 0x55ae6c0499b0 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f421dffb640 1 -- 172.21.15.137:0/4200283100 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42300bbcf0 con 0x55ae6c0499b0 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f421dffb640 1 -- 172.21.15.137:0/4200283100 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f42300b8360 con 0x55ae6c0499b0 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 10 client.27076 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 10 client.27076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 20 client.27076 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 4 client.27076 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae6bfbba20 con 0x55ae6c068c90 2024-09-12T12:32:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.890+0000 7f4239bdd640 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.891+0000 7f421dffb640 10 client.27076 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.891+0000 7f4239bdd640 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c06b080 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee7117c6cc19b17 server_cookie=3ab1136c9420e25d in_seq=0 out_seq=0 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.892+0000 7f421dffb640 1 -- 172.21.15.137:0/4200283100 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae6bfbba20 con 0x55ae6c068c90 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.892+0000 7f421dffb640 10 client.27076 handle_command_reply: tid=0 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.892+0000 7f423f3d3740 1 client.27076 shutdown 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 msgr2=0x55ae6c056070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4230000c00 tx=0x7f42300f5960 comp rx=0 tx=0).stop 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 shutdown_connections 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c06b080 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c0590c0 0x55ae6c056af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f421dffb640 0 client.27076 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 --2- 172.21.15.137:0/4200283100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 >> 172.21.15.137:0/4200283100 conn(0x55ae6c046060 msgr2=0x55ae6c04bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 shutdown_connections 2024-09-12T12:32:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 1 -- 172.21.15.137:0/4200283100 wait complete. 2024-09-12T12:32:50.899 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.893+0000 7f423f3d3740 20 client.27076 trim_cache size 0 max 16384 2024-09-12T12:32:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 Processor -- start 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 -- start start 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42341adec0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42341ab790 con 0x55ae6c0590c0 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423f3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42341a95e0 con 0x55ae6c0499b0 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f4239bdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423a3de640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f4239bdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0706b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46860/0 (socket says 172.21.15.137:46860) 2024-09-12T12:32:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f4239bdd640 1 -- 172.21.15.137:0/1169333952 learned_addr learned my addr 172.21.15.137:0/1169333952 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f423abdf640 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f4239bdd640 1 -- 172.21.15.137:0/1169333952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 msgr2=0x55ae6c056920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f4239bdd640 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f4239bdd640 1 -- 172.21.15.137:0/1169333952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 msgr2=0x55ae6c070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.912+0000 7f4239bdd640 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c070bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.913+0000 7f4239bdd640 1 -- 172.21.15.137:0/1169333952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6c064cb0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.913+0000 7f4239bdd640 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0706b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x55ae6c0558e0 tx=0x7f42240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb364e0587a2bf06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.913+0000 7f421ffff640 1 -- 172.21.15.137:0/1169333952 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240044e0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.913+0000 7f421ffff640 1 -- 172.21.15.137:0/1169333952 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4224005500 con 0x55ae6c04a2d0 2024-09-12T12:32:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.913+0000 7f421ffff640 1 -- 172.21.15.137:0/1169333952 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224010650 con 0x55ae6c04a2d0 2024-09-12T12:32:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 -- 172.21.15.137:0/1169333952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 msgr2=0x55ae6c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0706b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x55ae6c0558e0 tx=0x7f42240046c0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 -- 172.21.15.137:0/1169333952 shutdown_connections 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0590c0 0x55ae6c070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c0706b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 --2- 172.21.15.137:0/1169333952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0499b0 0x55ae6c056920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 -- 172.21.15.137:0/1169333952 >> 172.21.15.137:0/1169333952 conn(0x55ae6c046060 msgr2=0x55ae6c0580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 -- 172.21.15.137:0/1169333952 shutdown_connections 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.914+0000 7f423f3d3740 1 -- 172.21.15.137:0/1169333952 wait complete. 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 -- start start 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 Processor -- start 2024-09-12T12:32:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0499b0 0x55ae6c060e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c061360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0590c0 0x55ae6c05ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f42341a9c50 con 0x55ae6c04a2d0 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42341ab0c0 con 0x55ae6c0499b0 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f42341aa320 con 0x55ae6c0590c0 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423f3d3740 10 client.0 _ll_register_callbacks cb 0x7f4224009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f4239bdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c061360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f4239bdd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c061360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46866/0 (socket says 172.21.15.137:46866) 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f4239bdd640 1 -- 172.21.15.137:0/569758054 learned_addr learned my addr 172.21.15.137:0/569758054 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423a3de640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0499b0 0x55ae6c060e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f423abdf640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0590c0 0x55ae6c05ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f4239bdd640 1 -- 172.21.15.137:0/569758054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0590c0 msgr2=0x55ae6c05ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.915+0000 7f4239bdd640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0590c0 0x55ae6c05ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f4239bdd640 1 -- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0499b0 msgr2=0x55ae6c060e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f4239bdd640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0499b0 0x55ae6c060e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f4239bdd640 1 -- 172.21.15.137:0/569758054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6c056e60 con 0x55ae6c04a2d0 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f423a3de640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0499b0 0x55ae6c060e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f423abdf640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0590c0 0x55ae6c05ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f4239bdd640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c061360 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4224000c00 tx=0x7f4224005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79fac2129406443 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f421dffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f421dffb640 1 -- 172.21.15.137:0/569758054 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224035060 con 0x55ae6c04a2d0 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f421dffb640 1 -- 172.21.15.137:0/569758054 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f4224012110 con 0x55ae6c04a2d0 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4224005500 con 0x55ae6c04a2d0 2024-09-12T12:32:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f421dffb640 1 -- 172.21.15.137:0/569758054 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f42240044e0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f423f3d3740 10 client.17829 fetch_fsmap learned FSMap version 54 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ae6c0d3be0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.916+0000 7f421dffb640 1 -- 172.21.15.137:0/569758054 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4224005500 con 0x55ae6c04a2d0 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f421dffb640 1 -- 172.21.15.137:0/569758054 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 54) ==== 8346+0+0 (secure 0 0 0) 0x7f42240109d0 con 0x55ae6c04a2d0 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423f3d3740 10 client.17829 fetch_fsmap finished waiting for FSMap version 54 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423f3d3740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423f3d3740 20 client.17829 populate_metadata read hostname 'smithi137' 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423f3d3740 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c0d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423f3d3740 4 client.17829 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae6bfb5d70 con 0x55ae6c068c90 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423a3de640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c0d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f423a3de640 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c0d62f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c85cd4496dd65 server_cookie=500982590074916c in_seq=0 out_seq=0 2024-09-12T12:32:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:50.917+0000 7f421dffb640 10 client.17829 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f421dffb640 1 -- 172.21.15.137:0/569758054 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae6bfb5d70 con 0x55ae6c068c90 2024-09-12T12:32:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f421dffb640 10 client.17829 handle_command_reply: tid=0 2024-09-12T12:32:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 client.17829 shutdown 2024-09-12T12:32:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 msgr2=0x55ae6c061360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c061360 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4224000c00 tx=0x7f4224005f10 comp rx=0 tx=0).stop 2024-09-12T12:32:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 shutdown_connections 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae6c0590c0 0x55ae6c05ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae6c068c90 0x55ae6c0d62f0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae6c04a2d0 0x55ae6c061360 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 --2- 172.21.15.137:0/569758054 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae6c0499b0 0x55ae6c060e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f421dffb640 0 client.17829 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 >> 172.21.15.137:0/569758054 conn(0x55ae6c046060 msgr2=0x55ae6c0449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 shutdown_connections 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 1 -- 172.21.15.137:0/569758054 wait complete. 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.188+0000 7f423f3d3740 20 client.17829 trim_cache size 0 max 16384 2024-09-12T12:32:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f42200a2da0 msgr2=0x7f42200a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f42200a2da0 0x7f42200a5260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f422c011630 tx=0x7f422c009500 comp rx=0 tx=0).stop 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341009b0 msgr2=0x7f42341991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341009b0 0x7f42341991c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4230009230 tx=0x7f4230004040 comp rx=0 tx=0).stop 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 shutdown_connections 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f42200a2da0 0x7f42200a5260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42341069c0 0x7f42341a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4234101350 0x7f4234199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 --2- 172.21.15.137:0/158703828 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42341009b0 0x7f42341991c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 >> 172.21.15.137:0/158703828 conn(0x7f4234077990 msgr2=0x7f4234103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 shutdown_connections 2024-09-12T12:32:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:32:51.189+0000 7f423c642640 1 -- 172.21.15.137:0/158703828 wait complete. 2024-09-12T12:32:51.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 139617 22.13 MB/sec execute 263 sec latency 219.324 ms 2024-09-12T12:32:51.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:51 smithi171 ceph-mon[35098]: pgmap v624: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-09-12T12:32:51.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:51 smithi137 ceph-mon[30950]: pgmap v624: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-09-12T12:32:51.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:51 smithi189 ceph-mon[35475]: pgmap v624: 97 pgs: 97 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.7 MiB/s wr, 142 op/s 2024-09-12T12:32:52.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 140248 22.21 MB/sec execute 264 sec latency 91.663 ms 2024-09-12T12:32:53.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 140833 22.23 MB/sec execute 265 sec latency 68.351 ms 2024-09-12T12:32:53.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:53 smithi171 ceph-mon[35098]: pgmap v625: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 335 op/s 2024-09-12T12:32:53.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:53 smithi137 ceph-mon[30950]: pgmap v625: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 335 op/s 2024-09-12T12:32:53.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:53 smithi189 ceph-mon[35475]: pgmap v625: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 335 op/s 2024-09-12T12:32:54.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 141477 22.25 MB/sec execute 266 sec latency 117.252 ms 2024-09-12T12:32:54.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:54 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:32:54.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:54 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:32:54.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:54 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:32:55.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 142162 22.31 MB/sec execute 267 sec latency 113.497 ms 2024-09-12T12:32:55.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:55 smithi171 ceph-mon[35098]: pgmap v626: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-12T12:32:55.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:55 smithi137 ceph-mon[30950]: pgmap v626: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-12T12:32:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:55 smithi189 ceph-mon[35475]: pgmap v626: 97 pgs: 97 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 314 op/s 2024-09-12T12:32:56.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 142793 22.35 MB/sec execute 268 sec latency 141.720 ms 2024-09-12T12:32:57.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 143523 22.45 MB/sec execute 269 sec latency 102.407 ms 2024-09-12T12:32:57.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:57 smithi171 ceph-mon[35098]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.1 MiB/s wr, 643 op/s 2024-09-12T12:32:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:57 smithi137 ceph-mon[30950]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.1 MiB/s wr, 643 op/s 2024-09-12T12:32:57.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:57 smithi189 ceph-mon[35475]: pgmap v627: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 9.1 MiB/s wr, 643 op/s 2024-09-12T12:32:58.234 INFO:tasks.workunit.client.0.smithi137.stdout: 10 144108 22.45 MB/sec execute 270 sec latency 78.808 ms 2024-09-12T12:32:59.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 144816 22.52 MB/sec execute 271 sec latency 83.375 ms 2024-09-12T12:32:59.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:32:59 smithi171 ceph-mon[35098]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 527 op/s 2024-09-12T12:32:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:32:59 smithi137 ceph-mon[30950]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 527 op/s 2024-09-12T12:32:59.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:32:59 smithi189 ceph-mon[35475]: pgmap v628: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 527 op/s 2024-09-12T12:33:00.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145559 22.59 MB/sec execute 272 sec latency 93.710 ms 2024-09-12T12:33:01.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145765 22.54 MB/sec execute 273 sec latency 688.285 ms 2024-09-12T12:33:01.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:01 smithi171 ceph-mon[35098]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 557 op/s 2024-09-12T12:33:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:01 smithi137 ceph-mon[30950]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 557 op/s 2024-09-12T12:33:01.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:01 smithi189 ceph-mon[35475]: pgmap v629: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 557 op/s 2024-09-12T12:33:02.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145765 22.45 MB/sec execute 274 sec latency 1688.391 ms 2024-09-12T12:33:03.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145767 22.37 MB/sec execute 275 sec latency 2688.443 ms 2024-09-12T12:33:03.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:03 smithi171 ceph-mon[35098]: pgmap v630: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 740 op/s 2024-09-12T12:33:03.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:03 smithi137 ceph-mon[30950]: pgmap v630: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 740 op/s 2024-09-12T12:33:03.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:03 smithi189 ceph-mon[35475]: pgmap v630: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 740 op/s 2024-09-12T12:33:04.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145853 22.31 MB/sec execute 276 sec latency 2706.115 ms 2024-09-12T12:33:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 -- 172.21.15.137:0/4138912322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d441016d0 msgr2=0x7f4d44101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 --2- 172.21.15.137:0/4138912322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d441016d0 0x7f4d44101b10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009900 tx=0x7f4d3402a480 comp rx=0 tx=0).stop 2024-09-12T12:33:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 -- 172.21.15.137:0/4138912322 shutdown_connections 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 --2- 172.21.15.137:0/4138912322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d44102050 0x7f4d4410d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 --2- 172.21.15.137:0/4138912322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d441016d0 0x7f4d44101b10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 --2- 172.21.15.137:0/4138912322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d44105af0 0x7f4d44105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 -- 172.21.15.137:0/4138912322 >> 172.21.15.137:0/4138912322 conn(0x7f4d44075ec0 msgr2=0x7f4d440762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 -- 172.21.15.137:0/4138912322 shutdown_connections 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.284+0000 7f4d4b623640 1 -- 172.21.15.137:0/4138912322 wait complete. 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 Processor -- start 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 -- start start 2024-09-12T12:33:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d441016d0 0x7f4d44199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d44102050 0x7f4d441995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d44105af0 0x7f4d441a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d44110de0 con 0x7f4d44102050 2024-09-12T12:33:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4d44110c60 con 0x7f4d44105af0 2024-09-12T12:33:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d4b623640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d44110f60 con 0x7f4d441016d0 2024-09-12T12:33:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d44102050 0x7f4d441995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d44102050 0x7f4d441995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38924/0 (socket says 172.21.15.137:38924) 2024-09-12T12:33:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 -- 172.21.15.137:0/719379420 learned_addr learned my addr 172.21.15.137:0/719379420 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d493bf640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d441016d0 0x7f4d44199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d44105af0 0x7f4d441a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 -- 172.21.15.137:0/719379420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d441016d0 msgr2=0x7f4d44199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d441016d0 0x7f4d44199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 -- 172.21.15.137:0/719379420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d44105af0 msgr2=0x7f4d441a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d44105af0 0x7f4d441a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d48bbe640 1 -- 172.21.15.137:0/719379420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d441a4650 con 0x7f4d44102050 2024-09-12T12:33:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.285+0000 7f4d493bf640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d441016d0 0x7f4d44199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.286+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d44102050 0x7f4d441995d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006010 tx=0x7f4d340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.286+0000 7f4d3a7fc640 1 -- 172.21.15.137:0/719379420 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3403f030 con 0x7f4d44102050 2024-09-12T12:33:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.286+0000 7f4d3a7fc640 1 -- 172.21.15.137:0/719379420 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d34003ce0 con 0x7f4d44102050 2024-09-12T12:33:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.286+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d441a4820 con 0x7f4d44102050 2024-09-12T12:33:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.286+0000 7f4d3a7fc640 1 -- 172.21.15.137:0/719379420 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34035070 con 0x7f4d44102050 2024-09-12T12:33:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.286+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d441a4c80 con 0x7f4d44102050 2024-09-12T12:33:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d3a7fc640 1 -- 172.21.15.137:0/719379420 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4d34042030 con 0x7f4d44102050 2024-09-12T12:33:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 Processor -- start 2024-09-12T12:33:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 -- start start 2024-09-12T12:33:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0d0410 0x561daa0da770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0dd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561daa0342f0 con 0x561daa0dacb0 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561da9fdc270 con 0x561daa0d0410 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561daa0302b0 con 0x561daa0cfaf0 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d48bbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0dd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d48bbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0dd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38932/0 (socket says 172.21.15.137:38932) 2024-09-12T12:33:04.306 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d48bbe640 1 -- 172.21.15.137:0/1658813646 learned_addr learned my addr 172.21.15.137:0/1658813646 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d493bf640 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0d0410 0x561daa0da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.289+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d3a7fc640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4d300a2da0 0x7f4d300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d48bbe640 1 -- 172.21.15.137:0/1658813646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 msgr2=0x561daa0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d48bbe640 1 -- 172.21.15.137:0/1658813646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0d0410 msgr2=0x561daa0da770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0d0410 0x561daa0da770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d48bbe640 1 -- 172.21.15.137:0/1658813646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561da9fdb340 con 0x561daa0dacb0 2024-09-12T12:33:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4d300a2da0 0x7f4d300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d3a7fc640 1 -- 172.21.15.137:0/719379420 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f4d340ef9f0 con 0x7f4d44102050 2024-09-12T12:33:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0dd0a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4d340ed0d0 tx=0x7f4d340b7d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd81d38fb6cdadcc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d2f7fe640 1 -- 172.21.15.137:0/1658813646 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34049030 con 0x561daa0dacb0 2024-09-12T12:33:04.308 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d2f7fe640 1 -- 172.21.15.137:0/1658813646 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3403b580 con 0x561daa0dacb0 2024-09-12T12:33:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.290+0000 7f4d2f7fe640 1 -- 172.21.15.137:0/1658813646 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340bbb30 con 0x561daa0dacb0 2024-09-12T12:33:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1658813646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 msgr2=0x561daa0dd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4d300a2da0 0x7f4d300a5260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c005fd0 tx=0x7f4d3c004a40 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0dd0a0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4d340ed0d0 tx=0x7f4d340b7d40 comp rx=0 tx=0).stop 2024-09-12T12:33:04.309 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1658813646 shutdown_connections 2024-09-12T12:33:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0dd0a0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0d0410 0x561daa0da770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.310 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1658813646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1658813646 >> 172.21.15.137:0/1658813646 conn(0x561daa0cc060 msgr2=0x561daa0cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1658813646 shutdown_connections 2024-09-12T12:33:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1658813646 wait complete. 2024-09-12T12:33:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 -- start start 2024-09-12T12:33:04.311 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.291+0000 7f4d4e3ba740 1 Processor -- start 2024-09-12T12:33:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0cfaf0 0x561daa0d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x561daa0dd640 con 0x561daa0cfaf0 2024-09-12T12:33:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561da9fbd290 con 0x561daa0dacb0 2024-09-12T12:33:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x561daa0dd7c0 con 0x561daa0d0410 2024-09-12T12:33:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 10 client.0 _ll_register_callbacks cb 0x561da9fee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:33:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0cfaf0 0x561daa0d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0cfaf0 0x561daa0d7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38938/0 (socket says 172.21.15.137:38938) 2024-09-12T12:33:04.313 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 -- 172.21.15.137:0/1865891162 learned_addr learned my addr 172.21.15.137:0/1865891162 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 -- 172.21.15.137:0/1865891162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 msgr2=0x561daa0d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0d80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.314 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 -- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 msgr2=0x561daa0d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 -- 172.21.15.137:0/1865891162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561da9d47210 con 0x561daa0cfaf0 2024-09-12T12:33:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d493bf640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0cfaf0 0x561daa0d7b90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4d40011a60 tx=0x7f4d40011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f39be2312ff8fa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:33:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d2cff9640 1 -- 172.21.15.137:0/1865891162 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000fdb0 con 0x561daa0cfaf0 2024-09-12T12:33:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d2cff9640 1 -- 172.21.15.137:0/1865891162 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f4d40004020 con 0x561daa0cfaf0 2024-09-12T12:33:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561daa041a20 con 0x561daa0cfaf0 2024-09-12T12:33:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d2cff9640 1 -- 172.21.15.137:0/1865891162 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4d40004220 con 0x561daa0cfaf0 2024-09-12T12:33:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 10 client.17841 fetch_fsmap learned FSMap version 55 2024-09-12T12:33:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.292+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561da9e13620 con 0x561daa0cfaf0 2024-09-12T12:33:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d2cff9640 1 -- 172.21.15.137:0/1865891162 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561daa041a20 con 0x561daa0cfaf0 2024-09-12T12:33:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d2cff9640 1 -- 172.21.15.137:0/1865891162 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 55) ==== 8350+0+0 (secure 0 0 0) 0x7f4d40020020 con 0x561daa0cfaf0 2024-09-12T12:33:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d4e3ba740 10 client.17841 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T12:33:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d4e3ba740 10 client.17841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:33:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d4e3ba740 20 client.17841 populate_metadata read hostname 'smithi137' 2024-09-12T12:33:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa0ea950 0x561daa0ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d4e3ba740 4 client.17841 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:33:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561daa03bd70 con 0x561daa0ea950 2024-09-12T12:33:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa0ea950 0x561daa0ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d2cff9640 10 client.17841 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.293+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa0ea950 0x561daa0ecda0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca070547fdf7ac8 server_cookie=c72b9e8de4c1dbd6 in_seq=0 out_seq=0 2024-09-12T12:33:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d2cff9640 1 -- 172.21.15.137:0/1865891162 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561daa03bd70 con 0x561daa0ea950 2024-09-12T12:33:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d2cff9640 10 client.17841 handle_command_reply: tid=0 2024-09-12T12:33:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 client.17841 shutdown 2024-09-12T12:33:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0cfaf0 msgr2=0x561daa0d7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0cfaf0 0x561daa0d7b90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4d40011a60 tx=0x7f4d40011f20 comp rx=0 tx=0).stop 2024-09-12T12:33:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 shutdown_connections 2024-09-12T12:33:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa0ea950 0x561daa0ecda0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/1865891162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0cfaf0 0x561daa0d7b90 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d2cff9640 0 client.17841 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 >> 172.21.15.137:0/1865891162 conn(0x561daa0cc060 msgr2=0x561daa0d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 shutdown_connections 2024-09-12T12:33:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/1865891162 wait complete. 2024-09-12T12:33:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.294+0000 7f4d4e3ba740 20 client.17841 trim_cache size 0 max 16384 2024-09-12T12:33:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.308+0000 7f4d4e3ba740 1 Processor -- start 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.308+0000 7f4d4e3ba740 1 -- start start 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0cfaf0 0x561daa0d8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0f2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d441aff40 con 0x561daa0dacb0 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4d441ad810 con 0x561daa0cfaf0 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d441ab680 con 0x561daa0d0410 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0f2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38952/0 (socket says 172.21.15.137:38952) 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 -- 172.21.15.137:0/2432378065 learned_addr learned my addr 172.21.15.137:0/2432378065 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d493bf640 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0cfaf0 0x561daa0d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 -- 172.21.15.137:0/2432378065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 msgr2=0x561daa0f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 -- 172.21.15.137:0/2432378065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0cfaf0 msgr2=0x561daa0d8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0cfaf0 0x561daa0d8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 -- 172.21.15.137:0/2432378065 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561daa0cbe90 con 0x561daa0dacb0 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0f2c40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c01f820 tx=0x7f4d3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a36d63f837dbf94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d2effd640 1 -- 172.21.15.137:0/2432378065 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c01f220 con 0x561daa0dacb0 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d2effd640 1 -- 172.21.15.137:0/2432378065 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c016070 con 0x561daa0dacb0 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.309+0000 7f4d2effd640 1 -- 172.21.15.137:0/2432378065 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c01ee10 con 0x561daa0dacb0 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/2432378065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 msgr2=0x561daa0f2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0f2c40 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c01f820 tx=0x7f4d3c014000 comp rx=0 tx=0).stop 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/2432378065 shutdown_connections 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0dacb0 0x561daa0f2c40 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0d0410 0x561daa0f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/2432378065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0cfaf0 0x561daa0d8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/2432378065 >> 172.21.15.137:0/2432378065 conn(0x561daa0cc060 msgr2=0x561daa0d9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/2432378065 shutdown_connections 2024-09-12T12:33:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/2432378065 wait complete. 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 -- start start 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.310+0000 7f4d4e3ba740 1 Processor -- start 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0d0410 0x561daa0debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0df110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4d441abcd0 con 0x561daa0d0410 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4d441ad140 con 0x561daa0dacb0 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d441ac3a0 con 0x561daa0cfaf0 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 10 client.0 _ll_register_callbacks cb 0x7f4d40008760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0d0410 0x561daa0debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0d0410 0x561daa0debd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38968/0 (socket says 172.21.15.137:38968) 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 -- 172.21.15.137:0/365881052 learned_addr learned my addr 172.21.15.137:0/365881052 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d493bf640 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d49bc0640 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 -- 172.21.15.137:0/365881052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 msgr2=0x561daa0de690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0de690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 -- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 msgr2=0x561daa0df110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0df110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 -- 172.21.15.137:0/365881052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561daa0cd1f0 con 0x561daa0d0410 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d48bbe640 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0d0410 0x561daa0debd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4d34000c00 tx=0x7f4d3402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c948fea289fe19 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:33:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d2cff9640 1 -- 172.21.15.137:0/365881052 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340044e0 con 0x561daa0d0410 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d2cff9640 1 -- 172.21.15.137:0/365881052 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8484+0+0 (secure 0 0 0) 0x7f4d340ba2f0 con 0x561daa0d0410 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c016070 con 0x561daa0d0410 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.311+0000 7f4d2cff9640 1 -- 172.21.15.137:0/365881052 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4d3403bcb0 con 0x561daa0d0410 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 10 client.17847 fetch_fsmap learned FSMap version 55 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x561daa155790 con 0x561daa0d0410 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d2cff9640 1 -- 172.21.15.137:0/365881052 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d340d7030 con 0x561daa0d0410 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d2cff9640 1 -- 172.21.15.137:0/365881052 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 55) ==== 8350+0+0 (secure 0 0 0) 0x7f4d340ed3e0 con 0x561daa0d0410 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 10 client.17847 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 10 client.17847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 20 client.17847 populate_metadata read hostname 'smithi137' 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa155e20 0x561daa158240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 4 client.17847 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561da9fb8900 con 0x561daa155e20 2024-09-12T12:33:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.312+0000 7f4d493bf640 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa155e20 0x561daa158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.313+0000 7f4d2cff9640 10 client.17847 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.313+0000 7f4d493bf640 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa155e20 0x561daa158240 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f4bf300e957a01 server_cookie=d15c3257b26cbb08 in_seq=0 out_seq=0 2024-09-12T12:33:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.549+0000 7f4d2cff9640 1 -- 172.21.15.137:0/365881052 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561da9fb8900 con 0x561daa155e20 2024-09-12T12:33:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.549+0000 7f4d2cff9640 10 client.17847 handle_command_reply: tid=0 2024-09-12T12:33:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.549+0000 7f4d4e3ba740 1 client.17847 shutdown 2024-09-12T12:33:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.549+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0d0410 msgr2=0x561daa0debd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.549+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0d0410 0x561daa0debd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4d34000c00 tx=0x7f4d3402a480 comp rx=0 tx=0).stop 2024-09-12T12:33:04.550 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 shutdown_connections 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561daa0dacb0 0x561daa0df110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x561daa155e20 0x561daa158240 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d2cff9640 0 client.17847 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561daa0d0410 0x561daa0debd0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 --2- 172.21.15.137:0/365881052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561daa0cfaf0 0x561daa0de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 >> 172.21.15.137:0/365881052 conn(0x561daa0cc060 msgr2=0x561daa0d1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 shutdown_connections 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 1 -- 172.21.15.137:0/365881052 wait complete. 2024-09-12T12:33:04.551 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.550+0000 7f4d4e3ba740 20 client.17847 trim_cache size 0 max 16384 2024-09-12T12:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:33:04.552 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4d300a2da0 msgr2=0x7f4d300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4d300a2da0 0x7f4d300a5260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c005fd0 tx=0x7f4d3c004a40 comp rx=0 tx=0).stop 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d44102050 msgr2=0x7f4d441995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d44102050 0x7f4d441995d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4d34006010 tx=0x7f4d340042f0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 shutdown_connections 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4d300a2da0 0x7f4d300a5260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d44105af0 0x7f4d441a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4d44102050 0x7f4d441995d0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 --2- 172.21.15.137:0/719379420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d441016d0 0x7f4d44199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 >> 172.21.15.137:0/719379420 conn(0x7f4d44075ec0 msgr2=0x7f4d440ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 shutdown_connections 2024-09-12T12:33:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:04.551+0000 7f4d4b623640 1 -- 172.21.15.137:0/719379420 wait complete. 2024-09-12T12:33:05.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145853 22.22 MB/sec execute 277 sec latency 1845.462 ms 2024-09-12T12:33:05.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:05 smithi171 ceph-mon[35098]: pgmap v631: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 545 op/s 2024-09-12T12:33:05.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:05 smithi137 ceph-mon[30950]: pgmap v631: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 545 op/s 2024-09-12T12:33:05.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:05 smithi189 ceph-mon[35475]: pgmap v631: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 545 op/s 2024-09-12T12:33:06.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145853 22.14 MB/sec execute 278 sec latency 2845.590 ms 2024-09-12T12:33:07.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145853 22.07 MB/sec execute 279 sec latency 3845.692 ms 2024-09-12T12:33:07.700 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:07 smithi171 ceph-mon[35098]: pgmap v632: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 684 op/s 2024-09-12T12:33:07.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:07 smithi137 ceph-mon[30950]: pgmap v632: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 684 op/s 2024-09-12T12:33:07.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:07 smithi189 ceph-mon[35475]: pgmap v632: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 684 op/s 2024-09-12T12:33:08.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 145996 22.02 MB/sec execute 280 sec latency 4633.131 ms 2024-09-12T12:33:09.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 146379 22.00 MB/sec execute 281 sec latency 110.781 ms 2024-09-12T12:33:09.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:09 smithi171 ceph-mon[35098]: pgmap v633: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 355 op/s 2024-09-12T12:33:09.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:09 smithi137 ceph-mon[30950]: pgmap v633: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 355 op/s 2024-09-12T12:33:09.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:09 smithi189 ceph-mon[35475]: pgmap v633: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 355 op/s 2024-09-12T12:33:10.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 146697 22.00 MB/sec execute 282 sec latency 107.180 ms 2024-09-12T12:33:11.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 147345 22.06 MB/sec execute 283 sec latency 85.269 ms 2024-09-12T12:33:11.699 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:11 smithi171 ceph-mon[35098]: pgmap v634: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-09-12T12:33:11.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:11 smithi137 ceph-mon[30950]: pgmap v634: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-09-12T12:33:11.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:11 smithi189 ceph-mon[35475]: pgmap v634: 97 pgs: 97 active+clean; 1.2 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-09-12T12:33:12.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 148036 22.08 MB/sec execute 284 sec latency 104.541 ms 2024-09-12T12:33:13.235 INFO:tasks.workunit.client.0.smithi137.stdout: 10 148433 22.10 MB/sec execute 285 sec latency 162.735 ms 2024-09-12T12:33:13.748 INFO:tasks.workunit.client.0.smithi137.stdout:[178890] write failed on handle 10755 (Resource temporarily unavailable) 2024-09-12T12:33:13.749 INFO:tasks.workunit.client.0.smithi137.stdout:Child failed with status 1 2024-09-12T12:33:13.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:33:13.781 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-12T12:33:13.781 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T12:33:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:13 smithi137 ceph-mon[30950]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.7 MiB/s wr, 547 op/s 2024-09-12T12:33:13.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:13 smithi137 ceph-mon[30950]: osdmap e119: 12 total, 12 up, 12 in 2024-09-12T12:33:13.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:13 smithi189 ceph-mon[35475]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.7 MiB/s wr, 547 op/s 2024-09-12T12:33:13.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:13 smithi189 ceph-mon[35475]: osdmap e119: 12 total, 12 up, 12 in 2024-09-12T12:33:13.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:13 smithi171 ceph-mon[35098]: pgmap v635: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.7 MiB/s wr, 547 op/s 2024-09-12T12:33:13.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:13 smithi171 ceph-mon[35098]: osdmap e119: 12 total, 12 up, 12 in 2024-09-12T12:33:14.233 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi137 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-12T12:33:14.427 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=05658077be8f42f6a4aafa817f1afdc8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi137 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-12T12:33:14.429 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T12:33:14.440 INFO:tasks.quiescer:joining Quiescers 2024-09-12T12:33:14.440 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T12:33:14.441 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:33:14.819 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:15.589 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:15 smithi137 ceph-mon[30950]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 432 op/s 2024-09-12T12:33:15.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 -- 172.21.15.137:0/1868835313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102570 msgr2=0x7fee981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:15.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 --2- 172.21.15.137:0/1868835313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102570 0x7fee981029b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a480 comp rx=0 tx=0).stop 2024-09-12T12:33:15.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 -- 172.21.15.137:0/1868835313 shutdown_connections 2024-09-12T12:33:15.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 --2- 172.21.15.137:0/1868835313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee98102ef0 0x7fee9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 --2- 172.21.15.137:0/1868835313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102570 0x7fee981029b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 --2- 172.21.15.137:0/1868835313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee98100e20 0x7fee98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 -- 172.21.15.137:0/1868835313 >> 172.21.15.137:0/1868835313 conn(0x7fee980fc7d0 msgr2=0x7fee980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:15.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 -- 172.21.15.137:0/1868835313 shutdown_connections 2024-09-12T12:33:15.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 -- 172.21.15.137:0/1868835313 wait complete. 2024-09-12T12:33:15.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 Processor -- start 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.669+0000 7feea016a640 1 -- start start 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee98100e20 0x7fee9810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee98102570 0x7fee9810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102ef0 0x7fee9810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fee981120a0 con 0x7fee98102ef0 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fee98111f20 con 0x7fee98100e20 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fee98112220 con 0x7fee98102570 2024-09-12T12:33:15.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102ef0 0x7fee9810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:15.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102ef0 0x7fee9810ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33742/0 (socket says 172.21.15.137:33742) 2024-09-12T12:33:15.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 -- 172.21.15.137:0/1019498405 learned_addr learned my addr 172.21.15.137:0/1019498405 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:15.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9df06640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee98100e20 0x7fee9810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:15.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9d705640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee98102570 0x7fee9810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:15.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 -- 172.21.15.137:0/1019498405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee98102570 msgr2=0x7fee9810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:15.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee98102570 0x7fee9810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 -- 172.21.15.137:0/1019498405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee98100e20 msgr2=0x7fee9810e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee98100e20 0x7fee9810e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 -- 172.21.15.137:0/1019498405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee981a6a30 con 0x7fee98102ef0 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee9e707640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102ef0 0x7fee9810ef50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fee9000eaa0 tx=0x7fee9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee8affd640 1 -- 172.21.15.137:0/1019498405 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9000cdb0 con 0x7fee98102ef0 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee981a6d20 con 0x7fee98102ef0 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee8affd640 1 -- 172.21.15.137:0/1019498405 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee90005020 con 0x7fee98102ef0 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7fee8affd640 1 -- 172.21.15.137:0/1019498405 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee900116b0 con 0x7fee98102ef0 2024-09-12T12:33:15.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.670+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a71e0 con 0x7fee98102ef0 2024-09-12T12:33:15.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.671+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee98101200 con 0x7fee98102ef0 2024-09-12T12:33:15.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.673+0000 7fee8affd640 1 -- 172.21.15.137:0/1019498405 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fee90005560 con 0x7fee98102ef0 2024-09-12T12:33:15.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.673+0000 7fee8affd640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fee840a2cf0 0x7fee840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:15.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.673+0000 7fee9df06640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fee840a2cf0 0x7fee840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:15.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.673+0000 7fee8affd640 1 -- 172.21.15.137:0/1019498405 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7fee900cf980 con 0x7fee98102ef0 2024-09-12T12:33:15.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.674+0000 7fee9df06640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fee840a2cf0 0x7fee840a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee8c005d00 tx=0x7fee8c00b410 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:15.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.676+0000 7fee8affd640 1 -- 172.21.15.137:0/1019498405 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee90016030 con 0x7fee98102ef0 2024-09-12T12:33:15.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:15 smithi189 ceph-mon[35475]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 432 op/s 2024-09-12T12:33:15.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.882+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee9810fa00 con 0x7fee98102ef0 2024-09-12T12:33:15.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.883+0000 7fee8affd640 1 -- 172.21.15.137:0/1019498405 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11937 (secure 0 0 0) 0x7fee9810fa00 con 0x7fee98102ef0 2024-09-12T12:33:15.883 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:33:15.884 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":55,"btime":"2024-09-12T12:32:53:405756+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:32:52.963735+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26861":{"gid":26861,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/4049352466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4049352466},{"type":"v1","addr":"172.21.15.189:6835","nonce":4049352466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:33:15.884 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 55 2024-09-12T12:33:15.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fee840a2cf0 msgr2=0x7fee840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:15.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fee840a2cf0 0x7fee840a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fee8c005d00 tx=0x7fee8c00b410 comp rx=0 tx=0).stop 2024-09-12T12:33:15.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102ef0 msgr2=0x7fee9810ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:15.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102ef0 0x7fee9810ef50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fee9000eaa0 tx=0x7fee9000ef60 comp rx=0 tx=0).stop 2024-09-12T12:33:15.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 shutdown_connections 2024-09-12T12:33:15.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fee840a2cf0 0x7fee840a51b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee98102ef0 0x7fee9810ef50 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fee98102570 0x7fee9810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 --2- 172.21.15.137:0/1019498405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee98100e20 0x7fee9810e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:15.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 >> 172.21.15.137:0/1019498405 conn(0x7fee980fc7d0 msgr2=0x7fee980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:15.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 shutdown_connections 2024-09-12T12:33:15.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:15.885+0000 7feea016a640 1 -- 172.21.15.137:0/1019498405 wait complete. 2024-09-12T12:33:15.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:15 smithi171 ceph-mon[35098]: pgmap v637: 97 pgs: 97 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 432 op/s 2024-09-12T12:33:16.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:16 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1019498405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:16.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:16 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1019498405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:16.873 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24476 quiesce db --cancel --all' 2024-09-12T12:33:16.873 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24476 quiesce db --cancel --all 2024-09-12T12:33:16.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:16 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1019498405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:17.126 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:17.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:17 smithi137 ceph-mon[30950]: pgmap v638: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-12T12:33:17.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 -- 172.21.15.137:0/3220814044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102ef0 msgr2=0x7fe56c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 --2- 172.21.15.137:0/3220814044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102ef0 0x7fe56c10da40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe564009ca0 tx=0x7fe56402a950 comp rx=0 tx=0).stop 2024-09-12T12:33:17.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 -- 172.21.15.137:0/3220814044 shutdown_connections 2024-09-12T12:33:17.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 --2- 172.21.15.137:0/3220814044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102ef0 0x7fe56c10da40 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 --2- 172.21.15.137:0/3220814044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe56c102570 0x7fe56c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 --2- 172.21.15.137:0/3220814044 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c100e20 0x7fe56c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 -- 172.21.15.137:0/3220814044 >> 172.21.15.137:0/3220814044 conn(0x7fe56c0fc7d0 msgr2=0x7fe56c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 -- 172.21.15.137:0/3220814044 shutdown_connections 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.619+0000 7fe57224f640 1 -- 172.21.15.137:0/3220814044 wait complete. 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 Processor -- start 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 -- start start 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c100e20 0x7fe56c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102570 0x7fe56c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe56c102ef0 0x7fe56c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c110f80 con 0x7fe56c102570 2024-09-12T12:33:17.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c110e00 con 0x7fe56c100e20 2024-09-12T12:33:17.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe57224f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c111100 con 0x7fe56c102ef0 2024-09-12T12:33:17.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe563fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102570 0x7fe56c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe563fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102570 0x7fe56c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33772/0 (socket says 172.21.15.137:33772) 2024-09-12T12:33:17.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe563fff640 1 -- 172.21.15.137:0/2962763067 learned_addr learned my addr 172.21.15.137:0/2962763067 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c100e20 0x7fe56c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe56bfff640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe56c102ef0 0x7fe56c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe563fff640 1 -- 172.21.15.137:0/2962763067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe56c102ef0 msgr2=0x7fe56c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.620+0000 7fe563fff640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe56c102ef0 0x7fe56c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe563fff640 1 -- 172.21.15.137:0/2962763067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c100e20 msgr2=0x7fe56c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe563fff640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c100e20 0x7fe56c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe563fff640 1 -- 172.21.15.137:0/2962763067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c19d470 con 0x7fe56c102570 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c100e20 0x7fe56c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe563fff640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102570 0x7fe56c198a00 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d950 tx=0x7fe55400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe5697fa640 1 -- 172.21.15.137:0/2962763067 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554017030 con 0x7fe56c102570 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c19d760 con 0x7fe56c102570 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe5697fa640 1 -- 172.21.15.137:0/2962763067 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554004020 con 0x7fe56c102570 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe5697fa640 1 -- 172.21.15.137:0/2962763067 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5540047a0 con 0x7fe56c102570 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.621+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c1a5040 con 0x7fe56c102570 2024-09-12T12:33:17.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe5697fa640 1 -- 172.21.15.137:0/2962763067 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe5540041d0 con 0x7fe56c102570 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 Processor -- start 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 -- start start 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cc1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ccefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae24c26200 con 0x55ae24cc2320 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ae24bccdc0 con 0x55ae24cccbe0 2024-09-12T12:33:17.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe574628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae24c221c0 con 0x55ae24cc1a00 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33784/0 (socket says 172.21.15.137:33784) 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 -- 172.21.15.137:0/4153593521 learned_addr learned my addr 172.21.15.137:0/4153593521 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56bfff640 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cc1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe563fff640 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ccefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe5697fa640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5580a2cf0 0x7fe5580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 -- 172.21.15.137:0/4153593521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 msgr2=0x55ae24cc1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cc1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 -- 172.21.15.137:0/4153593521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 msgr2=0x55ae24ccefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ccefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe56b7fe640 1 -- 172.21.15.137:0/4153593521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae24997620 con 0x55ae24cc2320 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.624+0000 7fe563fff640 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ccefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.625+0000 7fe56bfff640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5580a2cf0 0x7fe5580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.625+0000 7fe5697fa640 1 -- 172.21.15.137:0/2962763067 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7fe5540ced30 con 0x7fe56c102570 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.625+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccc6a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0115f0 tx=0x7fe55c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acee8a300dad4073 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.625+0000 7fe561ffb640 1 -- 172.21.15.137:0/4153593521 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c01e070 con 0x55ae24cc2320 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.625+0000 7fe561ffb640 1 -- 172.21.15.137:0/4153593521 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c00fb20 con 0x55ae24cc2320 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.625+0000 7fe561ffb640 1 -- 172.21.15.137:0/4153593521 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c019c40 con 0x55ae24cc2320 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 -- 172.21.15.137:0/4153593521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 msgr2=0x55ae24ccc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccc6a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0115f0 tx=0x7fe55c011ab0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe56bfff640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5580a2cf0 0x7fe5580a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe564005e30 tx=0x7fe5640095c0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 -- 172.21.15.137:0/4153593521 shutdown_connections 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ccefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccc6a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 --2- 172.21.15.137:0/4153593521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cc1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 -- 172.21.15.137:0/4153593521 >> 172.21.15.137:0/4153593521 conn(0x55ae24cbdf90 msgr2=0x55ae24cbe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 -- 172.21.15.137:0/4153593521 shutdown_connections 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 -- 172.21.15.137:0/4153593521 wait complete. 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 -- start start 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.626+0000 7fe574628740 1 Processor -- start 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cc1a00 0x55ae24ccc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc2320 0x55ae24cc97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cccbe0 0x55ae24cc9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55ae24ccf7b0 con 0x55ae24cccbe0 2024-09-12T12:33:17.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ae24bc0c50 con 0x55ae24cc1a00 2024-09-12T12:33:17.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ae24ccf930 con 0x55ae24cc2320 2024-09-12T12:33:17.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 10 client.0 _ll_register_callbacks cb 0x55ae24bd2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cc1a00 0x55ae24ccc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56bfff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cccbe0 0x55ae24cc9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cc1a00 0x55ae24ccc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.137:54872/0 (socket says 172.21.15.137:54872) 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 -- 172.21.15.137:0/1460259299 learned_addr learned my addr 172.21.15.137:0/1460259299 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe563fff640 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc2320 0x55ae24cc97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 -- 172.21.15.137:0/1460259299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc2320 msgr2=0x55ae24cc97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc2320 0x55ae24cc97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 -- 172.21.15.137:0/1460259299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cccbe0 msgr2=0x55ae24cc9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cccbe0 0x55ae24cc9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 -- 172.21.15.137:0/1460259299 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae24cca540 con 0x55ae24cc1a00 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cc1a00 0x55ae24ccc520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe55c01c040 tx=0x7fe55c00f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c572a8b0bf7a6f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe54affd640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe54affd640 1 -- 172.21.15.137:0/1460259299 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c029060 con 0x55ae24cc1a00 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55c00fb20 con 0x55ae24cc1a00 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.627+0000 7fe54affd640 1 -- 172.21.15.137:0/1460259299 <== mon.2 v2:172.21.15.171:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7fe55c0199c0 con 0x55ae24cc1a00 2024-09-12T12:33:17.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe54affd640 1 -- 172.21.15.137:0/1460259299 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe55c01e040 con 0x55ae24cc1a00 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 10 client.27068 fetch_fsmap learned FSMap version 55 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ae24cdb2a0 con 0x55ae24cc1a00 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe54affd640 1 -- 172.21.15.137:0/1460259299 <== mon.2 v2:172.21.15.171:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c00fb20 con 0x55ae24cc1a00 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe54affd640 1 -- 172.21.15.137:0/1460259299 <== mon.2 v2:172.21.15.171:3300/0 5 ==== fsmap(e 55) ==== 8350+0+0 (secure 0 0 0) 0x7fe55c0035f0 con 0x55ae24cc1a00 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 10 client.27068 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 10 client.27068 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 20 client.27068 populate_metadata read hostname 'smithi137' 2024-09-12T12:33:17.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24cdc760 0x55ae24cdeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 4 client.27068 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae24c33770 con 0x55ae24cdc760 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.628+0000 7fe563fff640 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24cdc760 0x55ae24cdeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe54affd640 10 client.27068 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe563fff640 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24cdc760 0x55ae24cdeb50 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2809fd76afc3f15 server_cookie=9e107ea7a197023e in_seq=0 out_seq=0 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe54affd640 1 -- 172.21.15.137:0/1460259299 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe5540abe80 con 0x55ae24cdc760 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe54affd640 10 client.27068 handle_command_reply: tid=0 2024-09-12T12:33:17.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 client.27068 shutdown 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cc1a00 msgr2=0x55ae24ccc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cc1a00 0x55ae24ccc520 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe55c01c040 tx=0x7fe55c00f8f0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 shutdown_connections 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24cdc760 0x55ae24cdeb50 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cccbe0 0x55ae24cc9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe54affd640 0 client.27068 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc2320 0x55ae24cc97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 --2- 172.21.15.137:0/1460259299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cc1a00 0x55ae24ccc520 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.629+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 >> 172.21.15.137:0/1460259299 conn(0x55ae24cbdf90 msgr2=0x55ae24cbd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.630+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 shutdown_connections 2024-09-12T12:33:17.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.630+0000 7fe574628740 1 -- 172.21.15.137:0/1460259299 wait complete. 2024-09-12T12:33:17.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.630+0000 7fe574628740 20 client.27068 trim_cache size 0 max 16384 2024-09-12T12:33:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 -- 172.21.15.137:0/977782534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c103600 msgr2=0x7f007c10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 --2- 172.21.15.137:0/977782534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c103600 0x7f007c10bae0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0078009900 tx=0x7f007802a480 comp rx=0 tx=0).stop 2024-09-12T12:33:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 -- 172.21.15.137:0/977782534 shutdown_connections 2024-09-12T12:33:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 --2- 172.21.15.137:0/977782534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f007c10c0b0 0x7f007c10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 --2- 172.21.15.137:0/977782534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c103600 0x7f007c10bae0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 --2- 172.21.15.137:0/977782534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f007c102ce0 0x7f007c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 -- 172.21.15.137:0/977782534 >> 172.21.15.137:0/977782534 conn(0x7f007c077990 msgr2=0x7f007c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 -- 172.21.15.137:0/977782534 shutdown_connections 2024-09-12T12:33:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.646+0000 7f00843cf640 1 -- 172.21.15.137:0/977782534 wait complete. 2024-09-12T12:33:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 Processor -- start 2024-09-12T12:33:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 -- start start 2024-09-12T12:33:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f007c102ce0 0x7f007c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f007c103600 0x7f007c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c10c0b0 0x7f007c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 Processor -- start 2024-09-12T12:33:17.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 -- start start 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f007c111eb0 con 0x7f007c10c0b0 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f007c111d30 con 0x7f007c103600 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f00843cf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f007c112030 con 0x7f007c102ce0 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f008296c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c10c0b0 0x7f007c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f008216b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f007c102ce0 0x7f007c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f008296c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c10c0b0 0x7f007c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33796/0 (socket says 172.21.15.137:33796) 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f008296c640 1 -- 172.21.15.137:0/4231206585 learned_addr learned my addr 172.21.15.137:0/4231206585 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f008196a640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f007c103600 0x7f007c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cd8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.647+0000 7f008296c640 1 -- 172.21.15.137:0/4231206585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f007c102ce0 msgr2=0x7f007c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f008296c640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f007c102ce0 0x7f007c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f008296c640 1 -- 172.21.15.137:0/4231206585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f007c103600 msgr2=0x7f007c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ce4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1b0210 con 0x55ae24cc2320 2024-09-12T12:33:17.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1adae0 con 0x55ae24cccbe0 2024-09-12T12:33:17.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe574628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1ab930 con 0x55ae24cc1a00 2024-09-12T12:33:17.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe563fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cd8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe563fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33806/0 (socket says 172.21.15.137:33806) 2024-09-12T12:33:17.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cd8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:55306/0 (socket says 172.21.15.137:55306) 2024-09-12T12:33:17.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.647+0000 7fe563fff640 1 -- 172.21.15.137:0/2073990824 learned_addr learned my addr 172.21.15.137:0/2073990824 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.665 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f008296c640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f007c103600 0x7f007c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f008296c640 1 -- 172.21.15.137:0/4231206585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c19d330 con 0x7f007c10c0b0 2024-09-12T12:33:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f008216b640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f007c102ce0 0x7f007c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:33:17.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56bfff640 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ce4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.666 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f008196a640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f007c103600 0x7f007c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:33:17.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56b7fe640 1 -- 172.21.15.137:0/2073990824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 msgr2=0x55ae24ce4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ce4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56b7fe640 1 -- 172.21.15.137:0/2073990824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 msgr2=0x55ae24ccb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56b7fe640 1 -- 172.21.15.137:0/2073990824 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae24cbeb40 con 0x55ae24cc1a00 2024-09-12T12:33:17.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe563fff640 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:33:17.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56bfff640 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ce4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:17.668 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f008296c640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c10c0b0 0x7f007c19cc00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f007400eb10 tx=0x7f007400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cd8a10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe55c003a40 tx=0x7fe55c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ebb048a9564df8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f00737fe640 1 -- 172.21.15.137:0/4231206585 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074004b20 con 0x7f007c10c0b0 2024-09-12T12:33:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f00737fe640 1 -- 172.21.15.137:0/4231206585 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0074004cc0 con 0x7f007c10c0b0 2024-09-12T12:33:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f00737fe640 1 -- 172.21.15.137:0/4231206585 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0074010760 con 0x7f007c10c0b0 2024-09-12T12:33:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007c19d620 con 0x7f007c10c0b0 2024-09-12T12:33:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.648+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007c1a4eb0 con 0x7f007c10c0b0 2024-09-12T12:33:17.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe5617fa640 1 -- 172.21.15.137:0/2073990824 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c019410 con 0x55ae24cc1a00 2024-09-12T12:33:17.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe5617fa640 1 -- 172.21.15.137:0/2073990824 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c0195b0 con 0x55ae24cc1a00 2024-09-12T12:33:17.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.648+0000 7fe5617fa640 1 -- 172.21.15.137:0/2073990824 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c005350 con 0x55ae24cc1a00 2024-09-12T12:33:17.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 -- 172.21.15.137:0/2073990824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 msgr2=0x55ae24cd8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cd8a10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe55c003a40 tx=0x7fe55c011ab0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 -- 172.21.15.137:0/2073990824 shutdown_connections 2024-09-12T12:33:17.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24ce4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24ccb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 --2- 172.21.15.137:0/2073990824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24cd8a10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 -- 172.21.15.137:0/2073990824 >> 172.21.15.137:0/2073990824 conn(0x55ae24cbdf90 msgr2=0x55ae24ccb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 -- 172.21.15.137:0/2073990824 shutdown_connections 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.649+0000 7fe574628740 1 -- 172.21.15.137:0/2073990824 wait complete. 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.650+0000 7fe574628740 1 -- start start 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.650+0000 7fe574628740 1 Processor -- start 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.650+0000 7fe574628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24d3f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.650+0000 7fe574628740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24d3fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.650+0000 7fe574628740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24cd3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.650+0000 7fe574628740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1abfa0 con 0x55ae24cc2320 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24d3f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe574628740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1ad410 con 0x55ae24cccbe0 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe574628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1ac670 con 0x55ae24cc1a00 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe563fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24d3fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe56b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24d3f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.137:55320/0 (socket says 172.21.15.137:55320) 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe56b7fe640 1 -- 172.21.15.137:0/996395097 learned_addr learned my addr 172.21.15.137:0/996395097 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe574628740 10 client.0 _ll_register_callbacks cb 0x7fe55c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:33:17.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe56bfff640 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24cd3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe563fff640 1 -- 172.21.15.137:0/996395097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 msgr2=0x55ae24d3f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe563fff640 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24d3f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe563fff640 1 -- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 msgr2=0x55ae24cd3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe563fff640 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24cd3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe563fff640 1 -- 172.21.15.137:0/996395097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae24d40420 con 0x55ae24cc2320 2024-09-12T12:33:17.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24d3f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:33:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.651+0000 7f0087247740 1 Processor -- start 2024-09-12T12:33:17.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe563fff640 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24d3fb20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe554097c20 tx=0x7fe554004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489f307073cee31c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe54affd640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:33:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.651+0000 7f00737fe640 1 -- 172.21.15.137:0/4231206585 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0074005020 con 0x7f007c10c0b0 2024-09-12T12:33:17.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe54affd640 1 -- 172.21.15.137:0/996395097 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55409b250 con 0x55ae24cc2320 2024-09-12T12:33:17.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe54affd640 1 -- 172.21.15.137:0/996395097 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7fe5540978c0 con 0x55ae24cc2320 2024-09-12T12:33:17.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.651+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55c0195b0 con 0x55ae24cc2320 2024-09-12T12:33:17.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe54affd640 1 -- 172.21.15.137:0/996395097 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe55409f600 con 0x55ae24cc2320 2024-09-12T12:33:17.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 10 client.17877 fetch_fsmap learned FSMap version 55 2024-09-12T12:33:17.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55ae24cd5230 con 0x55ae24cc2320 2024-09-12T12:33:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f0087247740 1 -- start start 2024-09-12T12:33:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f0087247740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909611e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe54affd640 1 -- 172.21.15.137:0/996395097 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe55c0195b0 con 0x55ae24cc2320 2024-09-12T12:33:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f0087247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909612360 0x558909612780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f0087247740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909623740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f0087247740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589095722f0 con 0x558909613440 2024-09-12T12:33:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f0087247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55890951a270 con 0x558909612360 2024-09-12T12:33:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f0087247740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55890956e2b0 con 0x558909621350 2024-09-12T12:33:17.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe54affd640 1 -- 172.21.15.137:0/996395097 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 55) ==== 8350+0+0 (secure 0 0 0) 0x7fe5540a9020 con 0x55ae24cc2320 2024-09-12T12:33:17.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 10 client.17877 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T12:33:17.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 10 client.17877 resolve_mds: validated gid 24476 aka daemon mds.b 2024-09-12T12:33:17.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 20 client.17877 populate_metadata read hostname 'smithi137' 2024-09-12T12:33:17.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24d4b670 0x55ae24d4da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 4 client.17877 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T12:33:17.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55ae24c2dc80 con 0x55ae24d4b670 2024-09-12T12:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f008196a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909611e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f008196a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909611e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33822/0 (socket says 172.21.15.137:33822) 2024-09-12T12:33:17.682 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f008196a640 1 -- 172.21.15.137:0/1301660996 learned_addr learned my addr 172.21.15.137:0/1301660996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.652+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24d4b670 0x55ae24d4da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.652+0000 7f008216b640 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909612360 0x558909612780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008196a640 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909623740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f00737fe640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f00680a2da0 0x7f00680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008216b640 1 -- 172.21.15.137:0/1301660996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 msgr2=0x558909623740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.683 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008216b640 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909623740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008216b640 1 -- 172.21.15.137:0/1301660996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 msgr2=0x558909611e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008216b640 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909611e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008216b640 1 -- 172.21.15.137:0/1301660996 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558909519340 con 0x558909612360 2024-09-12T12:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008296c640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f00680a2da0 0x7f00680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008196a640 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909623740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:33:17.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.653+0000 7fe56b7fe640 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24d4b670 0x55ae24d4da60 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581551635ed0d236 server_cookie=eff693c80ae9d817 in_seq=0 out_seq=0 2024-09-12T12:33:17.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.653+0000 7fe54affd640 10 client.17877 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008216b640 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909612360 0x558909612780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f006c009fd0 tx=0x7f006c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0ea3eb0c1ce18f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f00737fe640 1 -- 172.21.15.137:0/4231206585 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7f00740cfa80 con 0x7f007c10c0b0 2024-09-12T12:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f008296c640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f00680a2da0 0x7f00680a5260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f007409bca0 tx=0x7f0074098370 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.685 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f0070bfe640 1 -- 172.21.15.137:0/1301660996 <== mon.2 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c004030 con 0x558909612360 2024-09-12T12:33:17.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.653+0000 7fe54affd640 1 -- 172.21.15.137:0/996395097 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55ae24c2dc80 con 0x55ae24d4b670 2024-09-12T12:33:17.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.653+0000 7fe54affd640 10 client.17877 handle_command_reply: tid=0 2024-09-12T12:33:17.686 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.653+0000 7f0070bfe640 1 -- 172.21.15.137:0/1301660996 <== mon.2 v2:172.21.15.171:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c005500 con 0x558909612360 2024-09-12T12:33:17.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.653+0000 7fe574628740 1 client.17877 shutdown 2024-09-12T12:33:17.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 msgr2=0x55ae24d3fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24d3fb20 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe554097c20 tx=0x7fe554004540 comp rx=0 tx=0).stop 2024-09-12T12:33:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0070bfe640 1 -- 172.21.15.137:0/1301660996 <== mon.2 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c004e10 con 0x558909612360 2024-09-12T12:33:17.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 shutdown_connections 2024-09-12T12:33:17.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ae24cccbe0 0x55ae24cd3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55ae24d4b670 0x55ae24d4da60 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ae24cc2320 0x55ae24d3fb20 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 --2- 172.21.15.137:0/996395097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ae24cc1a00 0x55ae24d3f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 >> 172.21.15.137:0/996395097 conn(0x55ae24cbdf90 msgr2=0x55ae24cbd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe54affd640 0 client.17877 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:17.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 shutdown_connections 2024-09-12T12:33:17.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 1 -- 172.21.15.137:0/996395097 wait complete. 2024-09-12T12:33:17.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe574628740 20 client.17877 trim_cache size 0 max 16384 2024-09-12T12:33:17.689 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:33:17.690 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 55, 2024-09-12T12:33:17.690 INFO:teuthology.orchestra.run.smithi137.stdout: "leader": 24476, 2024-09-12T12:33:17.690 INFO:teuthology.orchestra.run.smithi137.stdout: "set_version": 72, 2024-09-12T12:33:17.690 INFO:teuthology.orchestra.run.smithi137.stdout: "sets": {} 2024-09-12T12:33:17.690 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 -- 172.21.15.137:0/1301660996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909612360 msgr2=0x558909612780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909612360 0x558909612780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f006c009fd0 tx=0x7f006c0118d0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 -- 172.21.15.137:0/1301660996 shutdown_connections 2024-09-12T12:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909623740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909612360 0x558909612780 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 --2- 172.21.15.137:0/1301660996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909611e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 -- 172.21.15.137:0/1301660996 >> 172.21.15.137:0/1301660996 conn(0x55890960e470 msgr2=0x55890960e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 -- 172.21.15.137:0/1301660996 shutdown_connections 2024-09-12T12:33:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.654+0000 7f0087247740 1 -- 172.21.15.137:0/1301660996 wait complete. 2024-09-12T12:33:17.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5580a2cf0 msgr2=0x7fe5580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe57224f640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5580a2cf0 0x7fe5580a51b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe564005e30 tx=0x7fe5640095c0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102570 msgr2=0x7fe56c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.654+0000 7fe57224f640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102570 0x7fe56c198a00 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d950 tx=0x7fe55400de10 comp rx=0 tx=0).stop 2024-09-12T12:33:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 -- start start 2024-09-12T12:33:17.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 shutdown_connections 2024-09-12T12:33:17.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5580a2cf0 0x7fe5580a51b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe56c102ef0 0x7fe56c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe56c102570 0x7fe56c198a00 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 --2- 172.21.15.137:0/2962763067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe56c100e20 0x7fe56c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 >> 172.21.15.137:0/2962763067 conn(0x7fe56c0fc7d0 msgr2=0x7fe56c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 shutdown_connections 2024-09-12T12:33:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 Processor -- start 2024-09-12T12:33:17.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:17.655+0000 7fe57224f640 1 -- 172.21.15.137:0/2962763067 wait complete. 2024-09-12T12:33:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x55890960aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909613440 0x55890960af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x55890960b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5589096241b0 con 0x558909612360 2024-09-12T12:33:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5589094fb290 con 0x558909621350 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558909624330 con 0x558909613440 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f0087247740 10 client.0 _ll_register_callbacks cb 0x55890952c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f008196a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x55890960aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f008196a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x55890960aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33834/0 (socket says 172.21.15.137:33834) 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f008196a640 1 -- 172.21.15.137:0/3779737046 learned_addr learned my addr 172.21.15.137:0/3779737046 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f008196a640 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x55890960b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.655+0000 7f008216b640 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909613440 0x55890960af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.704 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f008196a640 1 -- 172.21.15.137:0/3779737046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909613440 msgr2=0x55890960af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f008196a640 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909613440 0x55890960af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f008196a640 1 -- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 msgr2=0x55890960b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f008196a640 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x55890960b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f008196a640 1 -- 172.21.15.137:0/3779737046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558909285210 con 0x558909612360 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f008196a640 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x55890960aa10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0078006010 tx=0x7f0078031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad34b6800acc07a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f0065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f0065ffb640 1 -- 172.21.15.137:0/3779737046 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078035040 con 0x558909612360 2024-09-12T12:33:17.705 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006c005500 con 0x558909612360 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f0065ffb640 1 -- 172.21.15.137:0/3779737046 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7f00780035f0 con 0x558909612360 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.656+0000 7f0065ffb640 1 -- 172.21.15.137:0/3779737046 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f00780037f0 con 0x558909612360 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 10 client.17883 fetch_fsmap learned FSMap version 55 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558909351620 con 0x558909612360 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0065ffb640 1 -- 172.21.15.137:0/3779737046 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c005500 con 0x558909612360 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0065ffb640 1 -- 172.21.15.137:0/3779737046 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 55) ==== 8350+0+0 (secure 0 0 0) 0x7f00780549a0 con 0x558909612360 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 10 client.17883 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T12:33:17.706 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 10 client.17883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:33:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 20 client.17883 populate_metadata read hostname 'smithi137' 2024-09-12T12:33:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589096314c0 0x5589096338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 4 client.17883 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T12:33:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55890957fa20 con 0x5589096314c0 2024-09-12T12:33:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.657+0000 7f008216b640 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589096314c0 0x5589096338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.658+0000 7f0065ffb640 10 client.17883 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:17.707 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.658+0000 7f008216b640 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589096314c0 0x5589096338b0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e6a67f4fdf635bc server_cookie=89209e535489df in_seq=0 out_seq=0 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.658+0000 7f0065ffb640 1 -- 172.21.15.137:0/3779737046 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55890957fa20 con 0x5589096314c0 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.658+0000 7f0065ffb640 10 client.17883 handle_command_reply: tid=0 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.658+0000 7f0087247740 1 client.17883 shutdown 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.658+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 msgr2=0x55890960aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.658+0000 7f0087247740 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x55890960aa10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0078006010 tx=0x7f0078031420 comp rx=0 tx=0).stop 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 shutdown_connections 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x55890960b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909613440 0x55890960af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x5589096314c0 0x5589096338b0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 --2- 172.21.15.137:0/3779737046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x55890960aa10 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0065ffb640 0 client.17883 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 >> 172.21.15.137:0/3779737046 conn(0x55890960e470 msgr2=0x5589096141d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 shutdown_connections 2024-09-12T12:33:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 1 -- 172.21.15.137:0/3779737046 wait complete. 2024-09-12T12:33:17.709 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.659+0000 7f0087247740 20 client.17883 trim_cache size 0 max 16384 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.675+0000 7f0087247740 1 Processor -- start 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f0087247740 1 -- start start 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f0087247740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909612360 0x558909620600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f0087247740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909620b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f0087247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x558909639100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f0087247740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1adfa0 con 0x558909613440 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f0087247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1ab870 con 0x558909621350 2024-09-12T12:33:17.710 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f0087247740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1a96c0 con 0x558909612360 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909620b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909620b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33838/0 (socket says 172.21.15.137:33838) 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 -- 172.21.15.137:0/2407873123 learned_addr learned my addr 172.21.15.137:0/2407873123 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008196a640 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909612360 0x558909620600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008196a640 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x558909639100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 -- 172.21.15.137:0/2407873123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909612360 msgr2=0x558909620600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909612360 0x558909620600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 -- 172.21.15.137:0/2407873123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 msgr2=0x558909639100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x558909639100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 -- 172.21.15.137:0/2407873123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589096089a0 con 0x558909613440 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.676+0000 7f008216b640 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909620b40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f006c000c00 tx=0x7f006c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2508221fe1d746 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.677+0000 7f0067fff640 1 -- 172.21.15.137:0/2407873123 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c0042d0 con 0x558909613440 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.677+0000 7f0067fff640 1 -- 172.21.15.137:0/2407873123 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c004470 con 0x558909613440 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.677+0000 7f0067fff640 1 -- 172.21.15.137:0/2407873123 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c00f7c0 con 0x558909613440 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.677+0000 7f0087247740 1 -- 172.21.15.137:0/2407873123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 msgr2=0x558909620b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:17.712 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.677+0000 7f0087247740 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909620b40 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f006c000c00 tx=0x7f006c009310 comp rx=0 tx=0).stop 2024-09-12T12:33:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.677+0000 7f0087247740 1 -- 172.21.15.137:0/2407873123 shutdown_connections 2024-09-12T12:33:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.677+0000 7f0087247740 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909621350 0x558909639100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909613440 0x558909620b40 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 --2- 172.21.15.137:0/2407873123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909612360 0x558909620600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 -- 172.21.15.137:0/2407873123 >> 172.21.15.137:0/2407873123 conn(0x55890960e470 msgr2=0x5589096092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 -- 172.21.15.137:0/2407873123 shutdown_connections 2024-09-12T12:33:17.713 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 -- 172.21.15.137:0/2407873123 wait complete. 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 -- start start 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 Processor -- start 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x558909629a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909613440 0x558909626590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909626ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1a9d30 con 0x558909612360 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1ab1a0 con 0x558909613440 2024-09-12T12:33:17.714 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f007c1aa400 con 0x558909621350 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f0087247740 10 client.0 _ll_register_callbacks cb 0x7f0078009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f008196a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x558909629a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f008196a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x558909629a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33852/0 (socket says 172.21.15.137:33852) 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.678+0000 7f008196a640 1 -- 172.21.15.137:0/1807992266 learned_addr learned my addr 172.21.15.137:0/1807992266 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008216b640 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909613440 0x558909626590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008196a640 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909626ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008196a640 1 -- 172.21.15.137:0/1807992266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 msgr2=0x558909626ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008196a640 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909626ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.715 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008196a640 1 -- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909613440 msgr2=0x558909626590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008196a640 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909613440 0x558909626590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008196a640 1 -- 172.21.15.137:0/1807992266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558909627060 con 0x558909612360 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f008196a640 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x558909629a50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0078054da0 tx=0x7f00780314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3c42c14994b013 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f0065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f0065ffb640 1 -- 172.21.15.137:0/1807992266 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0078035070 con 0x558909612360 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f0065ffb640 1 -- 172.21.15.137:0/1807992266 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7f00780035f0 con 0x558909612360 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.679+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006c004470 con 0x558909612360 2024-09-12T12:33:17.716 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0065ffb640 1 -- 172.21.15.137:0/1807992266 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0078003990 con 0x558909612360 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 10 client.17889 fetch_fsmap learned FSMap version 55 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55890969ceb0 con 0x558909612360 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0065ffb640 1 -- 172.21.15.137:0/1807992266 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006c004470 con 0x558909612360 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0065ffb640 1 -- 172.21.15.137:0/1807992266 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 55) ==== 8350+0+0 (secure 0 0 0) 0x7f007804dbd0 con 0x558909612360 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 10 client.17889 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 10 client.17889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 20 client.17889 populate_metadata read hostname 'smithi137' 2024-09-12T12:33:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55890969d5a0 0x55890969f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 4 client.17889 mds_command: new command op to 24476 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-12T12:33:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 --> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558909579d70 con 0x55890969d5a0 2024-09-12T12:33:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.680+0000 7f008216b640 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55890969d5a0 0x55890969f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.681+0000 7f0065ffb640 10 client.17889 ms_handle_connect on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:17.681+0000 7f008216b640 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55890969d5a0 0x55890969f9c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa9592ecf9736b7 server_cookie=1dea643697097d3d in_seq=0 out_seq=0 2024-09-12T12:33:17.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:17 smithi189 ceph-mon[35475]: pgmap v638: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-12T12:33:17.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:17 smithi171 ceph-mon[35098]: pgmap v638: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-12T12:33:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.152+0000 7f0065ffb640 1 -- 172.21.15.137:0/1807992266 <== mds.0 v2:172.21.15.171:6838/1705495422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558909579d70 con 0x55890969d5a0 2024-09-12T12:33:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.152+0000 7f0065ffb640 10 client.17889 handle_command_reply: tid=0 2024-09-12T12:33:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 client.17889 shutdown 2024-09-12T12:33:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 msgr2=0x558909629a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x558909629a50 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0078054da0 tx=0x7f00780314e0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 shutdown_connections 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558909621350 0x558909626ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558909613440 0x558909626590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.171:6838/1705495422,v1:172.21.15.171:6839/1705495422] conn(0x55890969d5a0 0x55890969f9c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 --2- 172.21.15.137:0/1807992266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558909612360 0x558909629a50 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0065ffb640 0 client.17889 ms_handle_reset on v2:172.21.15.171:6838/1705495422 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 >> 172.21.15.137:0/1807992266 conn(0x55890960e470 msgr2=0x55890960c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 shutdown_connections 2024-09-12T12:33:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 1 -- 172.21.15.137:0/1807992266 wait complete. 2024-09-12T12:33:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f0087247740 20 client.17889 trim_cache size 0 max 16384 2024-09-12T12:33:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:{ 2024-09-12T12:33:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "message": "", 2024-09-12T12:33:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout: "return_code": 0 2024-09-12T12:33:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stdout:} 2024-09-12T12:33:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f00680a2da0 msgr2=0x7f00680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f00843cf640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f00680a2da0 0x7f00680a5260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f007409bca0 tx=0x7f0074098370 comp rx=0 tx=0).stop 2024-09-12T12:33:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.153+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c10c0b0 msgr2=0x7f007c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c10c0b0 0x7f007c19cc00 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f007400eb10 tx=0x7f007400c2d0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 shutdown_connections 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f00680a2da0 0x7f00680a5260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f007c10c0b0 0x7f007c19cc00 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f007c103600 0x7f007c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 --2- 172.21.15.137:0/4231206585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f007c102ce0 0x7f007c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 >> 172.21.15.137:0/4231206585 conn(0x7f007c077990 msgr2=0x7f007c0fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 shutdown_connections 2024-09-12T12:33:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi137.front.sepia.ceph.com.smithi137.stderr:2024-09-12T12:33:18.154+0000 7f00843cf640 1 -- 172.21.15.137:0/4231206585 wait complete. 2024-09-12T12:33:19.098 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T12:33:19.099 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-12T12:33:19.110 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:33:19.118 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-12T12:33:19.128 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-12T12:33:19.128 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T12:33:19.128 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:33:19.129 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-12T12:33:19.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-12T12:33:19.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-12T12:33:19.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.412+0000 7f581aff1640 1 -- 172.21.15.137:0/2925107314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5814101350 msgr2=0x7f581410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 --2- 172.21.15.137:0/2925107314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5814101350 0x7f581410d880 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f580c009930 tx=0x7f580c02a420 comp rx=0 tx=0).stop 2024-09-12T12:33:19.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- 172.21.15.137:0/2925107314 shutdown_connections 2024-09-12T12:33:19.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 --2- 172.21.15.137:0/2925107314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5814101350 0x7f581410d880 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 --2- 172.21.15.137:0/2925107314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58141009b0 0x7f5814100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 --2- 172.21.15.137:0/2925107314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58141069c0 0x7f5814106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- 172.21.15.137:0/2925107314 >> 172.21.15.137:0/2925107314 conn(0x7f5814077990 msgr2=0x7f58140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- 172.21.15.137:0/2925107314 shutdown_connections 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- 172.21.15.137:0/2925107314 wait complete. 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 Processor -- start 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- start start 2024-09-12T12:33:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58141009b0 0x7f5814197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5814101350 0x7f5814197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58141069c0 0x7f581419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5814110e00 con 0x7f58141069c0 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5814110c80 con 0x7f58141009b0 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.413+0000 7f581aff1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5814110f80 con 0x7f5814101350 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58141069c0 0x7f581419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58141069c0 0x7f581419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55112/0 (socket says 172.21.15.137:55112) 2024-09-12T12:33:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 -- 172.21.15.137:0/2921293029 learned_addr learned my addr 172.21.15.137:0/2921293029 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f5818d8d640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58141009b0 0x7f5814197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f5813fff640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5814101350 0x7f5814197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 -- 172.21.15.137:0/2921293029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5814101350 msgr2=0x7f5814197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5814101350 0x7f5814197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 -- 172.21.15.137:0/2921293029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58141009b0 msgr2=0x7f5814197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58141009b0 0x7f5814197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 -- 172.21.15.137:0/2921293029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581419c200 con 0x7f58141069c0 2024-09-12T12:33:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f5818d8d640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58141009b0 0x7f5814197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:19.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581958e640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58141069c0 0x7f581419bad0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f580c005ab0 tx=0x7f580c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:19.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f5811ffb640 1 -- 172.21.15.137:0/2921293029 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c046070 con 0x7f58141069c0 2024-09-12T12:33:19.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581419c490 con 0x7f58141069c0 2024-09-12T12:33:19.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f5811ffb640 1 -- 172.21.15.137:0/2921293029 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c004440 con 0x7f58141069c0 2024-09-12T12:33:19.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f5811ffb640 1 -- 172.21.15.137:0/2921293029 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c035050 con 0x7f58141069c0 2024-09-12T12:33:19.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.414+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58140698a0 con 0x7f58141069c0 2024-09-12T12:33:19.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.415+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5814100e10 con 0x7f58141069c0 2024-09-12T12:33:19.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.417+0000 7f5811ffb640 1 -- 172.21.15.137:0/2921293029 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f580c049070 con 0x7f58141069c0 2024-09-12T12:33:19.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.417+0000 7f5811ffb640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58000a2da0 0x7f58000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.417+0000 7f5818d8d640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58000a2da0 0x7f58000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.417+0000 7f5811ffb640 1 -- 172.21.15.137:0/2921293029 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7f580c0f10c0 con 0x7f58141069c0 2024-09-12T12:33:19.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.418+0000 7f5818d8d640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58000a2da0 0x7f58000a5260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5804009370 tx=0x7f5804009260 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:19.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.421+0000 7f5811ffb640 1 -- 172.21.15.137:0/2921293029 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f580c050080 con 0x7f58141069c0 2024-09-12T12:33:19.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.610+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5814103d70 con 0x7f58000a2da0 2024-09-12T12:33:19.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.621+0000 7f5811ffb640 1 -- 172.21.15.137:0/2921293029 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f5814103d70 con 0x7f58000a2da0 2024-09-12T12:33:19.622 INFO:teuthology.orchestra.run.smithi137.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-12T00:00:00", "created": "2024-09-12T12:13:14", "first": "2024-09-12T12:14:00", "last": "2024-09-12T12:33:00", "last_pruned": "2024-09-12T12:33:03", "created_count": 20, "pruned_count": 14, "active": true} 2024-09-12T12:33:19.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58000a2da0 msgr2=0x7f58000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58000a2da0 0x7f58000a5260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5804009370 tx=0x7f5804009260 comp rx=0 tx=0).stop 2024-09-12T12:33:19.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58141069c0 msgr2=0x7f581419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58141069c0 0x7f581419bad0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f580c005ab0 tx=0x7f580c004040 comp rx=0 tx=0).stop 2024-09-12T12:33:19.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 shutdown_connections 2024-09-12T12:33:19.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f58000a2da0 0x7f58000a5260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f58141069c0 0x7f581419bad0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5814101350 0x7f5814197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 --2- 172.21.15.137:0/2921293029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58141009b0 0x7f5814197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 >> 172.21.15.137:0/2921293029 conn(0x7f5814077990 msgr2=0x7f5814103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:19.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 shutdown_connections 2024-09-12T12:33:19.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.623+0000 7f581aff1640 1 -- 172.21.15.137:0/2921293029 wait complete. 2024-09-12T12:33:19.634 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:19 smithi137 ceph-mon[30950]: pgmap v639: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-12T12:33:19.635 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-12T12:33:19.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 -- 172.21.15.137:0/2726823085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181009c0 msgr2=0x7f0418100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 --2- 172.21.15.137:0/2726823085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181009c0 0x7f0418100e20 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0404009930 tx=0x7f040402a420 comp rx=0 tx=0).stop 2024-09-12T12:33:19.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 -- 172.21.15.137:0/2726823085 shutdown_connections 2024-09-12T12:33:19.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 --2- 172.21.15.137:0/2726823085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418101360 0x7f041810d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 --2- 172.21.15.137:0/2726823085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181009c0 0x7f0418100e20 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 --2- 172.21.15.137:0/2726823085 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04181069d0 0x7f0418106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 -- 172.21.15.137:0/2726823085 >> 172.21.15.137:0/2726823085 conn(0x7f04180fc700 msgr2=0x7f04180feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:19.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 -- 172.21.15.137:0/2726823085 shutdown_connections 2024-09-12T12:33:19.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.736+0000 7f041f103640 1 -- 172.21.15.137:0/2726823085 wait complete. 2024-09-12T12:33:19.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 Processor -- start 2024-09-12T12:33:19.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 -- start start 2024-09-12T12:33:19.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04181009c0 0x7f0418197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418101360 0x7f0418197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181069d0 0x7f041819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0418110e10 con 0x7f04181069d0 2024-09-12T12:33:19.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0418110c90 con 0x7f04181009c0 2024-09-12T12:33:19.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041f103640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0418110f90 con 0x7f0418101360 2024-09-12T12:33:19.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181069d0 0x7f041819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181069d0 0x7f041819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55118/0 (socket says 172.21.15.137:55118) 2024-09-12T12:33:19.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 -- 172.21.15.137:0/2955366236 learned_addr learned my addr 172.21.15.137:0/2955366236 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:19.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f0417fff640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418101360 0x7f0418197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041ce9f640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04181009c0 0x7f0418197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 -- 172.21.15.137:0/2955366236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418101360 msgr2=0x7f0418197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418101360 0x7f0418197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 -- 172.21.15.137:0/2955366236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04181009c0 msgr2=0x7f0418197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04181009c0 0x7f0418197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 -- 172.21.15.137:0/2955366236 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041819c230 con 0x7f04181069d0 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041ce9f640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04181009c0 0x7f0418197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.737+0000 7f041d6a0640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181069d0 0x7f041819bb00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f041000eb10 tx=0x7f041000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.738+0000 7f0415ffb640 1 -- 172.21.15.137:0/2955366236 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410004b20 con 0x7f04181069d0 2024-09-12T12:33:19.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.738+0000 7f0415ffb640 1 -- 172.21.15.137:0/2955366236 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0410004cc0 con 0x7f04181069d0 2024-09-12T12:33:19.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.738+0000 7f0415ffb640 1 -- 172.21.15.137:0/2955366236 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410010760 con 0x7f04181069d0 2024-09-12T12:33:19.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.738+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041819c520 con 0x7f04181069d0 2024-09-12T12:33:19.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.738+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a3db0 con 0x7f04181069d0 2024-09-12T12:33:19.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.739+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418100e20 con 0x7f04181069d0 2024-09-12T12:33:19.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.740+0000 7f0415ffb640 1 -- 172.21.15.137:0/2955366236 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0410005020 con 0x7f04181069d0 2024-09-12T12:33:19.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.740+0000 7f0415ffb640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04080a2e20 0x7f04080a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:19.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.741+0000 7f041ce9f640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04080a2e20 0x7f04080a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:19.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.741+0000 7f0415ffb640 1 -- 172.21.15.137:0/2955366236 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7f04100cfb20 con 0x7f04181069d0 2024-09-12T12:33:19.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.741+0000 7f041ce9f640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04080a2e20 0x7f04080a52e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f040c005e10 tx=0x7f040c005da0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:19.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.744+0000 7f0415ffb640 1 -- 172.21.15.137:0/2955366236 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f041009c740 con 0x7f04181069d0 2024-09-12T12:33:19.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:19 smithi189 ceph-mon[35475]: pgmap v639: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-12T12:33:19.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.933+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 --> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f04180fd270 con 0x7f04080a2e20 2024-09-12T12:33:19.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.944+0000 7f0415ffb640 1 -- 172.21.15.137:0/2955366236 <== mgr.15171 v2:172.21.15.137:6800/141828178 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f04180fd270 con 0x7f04080a2e20 2024-09-12T12:33:19.945 INFO:teuthology.orchestra.run.smithi137.stdout:/ 1m 6m3h 2024-09-12T12:33:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04080a2e20 msgr2=0x7f04080a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04080a2e20 0x7f04080a52e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f040c005e10 tx=0x7f040c005da0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181069d0 msgr2=0x7f041819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181069d0 0x7f041819bb00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f041000eb10 tx=0x7f041000c2d0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 shutdown_connections 2024-09-12T12:33:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f04080a2e20 0x7f04080a52e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f04181069d0 0x7f041819bb00 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0418101360 0x7f0418197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 --2- 172.21.15.137:0/2955366236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04181009c0 0x7f0418197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 >> 172.21.15.137:0/2955366236 conn(0x7f04180fc700 msgr2=0x7f04180fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 shutdown_connections 2024-09-12T12:33:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:19.946+0000 7f041f103640 1 -- 172.21.15.137:0/2955366236 wait complete. 2024-09-12T12:33:19.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:19 smithi171 ceph-mon[35098]: pgmap v639: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.8 MiB/s wr, 455 op/s 2024-09-12T12:33:19.958 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T12:33:19.969 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T12:33:20.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.067+0000 7fa6ba02e640 1 -- 172.21.15.137:0/2800361140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4102450 msgr2=0x7fa6b410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.067+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/2800361140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4102450 0x7fa6b410e980 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009cd0 tx=0x7fa6ac02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- 172.21.15.137:0/2800361140 shutdown_connections 2024-09-12T12:33:20.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/2800361140 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4102450 0x7fa6b410e980 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/2800361140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6b4101ab0 0x7fa6b4101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/2800361140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b4107ac0 0x7fa6b4107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- 172.21.15.137:0/2800361140 >> 172.21.15.137:0/2800361140 conn(0x7fa6b40fd7d0 msgr2=0x7fa6b40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:20.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- 172.21.15.137:0/2800361140 shutdown_connections 2024-09-12T12:33:20.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- 172.21.15.137:0/2800361140 wait complete. 2024-09-12T12:33:20.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 Processor -- start 2024-09-12T12:33:20.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- start start 2024-09-12T12:33:20.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4101ab0 0x7fa6b4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b4102450 0x7fa6b4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6b4107ac0 0x7fa6b419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4111f00 con 0x7fa6b4101ab0 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4111d80 con 0x7fa6b4107ac0 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6ba02e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4112080 con 0x7fa6b4102450 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4101ab0 0x7fa6b4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4101ab0 0x7fa6b4198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55136/0 (socket says 172.21.15.137:55136) 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.068+0000 7fa6b37fe640 1 -- 172.21.15.137:0/3070320211 learned_addr learned my addr 172.21.15.137:0/3070320211 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6abfff640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b4102450 0x7fa6b4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b3fff640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6b4107ac0 0x7fa6b419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b37fe640 1 -- 172.21.15.137:0/3070320211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b4102450 msgr2=0x7fa6b4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b37fe640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b4102450 0x7fa6b4198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b37fe640 1 -- 172.21.15.137:0/3070320211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6b4107ac0 msgr2=0x7fa6b419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b37fe640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6b4107ac0 0x7fa6b419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b37fe640 1 -- 172.21.15.137:0/3070320211 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b419d310 con 0x7fa6b4101ab0 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b3fff640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6b4107ac0 0x7fa6b419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b37fe640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4101ab0 0x7fa6b4198320 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d950 tx=0x7fa6a400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b17fa640 1 -- 172.21.15.137:0/3070320211 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4017030 con 0x7fa6b4101ab0 2024-09-12T12:33:20.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b17fa640 1 -- 172.21.15.137:0/3070320211 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a4004020 con 0x7fa6b4101ab0 2024-09-12T12:33:20.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6b17fa640 1 -- 172.21.15.137:0/3070320211 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4005650 con 0x7fa6b4101ab0 2024-09-12T12:33:20.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b419d600 con 0x7fa6b4101ab0 2024-09-12T12:33:20.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.069+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a4e90 con 0x7fa6b4101ab0 2024-09-12T12:33:20.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.070+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b4101f10 con 0x7fa6b4101ab0 2024-09-12T12:33:20.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.071+0000 7fa6b17fa640 1 -- 172.21.15.137:0/3070320211 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa6a40041d0 con 0x7fa6b4101ab0 2024-09-12T12:33:20.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.072+0000 7fa6b17fa640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa6a00a2d80 0x7fa6a00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.072+0000 7fa6abfff640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa6a00a2d80 0x7fa6a00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.072+0000 7fa6b17fa640 1 -- 172.21.15.137:0/3070320211 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7fa6a40cf3d0 con 0x7fa6b4101ab0 2024-09-12T12:33:20.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.072+0000 7fa6abfff640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa6a00a2d80 0x7fa6a00a5240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4100e60 tx=0x7fa69c00a2d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.075+0000 7fa6b17fa640 1 -- 172.21.15.137:0/3070320211 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6a400e350 con 0x7fa6b4101ab0 2024-09-12T12:33:20.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.246+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6b41a51c0 con 0x7fa6b4101ab0 2024-09-12T12:33:20.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.248+0000 7fa6b17fa640 1 -- 172.21.15.137:0/3070320211 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+23279 (secure 0 0 0) 0x7fa6a409c070 con 0x7fa6b4101ab0 2024-09-12T12:33:20.249 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:33:20.249 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":119,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:33:12.450028+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/4099842792":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/123979974":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/2550482799":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2039555152":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1789953576":"2024-09-13T12:13:13.350302+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1789953576":"2024-09-13T12:13:13.350302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa6a00a2d80 msgr2=0x7fa6a00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa6a00a2d80 0x7fa6a00a5240 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4100e60 tx=0x7fa69c00a2d0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4101ab0 msgr2=0x7fa6b4198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4101ab0 0x7fa6b4198320 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400d950 tx=0x7fa6a400dd50 comp rx=0 tx=0).stop 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 shutdown_connections 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa6a00a2d80 0x7fa6a00a5240 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa6b4107ac0 0x7fa6b419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6b4102450 0x7fa6b4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 --2- 172.21.15.137:0/3070320211 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6b4101ab0 0x7fa6b4198320 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 >> 172.21.15.137:0/3070320211 conn(0x7fa6b40fd7d0 msgr2=0x7fa6b40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:20.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 shutdown_connections 2024-09-12T12:33:20.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.250+0000 7fa6ba02e640 1 -- 172.21.15.137:0/3070320211 wait complete. 2024-09-12T12:33:20.263 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T12:06:29.335398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T12:11:08.337441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T12:11:10.709230+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 119, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T12:33:20.264 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T12:33:20.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 -- 172.21.15.137:0/2527900262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5481054e0 msgr2=0x7fe54810d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 --2- 172.21.15.137:0/2527900262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5481054e0 0x7fe54810d030 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe540009ca0 tx=0x7fe54002a8c0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 -- 172.21.15.137:0/2527900262 shutdown_connections 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 --2- 172.21.15.137:0/2527900262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5481054e0 0x7fe54810d030 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 --2- 172.21.15.137:0/2527900262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe548100d40 0x7fe548104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 --2- 172.21.15.137:0/2527900262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe548100390 0x7fe548100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 -- 172.21.15.137:0/2527900262 >> 172.21.15.137:0/2527900262 conn(0x7fe548077170 msgr2=0x7fe5480fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 -- 172.21.15.137:0/2527900262 shutdown_connections 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.366+0000 7fe54f023640 1 -- 172.21.15.137:0/2527900262 wait complete. 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 Processor -- start 2024-09-12T12:33:20.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 -- start start 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe548100390 0x7fe548197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5481054e0 0x7fe5481977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54819bb50 0x7fe54819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe548110510 con 0x7fe54819bb50 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe548110390 con 0x7fe548100390 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54f023640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe548110690 con 0x7fe5481054e0 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54d5c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54819bb50 0x7fe54819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54d5c0640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54819bb50 0x7fe54819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55154/0 (socket says 172.21.15.137:55154) 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54d5c0640 1 -- 172.21.15.137:0/884305808 learned_addr learned my addr 172.21.15.137:0/884305808 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe547fff640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5481054e0 0x7fe5481977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.367+0000 7fe54cdbf640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe548100390 0x7fe548197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54d5c0640 1 -- 172.21.15.137:0/884305808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5481054e0 msgr2=0x7fe5481977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54d5c0640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5481054e0 0x7fe5481977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54d5c0640 1 -- 172.21.15.137:0/884305808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe548100390 msgr2=0x7fe548197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54d5c0640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe548100390 0x7fe548197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54d5c0640 1 -- 172.21.15.137:0/884305808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54819c640 con 0x7fe54819bb50 2024-09-12T12:33:20.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54cdbf640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe548100390 0x7fe548197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:20.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54d5c0640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54819bb50 0x7fe54819c000 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe540031a20 tx=0x7fe540031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe545ffb640 1 -- 172.21.15.137:0/884305808 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe540003c30 con 0x7fe54819bb50 2024-09-12T12:33:20.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54819c8d0 con 0x7fe54819bb50 2024-09-12T12:33:20.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe545ffb640 1 -- 172.21.15.137:0/884305808 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe540003dd0 con 0x7fe54819bb50 2024-09-12T12:33:20.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe545ffb640 1 -- 172.21.15.137:0/884305808 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54002fcf0 con 0x7fe54819bb50 2024-09-12T12:33:20.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.368+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481a40f0 con 0x7fe54819bb50 2024-09-12T12:33:20.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.369+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe548100770 con 0x7fe54819bb50 2024-09-12T12:33:20.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.370+0000 7fe545ffb640 1 -- 172.21.15.137:0/884305808 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe540004050 con 0x7fe54819bb50 2024-09-12T12:33:20.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.371+0000 7fe545ffb640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5300a2d00 0x7fe5300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.371+0000 7fe54cdbf640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5300a2d00 0x7fe5300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.371+0000 7fe545ffb640 1 -- 172.21.15.137:0/884305808 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7fe5400f0690 con 0x7fe54819bb50 2024-09-12T12:33:20.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.371+0000 7fe54cdbf640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5300a2d00 0x7fe5300a51c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5380058c0 tx=0x7fe538005ec0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.375+0000 7fe545ffb640 1 -- 172.21.15.137:0/884305808 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5400bd450 con 0x7fe54819bb50 2024-09-12T12:33:20.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.544+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe548100ee0 con 0x7fe54819bb50 2024-09-12T12:33:20.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.544+0000 7fe545ffb640 1 -- 172.21.15.137:0/884305808 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fe54002fe90 con 0x7fe54819bb50 2024-09-12T12:33:20.545 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-09-12T12:33:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.546+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5300a2d00 msgr2=0x7fe5300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.546+0000 7fe54f023640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5300a2d00 0x7fe5300a51c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5380058c0 tx=0x7fe538005ec0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.546+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54819bb50 msgr2=0x7fe54819c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.546+0000 7fe54f023640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54819bb50 0x7fe54819c000 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe540031a20 tx=0x7fe540031a50 comp rx=0 tx=0).stop 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 shutdown_connections 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe5300a2d00 0x7fe5300a51c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe54819bb50 0x7fe54819c000 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5481054e0 0x7fe5481977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 --2- 172.21.15.137:0/884305808 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe548100390 0x7fe548197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 >> 172.21.15.137:0/884305808 conn(0x7fe548077170 msgr2=0x7fe5480fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:20.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 shutdown_connections 2024-09-12T12:33:20.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.547+0000 7fe54f023640 1 -- 172.21.15.137:0/884305808 wait complete. 2024-09-12T12:33:20.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:20 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3070320211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:33:20.559 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T12:33:20.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 -- 172.21.15.137:0/4108014219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1200fffd0 msgr2=0x7fe120100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 --2- 172.21.15.137:0/4108014219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1200fffd0 0x7fe120100430 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe114009900 tx=0x7fe11402a410 comp rx=0 tx=0).stop 2024-09-12T12:33:20.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 -- 172.21.15.137:0/4108014219 shutdown_connections 2024-09-12T12:33:20.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 --2- 172.21.15.137:0/4108014219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe120100970 0x7fe12010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 --2- 172.21.15.137:0/4108014219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1200fffd0 0x7fe120100430 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 --2- 172.21.15.137:0/4108014219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe120106ab0 0x7fe120106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 -- 172.21.15.137:0/4108014219 >> 172.21.15.137:0/4108014219 conn(0x7fe120077170 msgr2=0x7fe1200fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 -- 172.21.15.137:0/4108014219 shutdown_connections 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 -- 172.21.15.137:0/4108014219 wait complete. 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.661+0000 7fe125ce6640 1 Processor -- start 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe125ce6640 1 -- start start 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe125ce6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1200fffd0 0x7fe1201984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe125ce6640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe120100970 0x7fe120198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe125ce6640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe120106ab0 0x7fe12019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe125ce6640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe120111f80 con 0x7fe120100970 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe125ce6640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe120111e00 con 0x7fe120106ab0 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe125ce6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe120112100 con 0x7fe1200fffd0 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe120100970 0x7fe120198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe120100970 0x7fe120198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55182/0 (socket says 172.21.15.137:55182) 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 -- 172.21.15.137:0/515472396 learned_addr learned my addr 172.21.15.137:0/515472396 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11f7fe640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1200fffd0 0x7fe1201984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11ffff640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe120106ab0 0x7fe12019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 -- 172.21.15.137:0/515472396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1200fffd0 msgr2=0x7fe1201984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1200fffd0 0x7fe1201984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 -- 172.21.15.137:0/515472396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe120106ab0 msgr2=0x7fe12019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe120106ab0 0x7fe12019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 -- 172.21.15.137:0/515472396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12019d4e0 con 0x7fe120100970 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11ffff640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe120106ab0 0x7fe12019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.662+0000 7fe11effd640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe120100970 0x7fe120198a30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe114005ab0 tx=0x7fe1140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.663+0000 7fe11cff9640 1 -- 172.21.15.137:0/515472396 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11403f030 con 0x7fe120100970 2024-09-12T12:33:20.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.663+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12019d770 con 0x7fe120100970 2024-09-12T12:33:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.663+0000 7fe11cff9640 1 -- 172.21.15.137:0/515472396 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe114003ce0 con 0x7fe120100970 2024-09-12T12:33:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.663+0000 7fe11cff9640 1 -- 172.21.15.137:0/515472396 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114035070 con 0x7fe120100970 2024-09-12T12:33:20.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.663+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1201a5060 con 0x7fe120100970 2024-09-12T12:33:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.664+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe120100430 con 0x7fe120100970 2024-09-12T12:33:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.665+0000 7fe11cff9640 1 -- 172.21.15.137:0/515472396 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe114042030 con 0x7fe120100970 2024-09-12T12:33:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.666+0000 7fe11cff9640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe1080a2d00 0x7fe1080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.666+0000 7fe11cff9640 1 -- 172.21.15.137:0/515472396 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7fe1140f0050 con 0x7fe120100970 2024-09-12T12:33:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.666+0000 7fe11f7fe640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe1080a2d00 0x7fe1080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.666+0000 7fe11f7fe640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe1080a2d00 0x7fe1080a51c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe110005d90 tx=0x7fe110005d20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.670+0000 7fe11cff9640 1 -- 172.21.15.137:0/515472396 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1140bcc70 con 0x7fe120100970 2024-09-12T12:33:20.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.837+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe120109fb0 con 0x7fe120100970 2024-09-12T12:33:20.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.837+0000 7fe11cff9640 1 -- 172.21.15.137:0/515472396 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7fe1140dc070 con 0x7fe120100970 2024-09-12T12:33:20.838 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-09-12T12:33:20.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe1080a2d00 msgr2=0x7fe1080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe1080a2d00 0x7fe1080a51c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe110005d90 tx=0x7fe110005d20 comp rx=0 tx=0).stop 2024-09-12T12:33:20.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe120100970 msgr2=0x7fe120198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe120100970 0x7fe120198a30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fe114005ab0 tx=0x7fe1140042f0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 shutdown_connections 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fe1080a2d00 0x7fe1080a51c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe120106ab0 0x7fe12019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe120100970 0x7fe120198a30 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 --2- 172.21.15.137:0/515472396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1200fffd0 0x7fe1201984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 >> 172.21.15.137:0/515472396 conn(0x7fe120077170 msgr2=0x7fe120104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:20.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.839+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 shutdown_connections 2024-09-12T12:33:20.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.840+0000 7fe125ce6640 1 -- 172.21.15.137:0/515472396 wait complete. 2024-09-12T12:33:20.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:20 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3070320211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:33:20.851 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T12:33:20.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:20 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3070320211' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:33:20.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1732315167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 msgr2=0x7f3e08100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1732315167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08100df0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009930 tx=0x7f3dfc02a420 comp rx=0 tx=0).stop 2024-09-12T12:33:20.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1732315167 shutdown_connections 2024-09-12T12:33:20.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1732315167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e08101330 0x7f3e0810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1732315167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08100df0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1732315167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e081069a0 0x7f3e08106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1732315167 >> 172.21.15.137:0/1732315167 conn(0x7f3e080fc6b0 msgr2=0x7f3e080fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:20.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1732315167 shutdown_connections 2024-09-12T12:33:20.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.953+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1732315167 wait complete. 2024-09-12T12:33:20.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 Processor -- start 2024-09-12T12:33:20.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 -- start start 2024-09-12T12:33:20.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e08101330 0x7f3e08197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e081069a0 0x7f3e0819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08110de0 con 0x7f3e08100990 2024-09-12T12:33:20.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08110c60 con 0x7f3e08101330 2024-09-12T12:33:20.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e0dfc4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08110f60 con 0x7f3e081069a0 2024-09-12T12:33:20.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55204/0 (socket says 172.21.15.137:55204) 2024-09-12T12:33:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 -- 172.21.15.137:0/1684177156 learned_addr learned my addr 172.21.15.137:0/1684177156 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e06ffd640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e08101330 0x7f3e08197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 -- 172.21.15.137:0/1684177156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e081069a0 msgr2=0x7f3e0819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e081069a0 0x7f3e0819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 -- 172.21.15.137:0/1684177156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e08101330 msgr2=0x7f3e08197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e08101330 0x7f3e08197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 -- 172.21.15.137:0/1684177156 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0819c300 con 0x7f3e08100990 2024-09-12T12:33:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.954+0000 7f3e077fe640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08197310 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009900 tx=0x7f3df8009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.955+0000 7f3e04ff9640 1 -- 172.21.15.137:0/1684177156 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df800d030 con 0x7f3e08100990 2024-09-12T12:33:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.955+0000 7f3e04ff9640 1 -- 172.21.15.137:0/1684177156 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df800d1d0 con 0x7f3e08100990 2024-09-12T12:33:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.955+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0819c5f0 con 0x7f3e08100990 2024-09-12T12:33:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.955+0000 7f3e04ff9640 1 -- 172.21.15.137:0/1684177156 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8005260 con 0x7f3e08100990 2024-09-12T12:33:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.955+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0806a360 con 0x7f3e08100990 2024-09-12T12:33:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.956+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e08100df0 con 0x7f3e08100990 2024-09-12T12:33:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.957+0000 7f3e04ff9640 1 -- 172.21.15.137:0/1684177156 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3df80078f0 con 0x7f3e08100990 2024-09-12T12:33:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.958+0000 7f3e04ff9640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3df00a2d00 0x7f3df00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.958+0000 7f3e04ff9640 1 -- 172.21.15.137:0/1684177156 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7f3df80d0840 con 0x7f3e08100990 2024-09-12T12:33:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.958+0000 7f3e06ffd640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3df00a2d00 0x7f3df00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.958+0000 7f3e06ffd640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3df00a2d00 0x7f3df00a51c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:20.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:20.962+0000 7f3e04ff9640 1 -- 172.21.15.137:0/1684177156 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3df809d460 con 0x7f3e08100990 2024-09-12T12:33:21.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.129+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e0806ae40 con 0x7f3e08100990 2024-09-12T12:33:21.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.129+0000 7f3e04ff9640 1 -- 172.21.15.137:0/1684177156 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7f3df809cd60 con 0x7f3e08100990 2024-09-12T12:33:21.130 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-09-12T12:33:21.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.131+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3df00a2d00 msgr2=0x7f3df00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:21.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.131+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3df00a2d00 0x7f3df00a51c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc000c00 tx=0x7f3dfc005a20 comp rx=0 tx=0).stop 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.131+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 msgr2=0x7f3e08197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.131+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08197310 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009900 tx=0x7f3df8009dc0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 shutdown_connections 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f3df00a2d00 0x7f3df00a51c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e081069a0 0x7f3e0819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e08101330 0x7f3e08197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 --2- 172.21.15.137:0/1684177156 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3e08100990 0x7f3e08197310 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 >> 172.21.15.137:0/1684177156 conn(0x7f3e080fc6b0 msgr2=0x7f3e080fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:21.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 shutdown_connections 2024-09-12T12:33:21.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.132+0000 7f3e0dfc4640 1 -- 172.21.15.137:0/1684177156 wait complete. 2024-09-12T12:33:21.144 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T12:33:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.243+0000 7f8ed0265640 1 -- 172.21.15.137:0/4118584741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8100d50 msgr2=0x7f8ec8104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.243+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4118584741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8100d50 0x7f8ec8104d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8009900 tx=0x7f8eb802a410 comp rx=0 tx=0).stop 2024-09-12T12:33:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.243+0000 7f8ed0265640 1 -- 172.21.15.137:0/4118584741 shutdown_connections 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.243+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4118584741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81054f0 0x7f8ec810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.243+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4118584741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec8100d50 0x7f8ec8104d90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.243+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4118584741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec81003a0 0x7f8ec8100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.243+0000 7f8ed0265640 1 -- 172.21.15.137:0/4118584741 >> 172.21.15.137:0/4118584741 conn(0x7f8ec80771e0 msgr2=0x7f8ec80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 -- 172.21.15.137:0/4118584741 shutdown_connections 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 -- 172.21.15.137:0/4118584741 wait complete. 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 Processor -- start 2024-09-12T12:33:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 -- start start 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81003a0 0x7f8ec8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec81054f0 0x7f8ec8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec819cd00 0x7f8ec819d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec810fd20 con 0x7f8ec81003a0 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec810fba0 con 0x7f8ec81054f0 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ed0265640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec810fea0 con 0x7f8ec819cd00 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ece001640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81003a0 0x7f8ec8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ece001640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81003a0 0x7f8ec8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55232/0 (socket says 172.21.15.137:55232) 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.244+0000 7f8ece001640 1 -- 172.21.15.137:0/4030338575 learned_addr learned my addr 172.21.15.137:0/4030338575 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ece802640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec819cd00 0x7f8ec819d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ecd800640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec81054f0 0x7f8ec8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ece001640 1 -- 172.21.15.137:0/4030338575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec819cd00 msgr2=0x7f8ec819d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ece001640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec819cd00 0x7f8ec819d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ece001640 1 -- 172.21.15.137:0/4030338575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec81054f0 msgr2=0x7f8ec8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ece001640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec81054f0 0x7f8ec8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ece001640 1 -- 172.21.15.137:0/4030338575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec819d7f0 con 0x7f8ec81003a0 2024-09-12T12:33:21.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ece001640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81003a0 0x7f8ec8198440 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e9f0 tx=0x7f8ec400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:21.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ebeffd640 1 -- 172.21.15.137:0/4030338575 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400cd80 con 0x7f8ec81003a0 2024-09-12T12:33:21.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec819dae0 con 0x7f8ec81003a0 2024-09-12T12:33:21.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ebeffd640 1 -- 172.21.15.137:0/4030338575 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4005500 con 0x7f8ec81003a0 2024-09-12T12:33:21.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ebeffd640 1 -- 172.21.15.137:0/4030338575 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4010690 con 0x7f8ec81003a0 2024-09-12T12:33:21.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.245+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec81a51f0 con 0x7f8ec81003a0 2024-09-12T12:33:21.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.246+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec8100780 con 0x7f8ec81003a0 2024-09-12T12:33:21.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.247+0000 7f8ebeffd640 1 -- 172.21.15.137:0/4030338575 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8ec40046c0 con 0x7f8ec81003a0 2024-09-12T12:33:21.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.248+0000 7f8ebeffd640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8eb40a2d00 0x7f8eb40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:21.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.248+0000 7f8ecd800640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8eb40a2d00 0x7f8eb40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:21.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.248+0000 7f8ebeffd640 1 -- 172.21.15.137:0/4030338575 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8528+0+0 (secure 0 0 0) 0x7f8ec40cf990 con 0x7f8ec81003a0 2024-09-12T12:33:21.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.248+0000 7f8ecd800640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8eb40a2d00 0x7f8eb40a51c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006010 tx=0x7f8eb8005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:21.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.252+0000 7f8ebeffd640 1 -- 172.21.15.137:0/4030338575 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ec409c5b0 con 0x7f8ec81003a0 2024-09-12T12:33:21.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.598+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8ec8100d50 con 0x7f8ec81003a0 2024-09-12T12:33:21.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.606+0000 7f8ebeffd640 1 -- 172.21.15.137:0/4030338575 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740908 (secure 0 0 0) 0x7f8ec409beb0 con 0x7f8ec81003a0 2024-09-12T12:33:21.617 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:33:21.617 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:33:21.617 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 25, 2024-09-12T12:33:21.617 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.617 INFO:teuthology.orchestra.run.smithi137.stdout: "active_gid": 15171, 2024-09-12T12:33:21.617 INFO:teuthology.orchestra.run.smithi137.stdout: "active_name": "x", 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: "active_addrs": { 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:6800", 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 141828178 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:6801", 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 141828178 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: "active_addr": "172.21.15.137:6801/141828178", 2024-09-12T12:33:21.619 INFO:teuthology.orchestra.run.smithi137.stdout: "active_change": "2024-09-12T12:13:13.350601+0000", 2024-09-12T12:33:21.620 INFO:teuthology.orchestra.run.smithi137.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T12:33:21.620 INFO:teuthology.orchestra.run.smithi137.stdout: "available": true, 2024-09-12T12:33:21.620 INFO:teuthology.orchestra.run.smithi137.stdout: "standbys": [ 2024-09-12T12:33:21.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.620 INFO:teuthology.orchestra.run.smithi137.stdout: "gid": 15165, 2024-09-12T12:33:21.620 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "y", 2024-09-12T12:33:21.620 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr_features": 4540701547738038271, 2024-09-12T12:33:21.621 INFO:teuthology.orchestra.run.smithi137.stdout: "available_modules": [ 2024-09-12T12:33:21.621 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.621 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "alerts", 2024-09-12T12:33:21.621 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.621 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.626 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.626 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:21.626 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:21.626 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.626 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.626 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.626 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.628 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.629 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.630 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.631 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.631 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.631 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.631 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.631 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.631 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.631 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.632 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.633 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.635 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.636 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.636 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.636 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.636 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.636 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.636 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_destination": { 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_destination", 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.637 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email address to send alerts to", 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_from_name": { 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_from_name", 2024-09-12T12:33:21.638 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "Ceph", 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email From: name", 2024-09-12T12:33:21.639 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_host": { 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_host", 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.640 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP server", 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.641 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_password": { 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_password", 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.642 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Password to authenticate with", 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.643 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_port": { 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_port", 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "465", 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.644 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP port", 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_sender": { 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_sender", 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.645 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP envelope sender", 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.646 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.647 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.647 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.647 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_ssl": { 2024-09-12T12:33:21.647 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_ssl", 2024-09-12T12:33:21.648 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.648 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.648 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.648 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.648 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.648 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.648 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_user": { 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_user", 2024-09-12T12:33:21.649 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "User to authenticate as", 2024-09-12T12:33:21.650 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "balancer", 2024-09-12T12:33:21.651 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "active": { 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "active", 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.652 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_time": { 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_time", 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0000", 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.654 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_weekday": { 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_weekday", 2024-09-12T12:33:21.655 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T12:33:21.656 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_max_iterations": { 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_max_iterations", 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.657 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "25", 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "250", 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.658 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_metrics": { 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_metrics", 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.659 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_step": { 2024-09-12T12:33:21.660 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_step", 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.5", 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0.001", 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "0.999", 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.661 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "aggressiveness of optimization", 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: "end_time": { 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_time", 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.662 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2359", 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.663 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: "end_weekday": { 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_weekday", 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:21.664 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.665 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.666 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.666 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.666 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.666 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.666 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.666 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.667 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.668 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.668 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.668 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.668 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.668 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.668 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.669 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.669 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.669 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.669 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.669 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.669 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.669 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.670 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.671 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.672 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.673 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.674 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: "min_score": { 2024-09-12T12:33:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_score", 2024-09-12T12:33:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.675 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.676 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "upmap", 2024-09-12T12:33:21.677 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.678 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.678 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.678 INFO:teuthology.orchestra.run.smithi137.stdout: "crush-compat", 2024-09-12T12:33:21.678 INFO:teuthology.orchestra.run.smithi137.stdout: "none", 2024-09-12T12:33:21.678 INFO:teuthology.orchestra.run.smithi137.stdout: "read", 2024-09-12T12:33:21.678 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap", 2024-09-12T12:33:21.678 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap-read" 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Balancer mode", 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_ids": { 2024-09-12T12:33:21.679 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_ids", 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.680 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.681 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.682 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:21.682 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.682 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.682 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.682 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T12:33:21.682 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.682 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_deviation": { 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_deviation", 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:21.683 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:21.684 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.684 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.684 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T12:33:21.684 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T12:33:21.684 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.684 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.684 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_optimizations": { 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_optimizations", 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.685 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.686 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm", 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_down_multiplier": { 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_down_multiplier", 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:21.687 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.688 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_refresh_rate": { 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_refresh_rate", 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.689 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "20", 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_starting_port": { 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_starting_port", 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4721", 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.691 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.692 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T12:33:21.692 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.692 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.692 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.692 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_ptrace": { 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_ptrace", 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.693 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_interval": { 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_interval", 2024-09-12T12:33:21.694 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T12:33:21.695 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_memory_target_ratio": { 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.696 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.7", 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm_log_destination": { 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm_log_destination", 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.698 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "file", 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "file,syslog", 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "syslog" 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.699 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "cgroups_split": { 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cgroups_split", 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.700 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.701 INFO:teuthology.orchestra.run.smithi137.stdout: "config_checks_enabled": { 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_checks_enabled", 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.702 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: "config_dashboard": { 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_dashboard", 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.703 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.704 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_alertmanager": { 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_alertmanager", 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.705 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_base": { 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_base", 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.707 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Container image name, without the tag", 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_elasticsearch": { 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_elasticsearch", 2024-09-12T12:33:21.708 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "elasticsearch container image", 2024-09-12T12:33:21.709 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_grafana": { 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_grafana", 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.710 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.712 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_haproxy": { 2024-09-12T12:33:21.712 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_haproxy", 2024-09-12T12:33:21.712 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.712 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.712 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.712 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T12:33:21.712 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "HAproxy container image", 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_agent": { 2024-09-12T12:33:21.713 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_agent", 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.714 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger agent container image", 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_collector": { 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_collector", 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.715 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger collector container image", 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.716 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_query": { 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_query", 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger query container image", 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.718 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_keepalived": { 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_keepalived", 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.719 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Keepalived container image", 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_loki": { 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_loki", 2024-09-12T12:33:21.720 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Loki container image", 2024-09-12T12:33:21.721 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_node_exporter": { 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_node_exporter", 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.722 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_nvmeof": { 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_nvmeof", 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.724 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Nvme-of container image", 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_prometheus": { 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_prometheus", 2024-09-12T12:33:21.725 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:21.726 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_promtail": { 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_promtail", 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.727 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Promtail container image", 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.728 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_samba": { 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_samba", 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.729 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Samba/SMB container image", 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_snmp_gateway": { 2024-09-12T12:33:21.730 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_snmp_gateway", 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.731 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SNMP Gateway container image", 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "container_init": { 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_init", 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.732 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T12:33:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.733 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "daemon_cache_timeout": { 2024-09-12T12:33:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "daemon_cache_timeout", 2024-09-12T12:33:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.734 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "default_cephadm_command_timeout": { 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "900", 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.736 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "default_registry": { 2024-09-12T12:33:21.737 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_registry", 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io", 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.738 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "device_cache_timeout": { 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_cache_timeout", 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1800", 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache device inventory", 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.740 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "device_enhanced_scan": { 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_enhanced_scan", 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.741 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "facts_cache_timeout": { 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "facts_cache_timeout", 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.743 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache host facts data", 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "grafana_dashboards_path": { 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "grafana_dashboards_path", 2024-09-12T12:33:21.744 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T12:33:21.745 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "host_check_interval": { 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "host_check_interval", 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.746 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to perform a host check", 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.747 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "hw_monitoring": { 2024-09-12T12:33:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hw_monitoring", 2024-09-12T12:33:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.748 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.749 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "inventory_list_all": { 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "inventory_list_all", 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.750 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.751 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.752 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.753 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "log_refresh_metadata": { 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_refresh_metadata", 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.754 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T12:33:21.755 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.756 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.758 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.759 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.760 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.761 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.762 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.763 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "*", 2024-09-12T12:33:21.764 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.765 INFO:teuthology.orchestra.run.smithi137.stdout: "max_count_per_host": { 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_count_per_host", 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.766 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of daemons per service per host", 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "max_osd_draining_count": { 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_osd_draining_count", 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.767 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.768 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "migration_current": { 2024-09-12T12:33:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "migration_current", 2024-09-12T12:33:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.769 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "internal - do not modify", 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.770 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.771 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "root", 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm-package", 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "root" 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T12:33:21.772 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "oob_default_addr": { 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "oob_default_addr", 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.773 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "169.254.1.1", 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "prometheus_alerts_path": { 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus_alerts_path", 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.775 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T12:33:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_insecure": { 2024-09-12T12:33:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_insecure", 2024-09-12T12:33:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.777 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.778 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_password": { 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_password", 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.779 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_url": { 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_url", 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.780 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.781 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_username": { 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_username", 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.782 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "secure_monitoring_stack": { 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "secure_monitoring_stack", 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.784 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "service_discovery_port": { 2024-09-12T12:33:21.785 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "service_discovery_port", 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8765", 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "cephadm service discovery port", 2024-09-12T12:33:21.786 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "ssh_config_file": { 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssh_config_file", 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.787 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T12:33:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.788 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "use_agent": { 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_agent", 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.789 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "use_repo_digest": { 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_repo_digest", 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.791 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_failed_host_check": { 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_failed_host_check", 2024-09-12T12:33:21.792 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T12:33:21.793 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_daemons": { 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_daemons", 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.794 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.795 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_hosts": { 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_hosts", 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.796 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.797 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crash", 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.798 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.799 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.800 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.801 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.802 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.803 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.804 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.805 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.806 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "retain_interval": { 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retain_interval", 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.807 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "31536000", 2024-09-12T12:33:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T12:33:21.808 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_recent_interval": { 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_recent_interval", 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.809 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1209600", 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.810 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dashboard", 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.811 INFO:teuthology.orchestra.run.smithi137.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.812 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T12:33:21.813 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.814 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.815 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.816 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_ENABLED": { 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.817 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T12:33:21.818 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.819 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.820 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.821 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.822 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.824 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T12:33:21.825 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.826 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.827 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.828 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T12:33:21.829 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T12:33:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.832 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.833 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.834 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.836 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T12:33:21.837 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.838 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.839 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.840 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.841 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.842 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_URL": { 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_URL", 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.843 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T12:33:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T12:33:21.844 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.845 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.846 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.848 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T12:33:21.848 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T12:33:21.848 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.848 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.849 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.850 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.851 INFO:teuthology.orchestra.run.smithi137.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.852 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T12:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.854 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.855 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.856 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.857 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.858 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.859 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T12:33:21.860 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.861 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.862 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.864 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T12:33:21.865 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.866 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.867 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.868 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.869 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.871 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.872 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.872 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.872 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.872 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.872 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.872 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.873 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T12:33:21.874 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8", 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.875 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.876 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.876 INFO:teuthology.orchestra.run.smithi137.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T12:33:21.876 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T12:33:21.876 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.876 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.876 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "45", 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.877 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.878 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T12:33:21.879 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.880 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.881 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.882 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.883 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.885 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T12:33:21.886 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.887 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.888 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.889 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: "cross_origin_url": { 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cross_origin_url", 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.890 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "crt_file": { 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crt_file", 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.892 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: "debug": { 2024-09-12T12:33:21.893 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "debug", 2024-09-12T12:33:21.894 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.894 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.894 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.894 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.894 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.894 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.894 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable/disable debug options", 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: "jwt_token_ttl": { 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "jwt_token_ttl", 2024-09-12T12:33:21.895 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "28800", 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.896 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.897 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.898 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.899 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.900 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.901 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.902 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.903 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.903 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.903 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.903 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.903 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.903 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.904 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.905 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.905 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.905 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.905 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.905 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.905 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.905 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.906 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.907 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: "motd": { 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "motd", 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.908 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "The message of the day", 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.909 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect_resolve_ip_addr": { 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.910 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:21.911 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.912 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.913 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8080", 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.914 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.915 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl_server_port": { 2024-09-12T12:33:21.916 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl_server_port", 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8443", 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.917 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.918 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.918 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.918 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.918 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-09-12T12:33:21.918 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-09-12T12:33:21.918 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "redirect", 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.919 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect" 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-09-12T12:33:21.920 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.921 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: "url_prefix": { 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url_prefix", 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.922 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.923 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.924 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_monitoring": { 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_monitoring", 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.925 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "monitor device health metrics", 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.926 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.927 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.928 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.929 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.931 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.932 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.933 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.934 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: "mark_out_threshold": { 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mark_out_threshold", 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.935 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2419200", 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_name": { 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_name", 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "device_health_metrics", 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.937 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: "retention_period": { 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retention_period", 2024-09-12T12:33:21.938 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15552000", 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain device health metrics", 2024-09-12T12:33:21.939 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_frequency": { 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_frequency", 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.940 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.941 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-09-12T12:33:21.941 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.941 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.941 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.941 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T12:33:21.941 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.941 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: "self_heal": { 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "self_heal", 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.942 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.944 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_threshold": { 2024-09-12T12:33:21.945 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_threshold", 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7257600", 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.946 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "diskprediction_local", 2024-09-12T12:33:21.947 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.948 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.948 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.948 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.948 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.948 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.948 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.948 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.949 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.950 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.951 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.952 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.953 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.954 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.955 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.956 INFO:teuthology.orchestra.run.smithi137.stdout: "predict_interval": { 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predict_interval", 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.957 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: "predictor_model": { 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predictor_model", 2024-09-12T12:33:21.958 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "prophetstor", 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.959 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.960 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.961 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "influx", 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": false, 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "influxdb python module not found", 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.962 INFO:teuthology.orchestra.run.smithi137.stdout: "batch_size": { 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "batch_size", 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5000", 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.963 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.964 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T12:33:21.964 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.964 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.964 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.964 INFO:teuthology.orchestra.run.smithi137.stdout: "database": { 2024-09-12T12:33:21.964 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "database", 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "ceph", 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.965 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": { 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hostname", 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.966 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server hostname", 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.967 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.968 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "30", 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "5", 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.969 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.970 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.971 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.972 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.973 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.974 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.975 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.976 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.977 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: "password": { 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "password", 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.978 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "password of InfluxDB server user", 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.979 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: "port": { 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "port", 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.980 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8086", 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server port", 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.981 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "false", 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.982 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: "threads": { 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threads", 2024-09-12T12:33:21.983 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "32", 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T12:33:21.984 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "username": { 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "username", 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.985 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "username of InfluxDB server user", 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "verify_ssl": { 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "verify_ssl", 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "true", 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.987 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "insights", 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.989 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.991 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.992 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.993 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.994 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.995 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.996 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:21.997 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "iostat", 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:21.998 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:21.999 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.000 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.001 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.002 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.003 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.004 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.005 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.006 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "k8sevents", 2024-09-12T12:33:22.007 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_event_retention_days": { 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ceph_event_retention_days", 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.008 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7", 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.009 INFO:teuthology.orchestra.run.smithi137.stdout: "config_check_secs": { 2024-09-12T12:33:22.010 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_check_secs", 2024-09-12T12:33:22.010 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.010 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.010 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.010 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.010 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "10", 2024-09-12T12:33:22.010 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.011 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.012 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.013 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.014 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.016 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.017 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.018 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.019 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "localpool", 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.020 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "failure_domain": { 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "failure_domain", 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "host", 2024-09-12T12:33:22.021 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "failure domain for any created local pool", 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.022 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.023 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.024 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.025 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.027 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.028 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.029 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.030 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: "min_size": { 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_size", 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.031 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default min_size for any created local pool", 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.032 INFO:teuthology.orchestra.run.smithi137.stdout: "num_rep": { 2024-09-12T12:33:22.033 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "num_rep", 2024-09-12T12:33:22.033 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.033 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.033 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.033 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3", 2024-09-12T12:33:22.033 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.033 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default replica count for any created local pool", 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_num": { 2024-09-12T12:33:22.034 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_num", 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "128", 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.035 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "prefix": { 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prefix", 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.036 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name prefix for any created local pool", 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.037 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "subtree": { 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "subtree", 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "rack", 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.038 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mds_autoscaler", 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.040 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.041 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.042 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.043 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.044 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.045 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.046 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.047 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.048 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirroring", 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.049 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.050 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.051 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.052 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.053 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.054 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.055 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.056 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.057 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.057 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.057 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.057 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.057 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.057 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.057 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "nfs", 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.059 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.060 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.061 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.062 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.063 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.064 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.065 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.066 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.067 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.068 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "fail_fs": { 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "fail_fs", 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.069 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.070 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.071 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.072 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.073 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.075 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.076 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.077 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.078 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator": { 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.079 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: "rook", 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: "test_orchestrator" 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.080 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Orchestrator backend", 2024-09-12T12:33:22.081 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.081 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.081 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.081 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.081 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_perf_query", 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.082 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.083 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.085 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.086 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.087 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.088 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.089 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.090 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_support", 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.091 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.092 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.093 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.094 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.095 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.096 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.097 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.098 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.099 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_autoscaler", 2024-09-12T12:33:22.100 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.101 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.102 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.103 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.104 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.105 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.106 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.107 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.108 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.109 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.110 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: "threshold": { 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threshold", 2024-09-12T12:33:22.111 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1.0", 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "scaling threshold", 2024-09-12T12:33:22.112 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "progress", 2024-09-12T12:33:22.113 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_pg_recovery_event": { 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_pg_recovery_event", 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.114 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.115 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.115 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.115 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.115 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T12:33:22.115 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.115 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.115 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.116 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.117 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.118 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.119 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.120 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.121 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.122 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.123 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.124 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.125 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: "max_completed_events": { 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_completed_events", 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.126 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "50", 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "number of past completed events to remember", 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long the module is going to sleep", 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.129 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus", 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.130 INFO:teuthology.orchestra.run.smithi137.stdout: "cache": { 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cache", 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.131 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: "exclude_perf_counters": { 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "exclude_perf_counters", 2024-09-12T12:33:22.132 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T12:33:22.133 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.134 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.135 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.136 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.137 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.138 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.139 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.140 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.141 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools": { 2024-09-12T12:33:22.142 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools", 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.143 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.144 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "300", 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.145 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_interval": { 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_interval", 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15.0", 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.146 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:22.147 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.148 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.149 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "9283", 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.150 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: "stale_cache_strategy": { 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stale_cache_strategy", 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "log", 2024-09-12T12:33:22.151 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.152 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "default", 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.153 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: "default", 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: "error" 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.154 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-09-12T12:33:22.155 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.156 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.157 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.158 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.159 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.160 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.161 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.162 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.163 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.163 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.163 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.163 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.163 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.163 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.163 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.164 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.164 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.164 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.164 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.164 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.164 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.165 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.165 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.165 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.165 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.165 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.165 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_snap_create": { 2024-09-12T12:33:22.166 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_snap_create", 2024-09-12T12:33:22.167 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.167 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.167 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.167 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.167 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.167 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.167 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.168 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.168 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.168 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.168 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.168 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.168 INFO:teuthology.orchestra.run.smithi137.stdout: "mirror_snapshot_schedule": { 2024-09-12T12:33:22.169 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T12:33:22.169 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.169 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.169 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.169 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.169 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.169 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.170 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.170 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.170 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.170 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.170 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.170 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.170 INFO:teuthology.orchestra.run.smithi137.stdout: "trash_purge_schedule": { 2024-09-12T12:33:22.171 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "trash_purge_schedule", 2024-09-12T12:33:22.171 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.171 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.171 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.171 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.171 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.171 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.172 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.172 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.172 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.172 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.172 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.172 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.173 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.173 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.173 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.173 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "restful", 2024-09-12T12:33:22.173 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.173 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.173 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_auth": { 2024-09-12T12:33:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_auth", 2024-09-12T12:33:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.174 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.175 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.175 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.175 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.175 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.175 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.175 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.176 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.176 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-09-12T12:33:22.176 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-09-12T12:33:22.176 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.176 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.176 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.176 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.177 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.177 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.177 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.177 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.177 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.178 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.178 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.178 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.178 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.178 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.178 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.180 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.181 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.181 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.181 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.181 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.181 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.181 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.182 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.182 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.182 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.182 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.182 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.182 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.182 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.183 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.184 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.184 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.184 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.184 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.184 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.184 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.184 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.185 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.186 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.187 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.188 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:22.188 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:22.188 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.188 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.188 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.188 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.189 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.189 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.189 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.189 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.189 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.189 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.190 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.190 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:22.190 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:22.190 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.190 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.190 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.191 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.191 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.191 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.191 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.191 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.191 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.191 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rgw", 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.192 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.193 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.194 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.194 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.194 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.194 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.194 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.194 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.194 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.195 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.195 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.195 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.195 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.195 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.195 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.196 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.197 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.198 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.199 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.200 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.201 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.201 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.201 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.201 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.201 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.201 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.201 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.202 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.202 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.202 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.202 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rook", 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.203 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.204 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.206 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.207 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.208 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.208 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.208 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.208 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.208 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.208 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.209 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.210 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.210 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.210 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.210 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.210 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.210 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.211 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.212 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.212 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.212 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.212 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.212 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.212 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.212 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.213 INFO:teuthology.orchestra.run.smithi137.stdout: "storage_class": { 2024-09-12T12:33:22.213 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "storage_class", 2024-09-12T12:33:22.213 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.213 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.213 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.213 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "local", 2024-09-12T12:33:22.214 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.214 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.214 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.214 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T12:33:22.214 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.214 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.214 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "selftest", 2024-09-12T12:33:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.215 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.216 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.216 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.216 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.216 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.216 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.216 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.216 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.217 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.217 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.217 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.217 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.217 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.217 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.217 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.219 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.219 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.219 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.219 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.219 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.219 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.219 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.221 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.221 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.221 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.221 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.221 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.221 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.221 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.222 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.222 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.222 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.222 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.222 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.222 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.222 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.223 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.224 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.225 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.225 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.225 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.225 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.225 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.225 INFO:teuthology.orchestra.run.smithi137.stdout: "roption1": { 2024-09-12T12:33:22.226 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption1", 2024-09-12T12:33:22.226 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.226 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.226 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.226 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.226 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.227 INFO:teuthology.orchestra.run.smithi137.stdout: "roption2": { 2024-09-12T12:33:22.228 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption2", 2024-09-12T12:33:22.228 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.228 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.228 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.228 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "xyz", 2024-09-12T12:33:22.228 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.229 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.229 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.229 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.229 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.229 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.229 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.230 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption1": { 2024-09-12T12:33:22.230 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption1", 2024-09-12T12:33:22.230 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.230 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.230 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.230 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.231 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.231 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.231 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.231 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.231 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.231 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.231 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.232 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption2": { 2024-09-12T12:33:22.232 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption2", 2024-09-12T12:33:22.232 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.232 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.232 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.232 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.233 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.233 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.233 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.233 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.233 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.233 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.233 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.234 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption3": { 2024-09-12T12:33:22.234 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption3", 2024-09-12T12:33:22.234 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.234 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.234 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.234 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.235 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.235 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.235 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.235 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.235 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.235 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.235 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.236 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption4": { 2024-09-12T12:33:22.236 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption4", 2024-09-12T12:33:22.236 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.236 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.236 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.236 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.237 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.237 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.237 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.237 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.237 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.237 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.237 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption5": { 2024-09-12T12:33:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption5", 2024-09-12T12:33:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.238 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.240 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption6": { 2024-09-12T12:33:22.240 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption6", 2024-09-12T12:33:22.240 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.240 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.240 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.240 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.241 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.244 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.244 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.244 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.245 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.245 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.245 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.245 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption7": { 2024-09-12T12:33:22.245 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption7", 2024-09-12T12:33:22.246 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.246 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.246 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.246 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.246 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:22.246 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "42", 2024-09-12T12:33:22.246 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: "testkey": { 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testkey", 2024-09-12T12:33:22.247 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.248 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.248 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.248 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.248 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.248 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.248 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.248 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.249 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.249 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.249 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.249 INFO:teuthology.orchestra.run.smithi137.stdout: "testlkey": { 2024-09-12T12:33:22.249 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testlkey", 2024-09-12T12:33:22.250 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.250 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.250 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.250 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.250 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.250 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.250 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.251 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.251 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.251 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.251 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.251 INFO:teuthology.orchestra.run.smithi137.stdout: "testnewline": { 2024-09-12T12:33:22.251 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testnewline", 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.252 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.253 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.253 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.253 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.253 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.253 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.253 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.254 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-09-12T12:33:22.254 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.254 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.254 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.254 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_m_granularity": { 2024-09-12T12:33:22.254 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_m_granularity", 2024-09-12T12:33:22.254 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.255 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.255 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.255 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.255 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.255 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.255 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.255 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: "dump_on_update": { 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dump_on_update", 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.256 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.257 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.257 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.257 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.257 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.257 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.257 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "dump database to debug log on update", 2024-09-12T12:33:22.257 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.258 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.258 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.258 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.258 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.258 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.258 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.260 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.260 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.260 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.260 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.260 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.260 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.261 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.261 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.261 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.261 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.261 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.261 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.262 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.263 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.263 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.263 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.263 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.263 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.264 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.265 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.266 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.266 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.266 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.266 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.266 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.266 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.267 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.267 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.267 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.267 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.267 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.267 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.267 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.269 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.269 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stats", 2024-09-12T12:33:22.269 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.269 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.269 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.269 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.269 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.270 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.270 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.270 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.270 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.270 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.270 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.270 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.271 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.272 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.272 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.272 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.272 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.272 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.272 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.273 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.273 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.273 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.273 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.273 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.273 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.273 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.274 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.274 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.274 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.274 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.274 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.274 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.275 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.275 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.275 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.275 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.275 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.275 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.276 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.276 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.276 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.276 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.276 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.276 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.276 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.277 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.277 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.277 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.277 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.277 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.277 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.278 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.278 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.278 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.278 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.278 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.278 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.278 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.280 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "status", 2024-09-12T12:33:22.280 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.280 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.280 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.280 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.281 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.281 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.281 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.281 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.281 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.281 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.281 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.282 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.283 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.283 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.283 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.283 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.283 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.283 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.284 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.285 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.285 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.285 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.285 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.285 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.285 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.286 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.286 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.286 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.286 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.286 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.286 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.287 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.287 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.287 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.287 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.287 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.287 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.287 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.288 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.288 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.288 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.288 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.288 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.289 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.289 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.289 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.289 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.289 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.289 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.290 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.290 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.290 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.290 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.290 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.290 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.290 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.291 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.291 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.291 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telegraf", 2024-09-12T12:33:22.291 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.292 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.292 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.292 INFO:teuthology.orchestra.run.smithi137.stdout: "address": { 2024-09-12T12:33:22.292 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "address", 2024-09-12T12:33:22.292 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.292 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.293 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.293 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T12:33:22.293 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.293 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.293 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:22.294 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:22.295 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.295 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.295 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.295 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15", 2024-09-12T12:33:22.295 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.295 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.295 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.296 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.296 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.296 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.296 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.296 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.297 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.297 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.297 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.297 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.297 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.297 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.297 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.298 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.299 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.299 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.299 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.299 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.299 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.299 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.300 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.300 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.300 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.300 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.300 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.300 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.300 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.301 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.301 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.301 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.301 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.301 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.301 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.302 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.303 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.303 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.303 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.303 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.303 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.303 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.303 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.304 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.304 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.304 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.304 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.304 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.304 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.305 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.305 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.305 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.305 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.305 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.305 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.305 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.306 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telemetry", 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_basic": { 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_basic", 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.308 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.308 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.308 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.308 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.308 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.308 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T12:33:22.308 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_crash": { 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_crash", 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.309 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.310 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.310 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.310 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.310 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.310 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T12:33:22.310 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.311 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.311 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.311 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_device": { 2024-09-12T12:33:22.311 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_device", 2024-09-12T12:33:22.311 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.311 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.312 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.312 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.312 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.312 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.312 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.312 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T12:33:22.313 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.313 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.313 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.313 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_ident": { 2024-09-12T12:33:22.313 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_ident", 2024-09-12T12:33:22.313 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.314 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.314 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.314 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.314 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.314 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.314 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.314 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T12:33:22.315 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.315 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.315 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.315 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_perf": { 2024-09-12T12:33:22.315 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_perf", 2024-09-12T12:33:22.315 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T12:33:22.316 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.317 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.317 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.317 INFO:teuthology.orchestra.run.smithi137.stdout: "contact": { 2024-09-12T12:33:22.317 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "contact", 2024-09-12T12:33:22.317 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.317 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.318 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.318 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.318 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.318 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.318 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.318 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.318 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.319 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.319 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.319 INFO:teuthology.orchestra.run.smithi137.stdout: "description": { 2024-09-12T12:33:22.319 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "description", 2024-09-12T12:33:22.319 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.319 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.320 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.321 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.321 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.321 INFO:teuthology.orchestra.run.smithi137.stdout: "device_url": { 2024-09-12T12:33:22.321 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_url", 2024-09-12T12:33:22.321 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.321 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.322 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.322 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T12:33:22.322 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.322 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.322 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.322 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.322 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.323 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.324 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.324 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.324 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.324 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.324 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.324 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.324 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.325 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.325 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:22.325 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:22.325 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.325 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.325 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.326 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "24", 2024-09-12T12:33:22.326 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "8", 2024-09-12T12:33:22.326 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.326 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.326 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.326 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.326 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: "last_opt_revision": { 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "last_opt_revision", 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.327 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1", 2024-09-12T12:33:22.328 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.328 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.328 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.328 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.328 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.328 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.328 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard": { 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard", 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.329 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.330 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.330 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.330 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.330 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.330 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.330 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.331 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard_description": { 2024-09-12T12:33:22.331 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard_description", 2024-09-12T12:33:22.331 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.331 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.331 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.331 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.332 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.332 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.332 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.332 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.332 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.333 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.333 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.333 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.333 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.333 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.333 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.334 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.334 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.334 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.334 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.334 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.334 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.334 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.335 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.335 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.335 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.335 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.335 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.335 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.336 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.336 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.336 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.336 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.336 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.337 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.337 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.337 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.337 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.337 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.337 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.337 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.338 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.338 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.338 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.338 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.338 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.338 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.340 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.340 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.340 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.340 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.340 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.340 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.341 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.341 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.341 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.341 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.341 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.341 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.342 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.342 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.342 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.342 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.342 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.342 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.342 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.343 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.343 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.343 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.343 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.343 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.343 INFO:teuthology.orchestra.run.smithi137.stdout: "organization": { 2024-09-12T12:33:22.344 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "organization", 2024-09-12T12:33:22.344 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.344 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.344 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.344 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.344 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.344 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.345 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.345 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.345 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.345 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.345 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.345 INFO:teuthology.orchestra.run.smithi137.stdout: "proxy": { 2024-09-12T12:33:22.346 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "proxy", 2024-09-12T12:33:22.346 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.346 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.346 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.346 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.346 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.346 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.347 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.347 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.347 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.347 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.347 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.347 INFO:teuthology.orchestra.run.smithi137.stdout: "url": { 2024-09-12T12:33:22.348 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url", 2024-09-12T12:33:22.348 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.348 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.348 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.348 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T12:33:22.348 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.348 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.349 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.349 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.349 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.349 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.349 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.349 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.349 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.350 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "test_orchestrator", 2024-09-12T12:33:22.350 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.350 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.350 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.350 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.351 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.351 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.351 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.351 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.351 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.351 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.351 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.352 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.352 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.352 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.352 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.352 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.352 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.352 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.353 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.354 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.354 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.354 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.354 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.354 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.354 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.354 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.355 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.355 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.355 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.355 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.355 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.355 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.356 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.356 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.356 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.356 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.356 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.356 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.356 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.357 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.358 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.358 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.358 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.358 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.358 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.358 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.359 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.359 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.359 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.359 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.359 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.359 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.359 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.360 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.360 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.360 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.360 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.360 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.360 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.361 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.361 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-09-12T12:33:22.361 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.361 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.361 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.361 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.361 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.362 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.362 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.362 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.362 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.362 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.362 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.363 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.364 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.364 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.364 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.364 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.364 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.365 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.365 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.365 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.365 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.365 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.365 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.366 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.366 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.366 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.366 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.366 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.366 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.366 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.367 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.367 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.367 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.367 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.367 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.367 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.367 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.368 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.369 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.369 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.369 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.369 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.369 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.369 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.369 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.370 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.370 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.370 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.370 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.370 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.370 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.370 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.371 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.371 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.371 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.371 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.371 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.371 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.371 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_clones": { 2024-09-12T12:33:22.372 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_clones", 2024-09-12T12:33:22.372 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.372 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.372 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.372 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4", 2024-09-12T12:33:22.372 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.373 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.373 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.373 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T12:33:22.373 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.373 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.373 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.374 INFO:teuthology.orchestra.run.smithi137.stdout: "periodic_async_work": { 2024-09-12T12:33:22.374 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "periodic_async_work", 2024-09-12T12:33:22.374 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.374 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.374 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.374 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.375 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.375 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.375 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.375 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Periodically check for async work", 2024-09-12T12:33:22.375 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.376 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.376 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.376 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_delay": { 2024-09-12T12:33:22.376 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_delay", 2024-09-12T12:33:22.376 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.377 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.377 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.377 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:22.377 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.377 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.377 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.377 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T12:33:22.378 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.378 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.378 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.378 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_no_wait": { 2024-09-12T12:33:22.378 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T12:33:22.379 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.379 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.379 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.379 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.379 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.379 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.379 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.380 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T12:33:22.380 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.380 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.380 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.380 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.380 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.381 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.381 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix", 2024-09-12T12:33:22.381 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.381 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.381 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.381 INFO:teuthology.orchestra.run.smithi137.stdout: "discovery_interval": { 2024-09-12T12:33:22.382 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "discovery_interval", 2024-09-12T12:33:22.382 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:22.382 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.382 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.382 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "100", 2024-09-12T12:33:22.382 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.382 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.383 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.383 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.383 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.383 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.383 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.384 INFO:teuthology.orchestra.run.smithi137.stdout: "identifier": { 2024-09-12T12:33:22.384 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "identifier", 2024-09-12T12:33:22.384 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.384 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.384 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.384 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.384 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.385 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:22.386 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:22.386 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.386 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.386 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.386 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:22.386 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.386 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.387 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.387 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.387 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.387 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.387 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.387 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.387 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.388 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.388 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.388 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.388 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.388 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.388 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.388 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.389 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.389 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.389 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.389 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.389 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.389 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.390 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.390 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.390 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.390 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.390 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.390 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.390 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.391 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.391 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.391 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.391 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.391 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.391 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.391 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.392 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.392 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.392 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.392 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.392 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.392 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.392 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.393 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.393 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.393 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.393 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.393 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.393 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.393 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.394 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.395 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.396 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.396 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.396 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.396 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.396 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.396 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.396 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.397 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.397 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.397 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.397 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.397 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.397 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_host": { 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_host", 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.398 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.399 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.399 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.399 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.399 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.399 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.399 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_port": { 2024-09-12T12:33:22.399 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_port", 2024-09-12T12:33:22.400 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.400 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.400 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.400 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10051", 2024-09-12T12:33:22.400 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.400 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.400 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.401 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.401 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.401 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.401 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.401 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.401 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_sender": { 2024-09-12T12:33:22.401 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_sender", 2024-09-12T12:33:22.402 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.402 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.402 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.402 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T12:33:22.402 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.402 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.402 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.403 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.403 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.403 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.403 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.403 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.403 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.403 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.404 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:22.404 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.404 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.404 INFO:teuthology.orchestra.run.smithi137.stdout: "gid": 15168, 2024-09-12T12:33:22.404 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "z", 2024-09-12T12:33:22.404 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr_features": 4540701547738038271, 2024-09-12T12:33:22.404 INFO:teuthology.orchestra.run.smithi137.stdout: "available_modules": [ 2024-09-12T12:33:22.405 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.405 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "alerts", 2024-09-12T12:33:22.405 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.405 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.405 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.405 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:22.405 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:22.406 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.406 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.406 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.406 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:22.406 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.406 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.406 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.407 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T12:33:22.407 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.407 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.407 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.407 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.407 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.407 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.408 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.408 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.408 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.408 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.408 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.408 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.409 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.409 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.409 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.409 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.409 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.409 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.410 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.410 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.410 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.410 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.410 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.410 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.411 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.411 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.411 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.411 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.411 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.411 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.411 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.412 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.412 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.412 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.412 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.412 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.413 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.414 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.414 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.414 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.414 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.414 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.414 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.414 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.415 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.415 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.415 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.415 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.415 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.415 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.415 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.416 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.416 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.416 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.416 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.416 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.417 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.417 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.417 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.417 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.417 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.417 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.417 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.418 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.418 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_destination": { 2024-09-12T12:33:22.418 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_destination", 2024-09-12T12:33:22.418 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.418 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.418 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.419 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.419 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.419 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.419 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.419 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email address to send alerts to", 2024-09-12T12:33:22.419 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.419 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_from_name": { 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_from_name", 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.420 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "Ceph", 2024-09-12T12:33:22.421 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.421 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.421 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.421 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email From: name", 2024-09-12T12:33:22.421 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.421 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.421 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.422 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_host": { 2024-09-12T12:33:22.422 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_host", 2024-09-12T12:33:22.422 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.422 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.422 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.422 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.423 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.423 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.423 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.423 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP server", 2024-09-12T12:33:22.423 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.423 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.424 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.424 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_password": { 2024-09-12T12:33:22.424 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_password", 2024-09-12T12:33:22.424 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.424 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.424 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.425 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.425 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.425 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.425 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.425 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Password to authenticate with", 2024-09-12T12:33:22.425 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.425 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.426 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.426 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_port": { 2024-09-12T12:33:22.426 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_port", 2024-09-12T12:33:22.426 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.426 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.426 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.427 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "465", 2024-09-12T12:33:22.427 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.427 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.427 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.427 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP port", 2024-09-12T12:33:22.427 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.427 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.428 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.428 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_sender": { 2024-09-12T12:33:22.428 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_sender", 2024-09-12T12:33:22.428 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.428 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.429 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.429 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.429 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.429 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.429 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.429 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP envelope sender", 2024-09-12T12:33:22.429 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.430 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.430 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.430 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_ssl": { 2024-09-12T12:33:22.430 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_ssl", 2024-09-12T12:33:22.430 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.430 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T12:33:22.431 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.432 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.432 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.432 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_user": { 2024-09-12T12:33:22.432 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_user", 2024-09-12T12:33:22.432 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.432 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.433 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.433 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.433 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.433 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.433 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.433 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "User to authenticate as", 2024-09-12T12:33:22.433 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.434 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.434 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.434 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.434 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.434 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "balancer", 2024-09-12T12:33:22.435 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.435 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.435 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.435 INFO:teuthology.orchestra.run.smithi137.stdout: "active": { 2024-09-12T12:33:22.435 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "active", 2024-09-12T12:33:22.435 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.435 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.436 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.436 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.436 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.436 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.436 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.436 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T12:33:22.437 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.437 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.437 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.437 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_time": { 2024-09-12T12:33:22.437 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_time", 2024-09-12T12:33:22.437 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0000", 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T12:33:22.438 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T12:33:22.439 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.439 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.439 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_weekday": { 2024-09-12T12:33:22.439 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_weekday", 2024-09-12T12:33:22.439 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:22.440 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.440 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.440 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:22.440 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-09-12T12:33:22.440 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-09-12T12:33:22.440 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.440 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T12:33:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T12:33:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_max_iterations": { 2024-09-12T12:33:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_max_iterations", 2024-09-12T12:33:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:22.442 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.442 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.442 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "25", 2024-09-12T12:33:22.442 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:22.442 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "250", 2024-09-12T12:33:22.442 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.443 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T12:33:22.443 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.443 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.443 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.443 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_metrics": { 2024-09-12T12:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_metrics", 2024-09-12T12:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T12:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T12:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T12:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_step": { 2024-09-12T12:33:22.446 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_step", 2024-09-12T12:33:22.446 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.446 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.446 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.446 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.5", 2024-09-12T12:33:22.446 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0.001", 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "0.999", 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "aggressiveness of optimization", 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.447 INFO:teuthology.orchestra.run.smithi137.stdout: "end_time": { 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_time", 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2359", 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.448 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.449 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T12:33:22.449 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T12:33:22.449 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.452 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.453 INFO:teuthology.orchestra.run.smithi137.stdout: "end_weekday": { 2024-09-12T12:33:22.453 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_weekday", 2024-09-12T12:33:22.453 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:22.453 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.453 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.454 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.455 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.456 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.456 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.456 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.456 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.456 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.456 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.457 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.457 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.457 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.457 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.457 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.457 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.457 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.458 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.459 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.459 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.459 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.459 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.459 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.459 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.460 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.460 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.460 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.460 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.460 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.460 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.461 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.461 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.461 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.461 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.461 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.461 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.462 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.463 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.463 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.463 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.463 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.463 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.463 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.464 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.464 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.464 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.464 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.464 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.464 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.464 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.465 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.465 INFO:teuthology.orchestra.run.smithi137.stdout: "min_score": { 2024-09-12T12:33:22.465 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_score", 2024-09-12T12:33:22.465 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.465 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.465 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.466 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:22.466 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.466 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.466 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.466 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T12:33:22.466 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.466 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.467 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.467 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-09-12T12:33:22.467 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-09-12T12:33:22.467 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.467 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "upmap", 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "crush-compat", 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "none", 2024-09-12T12:33:22.468 INFO:teuthology.orchestra.run.smithi137.stdout: "read", 2024-09-12T12:33:22.469 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap", 2024-09-12T12:33:22.469 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap-read" 2024-09-12T12:33:22.469 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.469 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Balancer mode", 2024-09-12T12:33:22.469 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.469 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.469 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.470 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_ids": { 2024-09-12T12:33:22.470 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_ids", 2024-09-12T12:33:22.470 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.470 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.470 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T12:33:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.471 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.472 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.472 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:22.472 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:22.472 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.472 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.473 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.474 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.474 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_deviation": { 2024-09-12T12:33:22.474 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_deviation", 2024-09-12T12:33:22.474 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.474 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.474 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.475 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:22.475 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:22.475 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.475 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.475 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T12:33:22.475 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T12:33:22.475 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.476 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.476 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_optimizations": { 2024-09-12T12:33:22.476 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_optimizations", 2024-09-12T12:33:22.476 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:22.476 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.477 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.478 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.478 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm", 2024-09-12T12:33:22.478 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.479 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.479 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.479 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_down_multiplier": { 2024-09-12T12:33:22.479 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_down_multiplier", 2024-09-12T12:33:22.479 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.479 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.479 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.480 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-09-12T12:33:22.480 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.480 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.480 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.480 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T12:33:22.480 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_refresh_rate": { 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_refresh_rate", 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.481 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.482 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "20", 2024-09-12T12:33:22.482 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.482 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.482 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.482 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T12:33:22.482 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.482 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.483 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.483 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_starting_port": { 2024-09-12T12:33:22.483 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_starting_port", 2024-09-12T12:33:22.483 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.483 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.483 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.484 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4721", 2024-09-12T12:33:22.484 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.484 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.484 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.484 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T12:33:22.484 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.485 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.485 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.485 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_ptrace": { 2024-09-12T12:33:22.485 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_ptrace", 2024-09-12T12:33:22.485 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.485 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.486 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.486 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.486 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.486 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.486 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.486 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T12:33:22.487 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T12:33:22.487 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.487 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.487 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_interval": { 2024-09-12T12:33:22.487 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_interval", 2024-09-12T12:33:22.487 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.488 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.488 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.488 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:22.488 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.488 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.488 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.488 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T12:33:22.489 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.489 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.489 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.489 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_memory_target_ratio": { 2024-09-12T12:33:22.489 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T12:33:22.489 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.490 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.490 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.490 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.7", 2024-09-12T12:33:22.490 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.490 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.490 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.490 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T12:33:22.491 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.491 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.491 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.491 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm_log_destination": { 2024-09-12T12:33:22.491 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm_log_destination", 2024-09-12T12:33:22.491 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.492 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.492 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.492 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.492 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.492 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.492 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.492 INFO:teuthology.orchestra.run.smithi137.stdout: "file", 2024-09-12T12:33:22.493 INFO:teuthology.orchestra.run.smithi137.stdout: "file,syslog", 2024-09-12T12:33:22.493 INFO:teuthology.orchestra.run.smithi137.stdout: "syslog" 2024-09-12T12:33:22.493 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.493 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T12:33:22.493 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.493 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.493 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.494 INFO:teuthology.orchestra.run.smithi137.stdout: "cgroups_split": { 2024-09-12T12:33:22.494 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cgroups_split", 2024-09-12T12:33:22.494 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.494 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.494 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.494 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.495 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.495 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.495 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.495 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T12:33:22.495 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.495 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.495 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.496 INFO:teuthology.orchestra.run.smithi137.stdout: "config_checks_enabled": { 2024-09-12T12:33:22.496 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_checks_enabled", 2024-09-12T12:33:22.496 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.496 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.496 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.496 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.497 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.497 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.497 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.497 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T12:33:22.497 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.497 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.497 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.498 INFO:teuthology.orchestra.run.smithi137.stdout: "config_dashboard": { 2024-09-12T12:33:22.498 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_dashboard", 2024-09-12T12:33:22.498 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.498 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.498 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.498 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.499 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.499 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.499 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.499 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T12:33:22.499 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.499 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.499 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.500 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.500 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_alertmanager": { 2024-09-12T12:33:22.500 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_alertmanager", 2024-09-12T12:33:22.500 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.500 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.500 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.501 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T12:33:22.501 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.501 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.501 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.501 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:22.501 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.501 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_base": { 2024-09-12T12:33:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_base", 2024-09-12T12:33:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.502 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.503 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T12:33:22.503 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.503 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.503 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.503 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Container image name, without the tag", 2024-09-12T12:33:22.503 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.503 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_elasticsearch": { 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_elasticsearch", 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.504 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T12:33:22.505 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.505 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.505 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.505 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "elasticsearch container image", 2024-09-12T12:33:22.505 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.505 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.506 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.506 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_grafana": { 2024-09-12T12:33:22.506 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_grafana", 2024-09-12T12:33:22.506 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.506 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.506 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.507 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T12:33:22.507 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.507 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.507 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.507 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:22.507 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.507 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.508 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.508 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.508 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_haproxy": { 2024-09-12T12:33:22.508 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_haproxy", 2024-09-12T12:33:22.508 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.508 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.509 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.509 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T12:33:22.509 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.509 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.509 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.509 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "HAproxy container image", 2024-09-12T12:33:22.509 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.510 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.510 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.510 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.510 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_agent": { 2024-09-12T12:33:22.510 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_agent", 2024-09-12T12:33:22.510 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.511 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.511 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.511 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T12:33:22.511 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.511 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.511 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.511 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger agent container image", 2024-09-12T12:33:22.512 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.512 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.512 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.512 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_collector": { 2024-09-12T12:33:22.512 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_collector", 2024-09-12T12:33:22.512 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.513 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.513 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.513 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T12:33:22.513 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.513 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.513 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.514 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger collector container image", 2024-09-12T12:33:22.514 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.514 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.514 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.514 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.514 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_query": { 2024-09-12T12:33:22.514 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_query", 2024-09-12T12:33:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T12:33:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger query container image", 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_keepalived": { 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_keepalived", 2024-09-12T12:33:22.516 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T12:33:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.517 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Keepalived container image", 2024-09-12T12:33:22.518 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.518 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.518 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.518 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_loki": { 2024-09-12T12:33:22.518 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_loki", 2024-09-12T12:33:22.518 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.519 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.519 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.519 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T12:33:22.519 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.519 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.519 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.519 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Loki container image", 2024-09-12T12:33:22.520 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.520 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.520 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.520 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_node_exporter": { 2024-09-12T12:33:22.520 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_node_exporter", 2024-09-12T12:33:22.520 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.521 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.521 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.521 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T12:33:22.521 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.521 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.521 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.521 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:22.522 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.522 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.522 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.522 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_nvmeof": { 2024-09-12T12:33:22.522 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_nvmeof", 2024-09-12T12:33:22.522 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.523 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.523 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.523 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T12:33:22.523 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.523 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.523 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.523 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Nvme-of container image", 2024-09-12T12:33:22.524 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.524 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.524 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.524 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_prometheus": { 2024-09-12T12:33:22.524 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_prometheus", 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.525 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:22.526 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.526 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.526 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.526 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_promtail": { 2024-09-12T12:33:22.526 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_promtail", 2024-09-12T12:33:22.527 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.527 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.527 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.527 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T12:33:22.527 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.527 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.527 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.528 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Promtail container image", 2024-09-12T12:33:22.528 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.528 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.528 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.528 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_samba": { 2024-09-12T12:33:22.528 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_samba", 2024-09-12T12:33:22.529 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.529 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.529 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.529 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T12:33:22.529 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.529 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.529 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Samba/SMB container image", 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_snmp_gateway": { 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_snmp_gateway", 2024-09-12T12:33:22.530 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.531 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.531 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.531 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T12:33:22.531 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.531 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.531 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.532 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SNMP Gateway container image", 2024-09-12T12:33:22.532 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.532 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.532 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.532 INFO:teuthology.orchestra.run.smithi137.stdout: "container_init": { 2024-09-12T12:33:22.532 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_init", 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.533 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T12:33:22.534 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.534 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.534 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.534 INFO:teuthology.orchestra.run.smithi137.stdout: "daemon_cache_timeout": { 2024-09-12T12:33:22.534 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "daemon_cache_timeout", 2024-09-12T12:33:22.534 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.535 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.535 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.535 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:22.535 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.535 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.535 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.536 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T12:33:22.536 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.536 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.536 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.536 INFO:teuthology.orchestra.run.smithi137.stdout: "default_cephadm_command_timeout": { 2024-09-12T12:33:22.536 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T12:33:22.537 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.537 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.537 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.537 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "900", 2024-09-12T12:33:22.537 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.537 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.537 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: "default_registry": { 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_registry", 2024-09-12T12:33:22.538 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io", 2024-09-12T12:33:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T12:33:22.540 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.540 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.540 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.540 INFO:teuthology.orchestra.run.smithi137.stdout: "device_cache_timeout": { 2024-09-12T12:33:22.540 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_cache_timeout", 2024-09-12T12:33:22.541 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.541 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.541 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.541 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1800", 2024-09-12T12:33:22.541 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.541 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.541 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.542 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache device inventory", 2024-09-12T12:33:22.542 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.542 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.542 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.542 INFO:teuthology.orchestra.run.smithi137.stdout: "device_enhanced_scan": { 2024-09-12T12:33:22.542 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_enhanced_scan", 2024-09-12T12:33:22.543 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.543 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.543 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.543 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.543 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.543 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.543 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.544 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T12:33:22.544 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.544 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.544 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.544 INFO:teuthology.orchestra.run.smithi137.stdout: "facts_cache_timeout": { 2024-09-12T12:33:22.545 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "facts_cache_timeout", 2024-09-12T12:33:22.545 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.545 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.545 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.545 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:22.545 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.545 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.546 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.546 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache host facts data", 2024-09-12T12:33:22.546 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.546 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.546 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.547 INFO:teuthology.orchestra.run.smithi137.stdout: "grafana_dashboards_path": { 2024-09-12T12:33:22.547 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "grafana_dashboards_path", 2024-09-12T12:33:22.547 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.547 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.547 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.547 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T12:33:22.547 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.548 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.548 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.548 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T12:33:22.548 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.548 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.548 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.549 INFO:teuthology.orchestra.run.smithi137.stdout: "host_check_interval": { 2024-09-12T12:33:22.549 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "host_check_interval", 2024-09-12T12:33:22.549 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.549 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.549 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.549 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:22.550 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.550 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.550 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.550 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to perform a host check", 2024-09-12T12:33:22.550 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.550 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.550 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: "hw_monitoring": { 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hw_monitoring", 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.551 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.552 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.552 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.552 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T12:33:22.552 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.552 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.552 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.553 INFO:teuthology.orchestra.run.smithi137.stdout: "inventory_list_all": { 2024-09-12T12:33:22.553 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "inventory_list_all", 2024-09-12T12:33:22.553 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.553 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.553 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.553 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.553 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.554 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.555 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.555 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.555 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.555 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.555 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.555 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.555 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.556 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.556 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.556 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.556 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.556 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.556 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.556 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.557 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.557 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.557 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.557 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.557 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.557 INFO:teuthology.orchestra.run.smithi137.stdout: "log_refresh_metadata": { 2024-09-12T12:33:22.558 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_refresh_metadata", 2024-09-12T12:33:22.558 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.558 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.558 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.558 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.558 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.558 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.559 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.559 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T12:33:22.559 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.559 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.559 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.559 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.560 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.561 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.561 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T12:33:22.561 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.561 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.561 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.561 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.562 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.562 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.562 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.562 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.562 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.562 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.562 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.563 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.563 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.563 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.563 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.563 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.563 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.563 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.564 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.564 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.564 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.564 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.564 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.564 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.565 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.566 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.566 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.566 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.566 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.566 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.566 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.567 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T12:33:22.567 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T12:33:22.567 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.567 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.567 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.567 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.567 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.568 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.568 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.568 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T12:33:22.568 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.568 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.568 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.569 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T12:33:22.569 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T12:33:22.569 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.569 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.569 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.569 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "*", 2024-09-12T12:33:22.569 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.570 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.570 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.570 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T12:33:22.570 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.570 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.570 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.570 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.571 INFO:teuthology.orchestra.run.smithi137.stdout: "max_count_per_host": { 2024-09-12T12:33:22.571 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_count_per_host", 2024-09-12T12:33:22.571 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.571 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.571 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.571 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.571 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.572 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.572 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.572 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of daemons per service per host", 2024-09-12T12:33:22.572 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.572 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.572 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.573 INFO:teuthology.orchestra.run.smithi137.stdout: "max_osd_draining_count": { 2024-09-12T12:33:22.573 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_osd_draining_count", 2024-09-12T12:33:22.573 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.573 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.573 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.573 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.573 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.574 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.574 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.574 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T12:33:22.574 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.574 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.574 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "migration_current": { 2024-09-12T12:33:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "migration_current", 2024-09-12T12:33:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.575 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "internal - do not modify", 2024-09-12T12:33:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-09-12T12:33:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-09-12T12:33:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "root", 2024-09-12T12:33:22.577 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.578 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.578 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.578 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm-package", 2024-09-12T12:33:22.578 INFO:teuthology.orchestra.run.smithi137.stdout: "root" 2024-09-12T12:33:22.578 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.578 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T12:33:22.578 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.579 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.579 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.579 INFO:teuthology.orchestra.run.smithi137.stdout: "oob_default_addr": { 2024-09-12T12:33:22.579 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "oob_default_addr", 2024-09-12T12:33:22.579 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.580 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.580 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.580 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "169.254.1.1", 2024-09-12T12:33:22.580 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.580 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.580 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.580 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T12:33:22.581 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.581 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.581 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.581 INFO:teuthology.orchestra.run.smithi137.stdout: "prometheus_alerts_path": { 2024-09-12T12:33:22.581 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus_alerts_path", 2024-09-12T12:33:22.581 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T12:33:22.582 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.583 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.583 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.583 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_insecure": { 2024-09-12T12:33:22.583 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_insecure", 2024-09-12T12:33:22.583 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.583 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T12:33:22.584 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.585 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.585 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.585 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_password": { 2024-09-12T12:33:22.585 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_password", 2024-09-12T12:33:22.585 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.586 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.586 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.586 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.586 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.586 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.586 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.587 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T12:33:22.587 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.587 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.587 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.587 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_url": { 2024-09-12T12:33:22.587 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_url", 2024-09-12T12:33:22.588 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.588 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.588 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.588 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.588 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.588 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.589 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.589 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T12:33:22.589 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.589 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.589 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.589 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_username": { 2024-09-12T12:33:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_username", 2024-09-12T12:33:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.590 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.591 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.591 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T12:33:22.591 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.591 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.591 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: "secure_monitoring_stack": { 2024-09-12T12:33:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "secure_monitoring_stack", 2024-09-12T12:33:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.593 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.593 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.593 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T12:33:22.593 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.593 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.593 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.594 INFO:teuthology.orchestra.run.smithi137.stdout: "service_discovery_port": { 2024-09-12T12:33:22.594 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "service_discovery_port", 2024-09-12T12:33:22.594 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.594 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.594 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.594 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8765", 2024-09-12T12:33:22.594 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "cephadm service discovery port", 2024-09-12T12:33:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.596 INFO:teuthology.orchestra.run.smithi137.stdout: "ssh_config_file": { 2024-09-12T12:33:22.596 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssh_config_file", 2024-09-12T12:33:22.596 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.596 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.596 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.596 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.596 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.597 INFO:teuthology.orchestra.run.smithi137.stdout: "use_agent": { 2024-09-12T12:33:22.598 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_agent", 2024-09-12T12:33:22.598 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.598 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.598 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.598 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.598 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.598 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: "use_repo_digest": { 2024-09-12T12:33:22.599 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_repo_digest", 2024-09-12T12:33:22.600 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.600 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.600 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.600 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.600 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.600 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T12:33:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.601 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_failed_host_check": { 2024-09-12T12:33:22.602 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_failed_host_check", 2024-09-12T12:33:22.602 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.602 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.602 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.602 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.602 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.602 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.603 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.603 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T12:33:22.603 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.603 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.603 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.604 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_daemons": { 2024-09-12T12:33:22.604 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_daemons", 2024-09-12T12:33:22.604 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.604 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.604 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.604 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.605 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.605 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.605 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.605 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T12:33:22.605 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.605 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.605 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.606 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_hosts": { 2024-09-12T12:33:22.606 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_hosts", 2024-09-12T12:33:22.606 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.606 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.606 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.606 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.607 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.607 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.607 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.607 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T12:33:22.607 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.607 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.607 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.608 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.608 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.608 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.608 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crash", 2024-09-12T12:33:22.608 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.609 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.609 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.609 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.609 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.609 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.609 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.610 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.610 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.610 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.610 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.610 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.610 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.611 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.611 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.611 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.611 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.611 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.611 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.612 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.612 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.612 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.612 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.612 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.613 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.614 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.614 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.614 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.614 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.614 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.614 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.615 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.617 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.618 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.618 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.618 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.618 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.618 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.618 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.619 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.619 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.619 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.619 INFO:teuthology.orchestra.run.smithi137.stdout: "retain_interval": { 2024-09-12T12:33:22.619 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retain_interval", 2024-09-12T12:33:22.619 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.620 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.620 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.620 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "31536000", 2024-09-12T12:33:22.620 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.620 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.620 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.620 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T12:33:22.621 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.621 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.621 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.621 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_recent_interval": { 2024-09-12T12:33:22.621 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_recent_interval", 2024-09-12T12:33:22.622 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.622 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.622 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.622 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1209600", 2024-09-12T12:33:22.622 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.622 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.622 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.623 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T12:33:22.623 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.623 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.623 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.623 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.623 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.624 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.624 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dashboard", 2024-09-12T12:33:22.624 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.624 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.624 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.624 INFO:teuthology.orchestra.run.smithi137.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T12:33:22.625 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T12:33:22.625 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.625 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.625 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.625 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.625 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.625 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.626 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.626 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.626 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.626 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.626 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.626 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T12:33:22.627 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T12:33:22.627 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.627 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.627 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.627 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.627 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.627 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T12:33:22.628 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T12:33:22.629 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.629 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.629 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.629 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.629 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.629 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.629 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.630 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.630 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.630 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.630 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.630 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_ENABLED": { 2024-09-12T12:33:22.630 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T12:33:22.631 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.631 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.631 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.631 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.631 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.631 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.631 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.632 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.632 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.632 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.632 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.632 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T12:33:22.632 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.633 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.634 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.634 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.634 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.634 INFO:teuthology.orchestra.run.smithi137.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T12:33:22.634 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T12:33:22.634 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.635 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.636 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.636 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.636 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T12:33:22.636 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T12:33:22.636 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.636 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.637 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.638 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.638 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.638 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.638 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T12:33:22.638 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T12:33:22.638 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.638 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.639 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.639 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.639 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.639 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.639 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.639 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.639 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.640 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.640 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.640 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.640 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T12:33:22.640 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T12:33:22.640 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.640 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.641 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.641 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.641 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.641 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.641 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.641 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.642 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.642 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.642 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.642 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.642 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T12:33:22.642 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T12:33:22.642 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.643 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.643 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.643 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.643 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.643 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.643 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.644 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.644 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.644 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.644 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.644 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.644 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T12:33:22.645 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T12:33:22.645 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.645 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.645 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.645 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.645 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.645 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.646 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.646 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.646 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.646 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.646 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.646 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T12:33:22.647 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T12:33:22.647 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.647 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.647 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.647 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.647 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.647 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.648 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.648 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.648 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.648 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.648 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.648 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T12:33:22.649 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T12:33:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T12:33:22.654 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.655 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.656 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.656 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.656 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T12:33:22.656 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T12:33:22.656 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.656 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.656 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.657 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.658 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_URL": { 2024-09-12T12:33:22.659 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_URL", 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.660 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.661 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.662 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.663 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.664 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.664 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.664 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.664 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.664 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.664 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.664 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.665 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.666 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.666 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.666 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.666 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.666 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.666 INFO:teuthology.orchestra.run.smithi137.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T12:33:22.666 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.667 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.668 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.669 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.670 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.671 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.671 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.671 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.671 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.671 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.671 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.671 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.672 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T12:33:22.673 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.674 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.675 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.676 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.677 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T12:33:22.678 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.679 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.680 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.681 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.682 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.683 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.684 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T12:33:22.685 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.686 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.687 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.689 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T12:33:22.690 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8", 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.691 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.692 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "45", 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.694 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.695 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.695 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.695 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.695 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.695 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.695 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.696 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T12:33:22.697 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.698 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.699 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.701 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T12:33:22.701 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T12:33:22.701 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.701 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.701 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.701 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:22.701 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.702 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.703 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T12:33:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T12:33:22.704 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.705 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "cross_origin_url": { 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cross_origin_url", 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.706 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "crt_file": { 2024-09-12T12:33:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crt_file", 2024-09-12T12:33:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.708 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: "debug": { 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "debug", 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.710 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable/disable debug options", 2024-09-12T12:33:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "jwt_token_ttl": { 2024-09-12T12:33:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "jwt_token_ttl", 2024-09-12T12:33:22.711 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "28800", 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.712 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.713 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.714 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.715 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.716 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.717 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.718 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.719 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.720 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.721 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.722 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "motd": { 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "motd", 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.723 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "The message of the day", 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.724 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect_resolve_ip_addr": { 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.725 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.726 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:22.728 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8080", 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.729 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.730 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.731 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl_server_port": { 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl_server_port", 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8443", 2024-09-12T12:33:22.732 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-09-12T12:33:22.733 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-09-12T12:33:22.734 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.734 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.734 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.734 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "redirect", 2024-09-12T12:33:22.734 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.734 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.734 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect" 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-09-12T12:33:22.736 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: "url_prefix": { 2024-09-12T12:33:22.737 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url_prefix", 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.738 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-09-12T12:33:22.739 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_monitoring": { 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_monitoring", 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.740 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "monitor device health metrics", 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.741 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.742 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.743 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.744 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.745 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.746 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.747 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.748 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.749 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: "mark_out_threshold": { 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mark_out_threshold", 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.750 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2419200", 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_name": { 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_name", 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "device_health_metrics", 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.752 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "retention_period": { 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retention_period", 2024-09-12T12:33:22.753 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15552000", 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain device health metrics", 2024-09-12T12:33:22.754 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_frequency": { 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_frequency", 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.755 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.757 INFO:teuthology.orchestra.run.smithi137.stdout: "self_heal": { 2024-09-12T12:33:22.757 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "self_heal", 2024-09-12T12:33:22.757 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.757 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.757 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.757 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.757 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:22.758 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.759 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_threshold": { 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_threshold", 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.760 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7257600", 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.761 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "diskprediction_local", 2024-09-12T12:33:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.763 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.764 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.765 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.766 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.767 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.768 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.769 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.770 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.771 INFO:teuthology.orchestra.run.smithi137.stdout: "predict_interval": { 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predict_interval", 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.772 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: "predictor_model": { 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predictor_model", 2024-09-12T12:33:22.773 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "prophetstor", 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.774 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.775 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.776 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "influx", 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": false, 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "influxdb python module not found", 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.777 INFO:teuthology.orchestra.run.smithi137.stdout: "batch_size": { 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "batch_size", 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5000", 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.778 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: "database": { 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "database", 2024-09-12T12:33:22.779 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "ceph", 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T12:33:22.780 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": { 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hostname", 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.781 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server hostname", 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.782 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "30", 2024-09-12T12:33:22.783 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "5", 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.784 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.785 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.786 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.787 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.789 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.790 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.791 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.792 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: "password": { 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "password", 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.793 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.794 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.794 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.794 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.794 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "password of InfluxDB server user", 2024-09-12T12:33:22.794 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.794 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.794 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "port": { 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "port", 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8086", 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.795 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server port", 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-09-12T12:33:22.796 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "false", 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.797 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: "threads": { 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threads", 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.798 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "32", 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.799 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: "username": { 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "username", 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.800 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "username of InfluxDB server user", 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.801 INFO:teuthology.orchestra.run.smithi137.stdout: "verify_ssl": { 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "verify_ssl", 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "true", 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.802 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "insights", 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.804 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.805 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.807 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.808 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.808 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.808 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.808 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.808 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.808 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.809 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.809 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.810 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.811 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.811 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.811 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.812 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.812 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.812 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.812 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.812 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.812 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.813 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.813 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.813 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.813 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.813 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.813 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.814 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "iostat", 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.815 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.816 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.816 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.816 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.816 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.816 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.816 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.816 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.817 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.818 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.818 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.818 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.818 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.818 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.818 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.819 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.819 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.819 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.819 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.819 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.819 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.820 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.821 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.821 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.821 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.821 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.821 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.821 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.821 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.822 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.823 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.824 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "k8sevents", 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.825 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.826 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.826 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_event_retention_days": { 2024-09-12T12:33:22.826 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ceph_event_retention_days", 2024-09-12T12:33:22.826 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.826 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.826 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7", 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: "config_check_secs": { 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_check_secs", 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.828 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "10", 2024-09-12T12:33:22.829 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.829 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.829 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T12:33:22.829 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.829 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.829 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.830 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.830 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.830 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.830 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.830 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.831 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.831 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.831 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.831 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.831 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.831 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.831 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.832 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.832 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.832 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.832 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.832 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.833 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.834 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.835 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.836 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.837 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.838 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.838 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.838 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.838 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.838 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.838 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.839 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.839 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.839 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.839 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.839 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.839 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.839 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.840 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.840 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "localpool", 2024-09-12T12:33:22.840 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.840 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.840 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: "failure_domain": { 2024-09-12T12:33:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "failure_domain", 2024-09-12T12:33:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "host", 2024-09-12T12:33:22.841 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.842 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.842 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.842 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "failure domain for any created local pool", 2024-09-12T12:33:22.842 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T12:33:22.842 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.842 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.843 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.844 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.845 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.845 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.845 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.845 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.845 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.845 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.846 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.847 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.848 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.848 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.848 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.848 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.848 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.848 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.848 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.849 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.850 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.851 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.851 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.851 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.851 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.851 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.851 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "min_size": { 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_size", 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.852 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.853 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.853 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default min_size for any created local pool", 2024-09-12T12:33:22.853 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T12:33:22.853 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.853 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.853 INFO:teuthology.orchestra.run.smithi137.stdout: "num_rep": { 2024-09-12T12:33:22.854 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "num_rep", 2024-09-12T12:33:22.854 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.854 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.854 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.854 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3", 2024-09-12T12:33:22.854 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.855 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.855 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.855 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default replica count for any created local pool", 2024-09-12T12:33:22.855 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.855 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.855 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.856 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_num": { 2024-09-12T12:33:22.856 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_num", 2024-09-12T12:33:22.856 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.856 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.856 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.857 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "128", 2024-09-12T12:33:22.857 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.857 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.861 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.861 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T12:33:22.861 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.861 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.861 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "prefix": { 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prefix", 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.862 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.863 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.863 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name prefix for any created local pool", 2024-09-12T12:33:22.863 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.863 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.863 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.863 INFO:teuthology.orchestra.run.smithi137.stdout: "subtree": { 2024-09-12T12:33:22.864 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "subtree", 2024-09-12T12:33:22.864 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.864 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.864 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.864 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "rack", 2024-09-12T12:33:22.864 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.864 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.866 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mds_autoscaler", 2024-09-12T12:33:22.866 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.866 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.866 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.866 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.866 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.867 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.868 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.869 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.869 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.869 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.869 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.869 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.869 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.870 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.870 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.870 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.870 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.870 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.870 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.870 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.871 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.872 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.873 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.873 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.873 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.873 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.873 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.873 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.873 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.874 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.875 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.875 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.875 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.875 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.875 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.875 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.875 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirroring", 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.876 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.877 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.878 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.879 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.880 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.881 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.882 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.883 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.884 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.885 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "nfs", 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.886 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.887 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.888 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.889 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.890 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.891 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.892 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.892 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.892 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.892 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.892 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.892 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.892 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.893 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.894 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.895 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.896 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.896 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.896 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-09-12T12:33:22.896 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "fail_fs": { 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "fail_fs", 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.897 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.899 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.900 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.901 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.901 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.901 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.901 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.901 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.901 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.902 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.903 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.903 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.903 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.903 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.903 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.903 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.904 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.905 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.906 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator": { 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.908 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: "rook", 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: "test_orchestrator" 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Orchestrator backend", 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.909 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_perf_query", 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.910 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.911 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.911 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.911 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.911 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.911 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.911 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.911 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.912 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.913 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.914 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.914 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.914 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.914 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.914 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.914 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.914 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.915 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.916 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.917 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.918 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.919 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.920 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.920 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.920 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_support", 2024-09-12T12:33:22.920 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.920 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.921 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.922 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.923 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.924 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.925 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.925 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.925 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.925 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.925 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.925 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.926 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.927 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.928 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_autoscaler", 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.930 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.931 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.932 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.933 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.934 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.935 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.936 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.936 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.936 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.936 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.936 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.936 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.936 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.937 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.938 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.939 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "threshold": { 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threshold", 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1.0", 2024-09-12T12:33:22.941 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "scaling threshold", 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.942 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "progress", 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_pg_recovery_event": { 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_pg_recovery_event", 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.943 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.944 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.945 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.947 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.948 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.949 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.950 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.951 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.952 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.952 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.952 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.952 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.952 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.952 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.952 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.953 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.954 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.955 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.955 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.955 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.955 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.955 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.955 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.956 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.956 INFO:teuthology.orchestra.run.smithi137.stdout: "max_completed_events": { 2024-09-12T12:33:22.956 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_completed_events", 2024-09-12T12:33:22.956 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.956 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.956 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.957 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "50", 2024-09-12T12:33:22.957 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.957 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.957 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.957 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "number of past completed events to remember", 2024-09-12T12:33:22.957 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.957 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.958 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long the module is going to sleep", 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.959 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus", 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.960 INFO:teuthology.orchestra.run.smithi137.stdout: "cache": { 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cache", 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.961 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: "exclude_perf_counters": { 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "exclude_perf_counters", 2024-09-12T12:33:22.962 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T12:33:22.963 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.964 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.965 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.965 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.965 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.965 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.965 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.965 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.965 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.966 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.967 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.968 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.969 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.970 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.971 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.972 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools": { 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools", 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.973 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.974 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "300", 2024-09-12T12:33:22.975 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.976 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.976 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.976 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.976 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.976 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.976 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_interval": { 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_interval", 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15.0", 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.977 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:22.978 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.979 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.980 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.981 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.981 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "9283", 2024-09-12T12:33:22.981 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.981 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.981 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.981 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T12:33:22.981 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "stale_cache_strategy": { 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stale_cache_strategy", 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.982 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "log", 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.983 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "default", 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.984 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: "default", 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: "error" 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-09-12T12:33:22.986 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:22.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:22.988 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-09-12T12:33:22.988 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:22.988 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:22.988 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:22.988 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:22.988 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.989 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.990 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.991 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.992 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:22.993 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.994 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.994 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:22.994 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:22.994 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:22.994 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:22.994 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:22.994 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.995 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.996 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.997 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.997 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.997 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.997 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.997 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_snap_create": { 2024-09-12T12:33:22.997 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_snap_create", 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:22.998 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: "mirror_snapshot_schedule": { 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:22.999 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.000 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.000 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.000 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.000 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.000 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.000 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.000 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "trash_purge_schedule": { 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "trash_purge_schedule", 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.001 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.002 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "restful", 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_auth": { 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_auth", 2024-09-12T12:33:23.003 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.004 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.004 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.004 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.004 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.004 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.004 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.004 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.005 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.006 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.007 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.008 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.008 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.008 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.008 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.008 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.008 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.008 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.010 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.011 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.012 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.013 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.014 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.014 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.014 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.014 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.014 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.014 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.016 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:23.017 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.018 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.019 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rgw", 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.020 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.021 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.022 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.023 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.024 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.024 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.024 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.024 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.024 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.024 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.025 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.026 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.026 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.026 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.026 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.026 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.026 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.026 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.027 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.028 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.029 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.029 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.029 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.029 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.029 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rook", 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.030 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.031 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.033 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.033 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.033 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.033 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.033 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.033 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.033 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.034 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.035 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.036 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.037 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.037 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.037 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.037 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.037 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.037 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.038 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: "storage_class": { 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "storage_class", 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.039 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "local", 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.040 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "selftest", 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.041 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.042 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.043 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.044 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.045 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.046 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.047 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.048 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.049 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: "roption1": { 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption1", 2024-09-12T12:33:23.050 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.051 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: "roption2": { 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption2", 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.052 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "xyz", 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.053 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption1": { 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption1", 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.054 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.055 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption2": { 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption2", 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.056 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.057 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.060 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.060 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption3": { 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption3", 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.061 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption4": { 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption4", 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.063 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.064 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption5": { 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption5", 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.065 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption6": { 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption6", 2024-09-12T12:33:23.066 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.067 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption7": { 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption7", 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.068 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "42", 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.069 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: "testkey": { 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testkey", 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.070 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.071 INFO:teuthology.orchestra.run.smithi137.stdout: "testlkey": { 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testlkey", 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.072 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: "testnewline": { 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testnewline", 2024-09-12T12:33:23.073 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.074 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-09-12T12:33:23.075 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_m_granularity": { 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_m_granularity", 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.076 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "dump_on_update": { 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dump_on_update", 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.078 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "dump database to debug log on update", 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.079 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.080 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.081 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.082 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.083 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.084 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.084 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.084 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.084 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.084 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.084 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.085 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.086 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.087 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.088 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.088 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.088 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.088 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.088 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.088 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stats", 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.089 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.090 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.091 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.092 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.092 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.092 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.092 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.092 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.092 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.094 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.095 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.096 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.097 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "status", 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.098 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.099 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.100 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.101 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.102 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.103 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.104 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.105 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.106 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.107 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telegraf", 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: "address": { 2024-09-12T12:33:23.108 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "address", 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.109 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.110 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15", 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.111 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.112 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.112 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.112 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.112 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.112 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.112 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.113 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.114 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.114 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.114 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.114 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.114 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.114 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.115 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.116 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.116 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.116 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.116 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.116 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.116 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.116 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.117 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.118 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.119 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.120 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telemetry", 2024-09-12T12:33:23.121 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.122 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.122 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.122 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_basic": { 2024-09-12T12:33:23.122 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_basic", 2024-09-12T12:33:23.122 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.122 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.122 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.123 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_crash": { 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_crash", 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.124 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.125 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_device": { 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_device", 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.126 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_ident": { 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_ident", 2024-09-12T12:33:23.127 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T12:33:23.128 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_perf": { 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_perf", 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.129 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.130 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: "contact": { 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "contact", 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.131 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.132 INFO:teuthology.orchestra.run.smithi137.stdout: "description": { 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "description", 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.133 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: "device_url": { 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_url", 2024-09-12T12:33:23.134 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.135 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.136 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.137 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.138 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "24", 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "8", 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.139 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.140 INFO:teuthology.orchestra.run.smithi137.stdout: "last_opt_revision": { 2024-09-12T12:33:23.140 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "last_opt_revision", 2024-09-12T12:33:23.140 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.140 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.140 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.140 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1", 2024-09-12T12:33:23.140 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.141 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard": { 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard", 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.142 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard_description": { 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard_description", 2024-09-12T12:33:23.143 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.144 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.144 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.144 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.144 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.144 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.144 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.144 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.145 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.146 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.147 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.148 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.149 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.150 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.151 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.152 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.153 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.154 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.154 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.154 INFO:teuthology.orchestra.run.smithi137.stdout: "organization": { 2024-09-12T12:33:23.154 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "organization", 2024-09-12T12:33:23.154 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.154 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.154 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.155 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: "proxy": { 2024-09-12T12:33:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "proxy", 2024-09-12T12:33:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.156 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.157 INFO:teuthology.orchestra.run.smithi137.stdout: "url": { 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url", 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.158 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.159 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "test_orchestrator", 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.160 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.161 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.161 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.161 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.161 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.161 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.161 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.161 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.162 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.163 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.164 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.165 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.165 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.165 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.165 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.165 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.165 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.166 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.166 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.166 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.166 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.166 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.166 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.166 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.167 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.167 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.167 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.167 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.167 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.167 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.167 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.168 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.168 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.168 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.168 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.168 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.168 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.168 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.169 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.169 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.169 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.169 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.169 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.169 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.170 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.171 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.171 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.171 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.171 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.171 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.171 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.171 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.172 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.172 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.172 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.172 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.172 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.173 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.173 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.173 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.173 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.173 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.174 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.174 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.174 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.175 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.176 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.176 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.176 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.176 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.176 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.176 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.176 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.177 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.179 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.180 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.180 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.180 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.180 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.180 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.180 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.180 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_clones": { 2024-09-12T12:33:23.181 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_clones", 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4", 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.182 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: "periodic_async_work": { 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "periodic_async_work", 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.183 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Periodically check for async work", 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.184 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.185 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.185 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.185 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_delay": { 2024-09-12T12:33:23.185 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_delay", 2024-09-12T12:33:23.185 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.185 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.185 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.186 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:23.186 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.186 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.186 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.186 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T12:33:23.186 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.187 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.187 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.187 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.187 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_no_wait": { 2024-09-12T12:33:23.187 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T12:33:23.187 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.188 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.188 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.188 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.188 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.188 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.188 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.188 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T12:33:23.189 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.189 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.189 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.189 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.190 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.190 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.190 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix", 2024-09-12T12:33:23.190 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.191 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.191 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.191 INFO:teuthology.orchestra.run.smithi137.stdout: "discovery_interval": { 2024-09-12T12:33:23.192 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "discovery_interval", 2024-09-12T12:33:23.192 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:23.192 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.192 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.193 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "100", 2024-09-12T12:33:23.193 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.193 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.193 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.193 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.194 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.194 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.194 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.195 INFO:teuthology.orchestra.run.smithi137.stdout: "identifier": { 2024-09-12T12:33:23.195 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "identifier", 2024-09-12T12:33:23.195 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.195 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.195 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.196 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.196 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.196 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.196 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.196 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.197 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.197 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.197 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.197 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:23.198 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:23.198 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.198 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.198 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.198 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:23.199 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.199 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.199 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.199 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.200 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.200 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.200 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.200 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.201 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.201 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.201 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.201 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.202 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.202 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.202 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.202 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.202 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.203 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.203 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.203 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.203 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.203 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.203 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.203 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.204 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.204 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.204 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.204 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.205 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.205 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.205 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.205 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.205 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.205 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.205 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.206 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.206 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.206 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.206 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.206 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.206 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.207 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.207 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.207 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.207 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.207 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.207 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.207 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.208 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.208 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.208 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.208 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.208 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.208 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.208 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.209 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.210 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.210 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.210 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.210 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.210 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.210 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.210 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_host": { 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_host", 2024-09-12T12:33:23.211 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.212 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.212 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.212 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.212 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.212 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.212 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.212 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.213 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.213 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.213 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.213 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_port": { 2024-09-12T12:33:23.213 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_port", 2024-09-12T12:33:23.213 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10051", 2024-09-12T12:33:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.214 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.215 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.215 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.215 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.215 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_sender": { 2024-09-12T12:33:23.215 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_sender", 2024-09-12T12:33:23.215 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.216 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.216 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.216 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T12:33:23.216 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.216 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.216 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.216 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:23.217 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.218 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.218 INFO:teuthology.orchestra.run.smithi137.stdout: "modules": [ 2024-09-12T12:33:23.218 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-09-12T12:33:23.218 INFO:teuthology.orchestra.run.smithi137.stdout: "dashboard", 2024-09-12T12:33:23.219 INFO:teuthology.orchestra.run.smithi137.stdout: "iostat", 2024-09-12T12:33:23.219 INFO:teuthology.orchestra.run.smithi137.stdout: "nfs", 2024-09-12T12:33:23.219 INFO:teuthology.orchestra.run.smithi137.stdout: "restful", 2024-09-12T12:33:23.219 INFO:teuthology.orchestra.run.smithi137.stdout: "snap_schedule" 2024-09-12T12:33:23.219 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.219 INFO:teuthology.orchestra.run.smithi137.stdout: "available_modules": [ 2024-09-12T12:33:23.219 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "alerts", 2024-09-12T12:33:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.221 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.221 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.221 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:23.221 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.221 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.222 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.222 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T12:33:23.222 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.222 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.222 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.222 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.223 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.223 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.223 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.223 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.223 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.223 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.224 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.224 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.224 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.224 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.224 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.224 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.224 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.225 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.225 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.225 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.225 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.225 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.225 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.226 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.226 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.226 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.226 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.226 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.226 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.226 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.227 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.227 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.227 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.227 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.227 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.227 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.228 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.229 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.229 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.229 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.229 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.229 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.229 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.230 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.230 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.230 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.230 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.230 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.230 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.231 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.231 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.231 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.231 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.231 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.232 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.232 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.232 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.232 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.232 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.232 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.232 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_destination": { 2024-09-12T12:33:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_destination", 2024-09-12T12:33:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.233 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email address to send alerts to", 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.234 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.235 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_from_name": { 2024-09-12T12:33:23.235 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_from_name", 2024-09-12T12:33:23.235 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.235 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.235 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.236 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "Ceph", 2024-09-12T12:33:23.236 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.236 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.236 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.236 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email From: name", 2024-09-12T12:33:23.236 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_host": { 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_host", 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.237 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.238 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.238 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.238 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.238 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.238 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP server", 2024-09-12T12:33:23.238 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.239 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.239 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.239 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_password": { 2024-09-12T12:33:23.239 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_password", 2024-09-12T12:33:23.239 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.239 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.240 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.240 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.240 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.240 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.240 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.240 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Password to authenticate with", 2024-09-12T12:33:23.240 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.241 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.241 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.241 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_port": { 2024-09-12T12:33:23.241 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_port", 2024-09-12T12:33:23.241 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.242 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.242 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.242 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "465", 2024-09-12T12:33:23.242 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.242 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.242 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.242 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP port", 2024-09-12T12:33:23.243 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.243 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.243 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.243 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_sender": { 2024-09-12T12:33:23.243 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_sender", 2024-09-12T12:33:23.243 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.244 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.244 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.244 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.244 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.244 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.244 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.244 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP envelope sender", 2024-09-12T12:33:23.245 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.245 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.245 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.245 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_ssl": { 2024-09-12T12:33:23.245 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_ssl", 2024-09-12T12:33:23.245 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.246 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.246 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.246 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.246 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.246 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.246 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.246 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T12:33:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_user": { 2024-09-12T12:33:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_user", 2024-09-12T12:33:23.247 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.248 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.248 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.248 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.248 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.248 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.248 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.248 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "User to authenticate as", 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.249 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "balancer", 2024-09-12T12:33:23.250 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.250 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.250 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.250 INFO:teuthology.orchestra.run.smithi137.stdout: "active": { 2024-09-12T12:33:23.250 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "active", 2024-09-12T12:33:23.250 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T12:33:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.252 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.252 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.252 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_time": { 2024-09-12T12:33:23.252 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_time", 2024-09-12T12:33:23.252 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.252 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.253 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.253 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0000", 2024-09-12T12:33:23.253 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.253 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.253 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.253 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T12:33:23.254 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T12:33:23.254 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.254 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.254 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_weekday": { 2024-09-12T12:33:23.254 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_weekday", 2024-09-12T12:33:23.254 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:23.255 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.255 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.255 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:23.255 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-09-12T12:33:23.255 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-09-12T12:33:23.255 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.255 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T12:33:23.256 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T12:33:23.256 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.256 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.256 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_max_iterations": { 2024-09-12T12:33:23.256 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_max_iterations", 2024-09-12T12:33:23.256 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:23.257 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.257 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.257 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "25", 2024-09-12T12:33:23.257 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:23.257 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "250", 2024-09-12T12:33:23.257 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.257 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T12:33:23.258 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.258 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.258 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.258 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.258 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_metrics": { 2024-09-12T12:33:23.258 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_metrics", 2024-09-12T12:33:23.258 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.259 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.259 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.259 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T12:33:23.259 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.259 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.259 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.260 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T12:33:23.260 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T12:33:23.260 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.260 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.260 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_step": { 2024-09-12T12:33:23.260 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_step", 2024-09-12T12:33:23.261 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.261 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.261 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.261 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.5", 2024-09-12T12:33:23.261 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0.001", 2024-09-12T12:33:23.261 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "0.999", 2024-09-12T12:33:23.264 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.264 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "aggressiveness of optimization", 2024-09-12T12:33:23.264 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T12:33:23.264 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.264 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.265 INFO:teuthology.orchestra.run.smithi137.stdout: "end_time": { 2024-09-12T12:33:23.265 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_time", 2024-09-12T12:33:23.265 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.265 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.265 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.265 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2359", 2024-09-12T12:33:23.265 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T12:33:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T12:33:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.267 INFO:teuthology.orchestra.run.smithi137.stdout: "end_weekday": { 2024-09-12T12:33:23.267 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_weekday", 2024-09-12T12:33:23.267 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:23.267 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.267 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.267 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:23.267 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-09-12T12:33:23.268 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-09-12T12:33:23.268 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.268 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T12:33:23.268 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T12:33:23.268 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.268 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.269 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.269 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.269 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.269 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.269 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.269 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.269 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.270 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.270 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.270 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.270 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.270 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.270 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.271 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.271 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.271 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.271 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.271 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.271 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.271 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.272 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.272 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.272 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.272 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.272 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.272 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.273 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.273 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.273 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.273 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.273 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.273 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.273 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.274 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.274 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.274 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.274 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.275 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.275 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.275 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.275 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.275 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.275 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.275 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.276 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.276 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.276 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.276 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.276 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.276 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.276 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.277 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.277 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.277 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.277 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.277 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.278 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "min_score": { 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_score", 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.279 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.280 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.280 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:23.280 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.280 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.280 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.280 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T12:33:23.281 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.281 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.281 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.281 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-09-12T12:33:23.281 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-09-12T12:33:23.281 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.282 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.282 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.282 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "upmap", 2024-09-12T12:33:23.282 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.282 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.282 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.282 INFO:teuthology.orchestra.run.smithi137.stdout: "crush-compat", 2024-09-12T12:33:23.283 INFO:teuthology.orchestra.run.smithi137.stdout: "none", 2024-09-12T12:33:23.283 INFO:teuthology.orchestra.run.smithi137.stdout: "read", 2024-09-12T12:33:23.283 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap", 2024-09-12T12:33:23.283 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap-read" 2024-09-12T12:33:23.283 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.283 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Balancer mode", 2024-09-12T12:33:23.283 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_ids": { 2024-09-12T12:33:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_ids", 2024-09-12T12:33:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.284 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.285 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.285 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.285 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.285 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.285 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.285 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T12:33:23.285 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.286 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.287 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.287 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:23.287 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.287 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.287 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.287 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T12:33:23.287 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.288 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.288 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.288 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_deviation": { 2024-09-12T12:33:23.288 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_deviation", 2024-09-12T12:33:23.288 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.289 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.289 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.289 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:23.289 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:23.289 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.289 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.289 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T12:33:23.290 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T12:33:23.290 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.290 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.290 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_optimizations": { 2024-09-12T12:33:23.290 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_optimizations", 2024-09-12T12:33:23.290 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:23.291 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.291 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.291 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.291 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.291 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.291 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.291 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T12:33:23.292 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.292 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.292 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.292 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.292 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.293 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm", 2024-09-12T12:33:23.293 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.293 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.293 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.293 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_down_multiplier": { 2024-09-12T12:33:23.293 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_down_multiplier", 2024-09-12T12:33:23.293 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.294 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.294 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.294 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-09-12T12:33:23.294 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.294 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.294 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.295 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T12:33:23.295 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.295 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.295 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.295 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_refresh_rate": { 2024-09-12T12:33:23.295 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_refresh_rate", 2024-09-12T12:33:23.296 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.296 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.296 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.296 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "20", 2024-09-12T12:33:23.296 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.296 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.297 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.297 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T12:33:23.297 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.297 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.297 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.297 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_starting_port": { 2024-09-12T12:33:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_starting_port", 2024-09-12T12:33:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4721", 2024-09-12T12:33:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.298 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.299 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.299 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T12:33:23.299 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.299 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.299 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.299 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_ptrace": { 2024-09-12T12:33:23.300 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_ptrace", 2024-09-12T12:33:23.300 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.300 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.300 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.300 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.300 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.300 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.301 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.301 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T12:33:23.301 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T12:33:23.301 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.301 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.301 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_interval": { 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_interval", 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.302 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T12:33:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_memory_target_ratio": { 2024-09-12T12:33:23.303 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T12:33:23.304 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.304 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.304 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.304 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.7", 2024-09-12T12:33:23.304 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.304 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.305 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.305 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T12:33:23.305 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.305 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.305 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.305 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm_log_destination": { 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm_log_destination", 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.306 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.307 INFO:teuthology.orchestra.run.smithi137.stdout: "file", 2024-09-12T12:33:23.307 INFO:teuthology.orchestra.run.smithi137.stdout: "file,syslog", 2024-09-12T12:33:23.307 INFO:teuthology.orchestra.run.smithi137.stdout: "syslog" 2024-09-12T12:33:23.307 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.307 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T12:33:23.307 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.307 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.308 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.308 INFO:teuthology.orchestra.run.smithi137.stdout: "cgroups_split": { 2024-09-12T12:33:23.308 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cgroups_split", 2024-09-12T12:33:23.308 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.308 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.308 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.309 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.309 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.309 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.309 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.309 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T12:33:23.309 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.309 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.310 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.310 INFO:teuthology.orchestra.run.smithi137.stdout: "config_checks_enabled": { 2024-09-12T12:33:23.310 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_checks_enabled", 2024-09-12T12:33:23.310 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.310 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.310 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.311 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.311 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.311 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.311 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.311 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T12:33:23.311 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.311 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.312 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.312 INFO:teuthology.orchestra.run.smithi137.stdout: "config_dashboard": { 2024-09-12T12:33:23.312 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_dashboard", 2024-09-12T12:33:23.312 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.312 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.313 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_alertmanager": { 2024-09-12T12:33:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_alertmanager", 2024-09-12T12:33:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.314 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.315 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T12:33:23.315 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.315 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.315 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.315 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:23.315 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.315 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.316 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.316 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_base": { 2024-09-12T12:33:23.316 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_base", 2024-09-12T12:33:23.316 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.316 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.316 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.317 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T12:33:23.317 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.317 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.317 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.317 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Container image name, without the tag", 2024-09-12T12:33:23.317 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.318 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.318 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.318 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_elasticsearch": { 2024-09-12T12:33:23.318 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_elasticsearch", 2024-09-12T12:33:23.318 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.318 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "elasticsearch container image", 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.319 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.320 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.320 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_grafana": { 2024-09-12T12:33:23.320 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_grafana", 2024-09-12T12:33:23.320 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.320 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.320 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T12:33:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.321 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.322 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.322 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_haproxy": { 2024-09-12T12:33:23.322 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_haproxy", 2024-09-12T12:33:23.322 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.322 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.322 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "HAproxy container image", 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.323 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.324 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_agent": { 2024-09-12T12:33:23.324 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_agent", 2024-09-12T12:33:23.324 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.324 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.324 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.325 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T12:33:23.325 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.325 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.325 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.325 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger agent container image", 2024-09-12T12:33:23.325 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.325 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.326 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.326 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_collector": { 2024-09-12T12:33:23.326 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_collector", 2024-09-12T12:33:23.326 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.326 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.326 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.327 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T12:33:23.327 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.327 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.327 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.327 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger collector container image", 2024-09-12T12:33:23.327 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.327 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_query": { 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_query", 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.328 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T12:33:23.329 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.329 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.329 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.329 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger query container image", 2024-09-12T12:33:23.329 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.329 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.330 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.330 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_keepalived": { 2024-09-12T12:33:23.330 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_keepalived", 2024-09-12T12:33:23.330 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.330 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.330 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.331 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T12:33:23.331 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.331 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.331 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.331 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Keepalived container image", 2024-09-12T12:33:23.331 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_loki": { 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_loki", 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.332 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Loki container image", 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.333 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.334 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.334 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_node_exporter": { 2024-09-12T12:33:23.334 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_node_exporter", 2024-09-12T12:33:23.334 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.334 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.334 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.334 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T12:33:23.335 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.335 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.335 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.335 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:23.335 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.335 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.335 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.336 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_nvmeof": { 2024-09-12T12:33:23.336 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_nvmeof", 2024-09-12T12:33:23.336 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.336 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.336 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Nvme-of container image", 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.337 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_prometheus": { 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_prometheus", 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T12:33:23.338 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-09-12T12:33:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.340 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_promtail": { 2024-09-12T12:33:23.340 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_promtail", 2024-09-12T12:33:23.340 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.340 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.340 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.340 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T12:33:23.341 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.341 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.341 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.341 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Promtail container image", 2024-09-12T12:33:23.341 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.341 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.341 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.342 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_samba": { 2024-09-12T12:33:23.342 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_samba", 2024-09-12T12:33:23.342 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.342 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.342 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.342 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Samba/SMB container image", 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.344 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_snmp_gateway": { 2024-09-12T12:33:23.344 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_snmp_gateway", 2024-09-12T12:33:23.344 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.344 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.344 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.344 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T12:33:23.345 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.345 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.345 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.345 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SNMP Gateway container image", 2024-09-12T12:33:23.345 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.345 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.345 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.346 INFO:teuthology.orchestra.run.smithi137.stdout: "container_init": { 2024-09-12T12:33:23.346 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_init", 2024-09-12T12:33:23.346 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.346 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.346 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.347 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.347 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.347 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.347 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.347 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T12:33:23.347 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.347 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.348 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.348 INFO:teuthology.orchestra.run.smithi137.stdout: "daemon_cache_timeout": { 2024-09-12T12:33:23.348 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "daemon_cache_timeout", 2024-09-12T12:33:23.348 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.348 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.349 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.349 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:23.349 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.349 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.349 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.349 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T12:33:23.349 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.350 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.350 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.350 INFO:teuthology.orchestra.run.smithi137.stdout: "default_cephadm_command_timeout": { 2024-09-12T12:33:23.350 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T12:33:23.350 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.350 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.351 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.351 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "900", 2024-09-12T12:33:23.351 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.351 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.351 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.351 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T12:33:23.351 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.352 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.352 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.352 INFO:teuthology.orchestra.run.smithi137.stdout: "default_registry": { 2024-09-12T12:33:23.352 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_registry", 2024-09-12T12:33:23.352 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io", 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T12:33:23.353 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.354 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.354 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.354 INFO:teuthology.orchestra.run.smithi137.stdout: "device_cache_timeout": { 2024-09-12T12:33:23.354 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_cache_timeout", 2024-09-12T12:33:23.354 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.354 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1800", 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache device inventory", 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.355 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.356 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.356 INFO:teuthology.orchestra.run.smithi137.stdout: "device_enhanced_scan": { 2024-09-12T12:33:23.356 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_enhanced_scan", 2024-09-12T12:33:23.356 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.356 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.356 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T12:33:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.357 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: "facts_cache_timeout": { 2024-09-12T12:33:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "facts_cache_timeout", 2024-09-12T12:33:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.358 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.359 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:23.359 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.359 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.359 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.359 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache host facts data", 2024-09-12T12:33:23.359 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.360 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.360 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.360 INFO:teuthology.orchestra.run.smithi137.stdout: "grafana_dashboards_path": { 2024-09-12T12:33:23.360 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "grafana_dashboards_path", 2024-09-12T12:33:23.360 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.360 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.361 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.361 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T12:33:23.361 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.361 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.361 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.361 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T12:33:23.362 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.362 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.362 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.362 INFO:teuthology.orchestra.run.smithi137.stdout: "host_check_interval": { 2024-09-12T12:33:23.362 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "host_check_interval", 2024-09-12T12:33:23.362 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.363 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.363 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.363 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:23.363 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.363 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.363 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to perform a host check", 2024-09-12T12:33:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: "hw_monitoring": { 2024-09-12T12:33:23.364 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hw_monitoring", 2024-09-12T12:33:23.365 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.365 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.365 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.365 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.365 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.365 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.366 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.366 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T12:33:23.366 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.366 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.366 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.366 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.366 INFO:teuthology.orchestra.run.smithi137.stdout: "inventory_list_all": { 2024-09-12T12:33:23.367 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "inventory_list_all", 2024-09-12T12:33:23.367 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.367 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.367 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.367 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.367 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.367 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.368 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.368 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T12:33:23.368 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.368 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.368 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.368 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.369 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.369 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.369 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.369 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.369 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.369 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.369 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.370 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.370 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.370 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.370 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.370 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.370 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.370 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.371 INFO:teuthology.orchestra.run.smithi137.stdout: "log_refresh_metadata": { 2024-09-12T12:33:23.372 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_refresh_metadata", 2024-09-12T12:33:23.372 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.372 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.372 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.372 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.372 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.372 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.373 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.373 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T12:33:23.373 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.373 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.373 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.373 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.374 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.374 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.374 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.374 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.374 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.374 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.375 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.375 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.375 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T12:33:23.375 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.375 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.375 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.376 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.376 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.376 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.376 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.376 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.376 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.377 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.377 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.377 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.377 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.377 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.377 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.378 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.378 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.378 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.378 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.378 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.378 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.378 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.379 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.379 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.379 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.379 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.379 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.379 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.380 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.380 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.380 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.380 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.380 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.380 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.380 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.381 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.381 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T12:33:23.381 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T12:33:23.381 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.381 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.382 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.382 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.382 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.382 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.382 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.382 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T12:33:23.382 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.383 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.384 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "*", 2024-09-12T12:33:23.384 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.384 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.384 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.384 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T12:33:23.384 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.384 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: "max_count_per_host": { 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_count_per_host", 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.385 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.386 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.386 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.386 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.386 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of daemons per service per host", 2024-09-12T12:33:23.386 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.386 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.386 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.387 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.387 INFO:teuthology.orchestra.run.smithi137.stdout: "max_osd_draining_count": { 2024-09-12T12:33:23.387 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_osd_draining_count", 2024-09-12T12:33:23.387 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.387 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.387 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.387 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.388 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.388 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.388 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.388 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T12:33:23.388 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.389 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.389 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.389 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.389 INFO:teuthology.orchestra.run.smithi137.stdout: "migration_current": { 2024-09-12T12:33:23.389 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "migration_current", 2024-09-12T12:33:23.389 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.389 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.390 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.390 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.390 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.390 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.390 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.390 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "internal - do not modify", 2024-09-12T12:33:23.390 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.391 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.391 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.391 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.391 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-09-12T12:33:23.391 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-09-12T12:33:23.391 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.391 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.392 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.392 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "root", 2024-09-12T12:33:23.392 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.392 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.392 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.392 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm-package", 2024-09-12T12:33:23.393 INFO:teuthology.orchestra.run.smithi137.stdout: "root" 2024-09-12T12:33:23.393 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.393 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T12:33:23.393 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.393 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.393 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.393 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.394 INFO:teuthology.orchestra.run.smithi137.stdout: "oob_default_addr": { 2024-09-12T12:33:23.394 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "oob_default_addr", 2024-09-12T12:33:23.394 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.394 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.394 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.394 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "169.254.1.1", 2024-09-12T12:33:23.394 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T12:33:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.395 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.396 INFO:teuthology.orchestra.run.smithi137.stdout: "prometheus_alerts_path": { 2024-09-12T12:33:23.396 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus_alerts_path", 2024-09-12T12:33:23.396 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.396 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.396 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.396 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T12:33:23.396 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.397 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.397 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.397 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T12:33:23.397 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.397 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.397 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.397 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.398 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_insecure": { 2024-09-12T12:33:23.398 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_insecure", 2024-09-12T12:33:23.398 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.398 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.398 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.398 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.398 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.399 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.399 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.399 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T12:33:23.399 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.399 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.399 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.399 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.400 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_password": { 2024-09-12T12:33:23.400 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_password", 2024-09-12T12:33:23.400 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.400 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.400 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.400 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.400 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.401 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.401 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.401 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T12:33:23.401 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.401 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.401 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.401 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.402 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_url": { 2024-09-12T12:33:23.402 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_url", 2024-09-12T12:33:23.402 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.402 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.402 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.402 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.402 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.403 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_username": { 2024-09-12T12:33:23.404 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_username", 2024-09-12T12:33:23.404 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.404 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.404 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.404 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.404 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.405 INFO:teuthology.orchestra.run.smithi137.stdout: "secure_monitoring_stack": { 2024-09-12T12:33:23.406 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "secure_monitoring_stack", 2024-09-12T12:33:23.406 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.406 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.406 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.406 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.406 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.407 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.407 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.407 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T12:33:23.407 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.407 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.407 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.407 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.408 INFO:teuthology.orchestra.run.smithi137.stdout: "service_discovery_port": { 2024-09-12T12:33:23.408 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "service_discovery_port", 2024-09-12T12:33:23.408 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.408 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.408 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.408 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8765", 2024-09-12T12:33:23.409 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.409 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.409 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.409 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "cephadm service discovery port", 2024-09-12T12:33:23.409 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.409 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.409 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.410 INFO:teuthology.orchestra.run.smithi137.stdout: "ssh_config_file": { 2024-09-12T12:33:23.410 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssh_config_file", 2024-09-12T12:33:23.410 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.410 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.410 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.411 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.411 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.411 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.411 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.411 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T12:33:23.411 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.411 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.412 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.412 INFO:teuthology.orchestra.run.smithi137.stdout: "use_agent": { 2024-09-12T12:33:23.412 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_agent", 2024-09-12T12:33:23.412 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.412 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T12:33:23.413 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.414 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.414 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.414 INFO:teuthology.orchestra.run.smithi137.stdout: "use_repo_digest": { 2024-09-12T12:33:23.414 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_repo_digest", 2024-09-12T12:33:23.414 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.414 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.415 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.416 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.416 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_failed_host_check": { 2024-09-12T12:33:23.416 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_failed_host_check", 2024-09-12T12:33:23.416 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.416 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.416 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.417 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.417 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.417 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.417 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.417 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T12:33:23.417 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.417 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_daemons": { 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_daemons", 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.418 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.419 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.419 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.419 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.419 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T12:33:23.419 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.419 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.419 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.420 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_hosts": { 2024-09-12T12:33:23.420 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_hosts", 2024-09-12T12:33:23.420 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.420 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.420 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.420 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.421 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.421 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.421 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.421 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T12:33:23.421 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.421 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.422 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.422 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.422 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.422 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crash", 2024-09-12T12:33:23.423 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.423 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.423 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.423 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.423 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.423 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.423 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.424 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.424 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.424 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.424 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.424 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.424 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.425 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.425 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.425 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.425 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.425 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.425 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.425 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.426 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.426 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.426 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.426 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.426 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.426 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.427 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.427 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.427 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.427 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.427 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.427 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.427 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.428 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.428 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.428 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.428 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.428 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.428 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.429 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.429 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.429 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.429 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.429 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.429 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.429 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.430 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.430 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.430 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.430 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.430 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.430 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.430 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.431 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.431 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.431 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.431 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.431 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.432 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.433 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.433 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.433 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.433 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.433 INFO:teuthology.orchestra.run.smithi137.stdout: "retain_interval": { 2024-09-12T12:33:23.433 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retain_interval", 2024-09-12T12:33:23.434 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.434 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.434 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.434 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "31536000", 2024-09-12T12:33:23.434 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.434 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.434 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.435 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T12:33:23.435 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.435 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.435 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.435 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_recent_interval": { 2024-09-12T12:33:23.435 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_recent_interval", 2024-09-12T12:33:23.436 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.436 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.436 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.436 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1209600", 2024-09-12T12:33:23.436 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.436 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dashboard", 2024-09-12T12:33:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T12:33:23.438 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T12:33:23.439 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.439 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.439 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.439 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.439 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.439 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.440 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.440 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.440 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.440 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.440 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.440 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T12:33:23.441 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T12:33:23.441 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.441 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.441 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.441 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.441 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.441 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.442 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.442 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.442 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.442 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.442 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.442 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T12:33:23.443 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T12:33:23.443 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.443 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.443 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.443 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.443 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.444 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.444 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.444 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.444 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.444 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.444 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.445 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_ENABLED": { 2024-09-12T12:33:23.445 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T12:33:23.445 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.445 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.445 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.445 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.446 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.446 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.446 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.446 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.446 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.446 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.446 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.447 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T12:33:23.447 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T12:33:23.447 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.447 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.447 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.448 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.448 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.448 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.448 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.448 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.448 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.449 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.449 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.449 INFO:teuthology.orchestra.run.smithi137.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T12:33:23.449 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T12:33:23.449 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.449 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.450 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.450 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.450 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.450 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.450 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.450 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.450 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.451 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.451 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.451 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T12:33:23.451 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T12:33:23.451 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.451 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.452 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.453 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.453 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.453 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T12:33:23.453 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T12:33:23.453 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.453 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.454 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T12:33:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T12:33:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.455 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.456 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.456 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.456 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.456 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.456 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.456 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.456 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.457 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.457 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T12:33:23.457 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T12:33:23.457 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.457 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.458 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T12:33:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T12:33:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.459 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.460 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T12:33:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T12:33:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.461 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.462 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T12:33:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T12:33:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.463 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.464 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.465 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.466 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.467 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T12:33:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T12:33:23.468 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.469 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T12:33:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T12:33:23.470 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.471 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_URL": { 2024-09-12T12:33:23.472 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_URL", 2024-09-12T12:33:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.473 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T12:33:23.474 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T12:33:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:23.475 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.476 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T12:33:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T12:33:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.477 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.478 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T12:33:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T12:33:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.479 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.480 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T12:33:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T12:33:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.481 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.482 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.483 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.483 INFO:teuthology.orchestra.run.smithi137.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T12:33:23.483 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T12:33:23.483 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.483 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.483 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.484 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.484 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.484 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.484 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.484 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.484 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.484 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.485 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.485 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T12:33:23.485 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T12:33:23.485 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.485 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.485 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.486 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.486 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.486 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.486 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.486 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.486 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.486 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.487 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.487 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T12:33:23.487 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T12:33:23.487 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.487 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.487 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.488 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.488 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.488 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.488 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.488 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.488 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.488 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.489 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.490 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.490 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.490 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.490 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.490 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.490 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.491 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.491 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T12:33:23.491 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T12:33:23.491 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.491 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.491 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.492 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.493 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T12:33:23.493 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T12:33:23.493 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.493 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.493 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.494 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.494 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.494 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.494 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.494 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.494 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.494 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.495 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.495 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T12:33:23.495 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T12:33:23.495 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.495 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.495 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.496 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.496 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.496 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.496 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.496 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.496 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.497 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.498 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.498 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.498 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.498 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.498 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.498 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.499 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.500 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.500 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.500 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.500 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.500 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.500 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.500 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.501 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.501 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T12:33:23.501 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T12:33:23.501 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.501 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.501 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.502 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.502 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.502 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.502 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.502 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.502 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.502 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.503 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.503 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T12:33:23.503 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T12:33:23.503 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.503 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.503 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.504 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.504 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.504 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.504 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.504 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.504 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.504 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.505 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.505 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.505 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T12:33:23.505 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T12:33:23.505 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.505 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.506 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.506 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T12:33:23.506 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.506 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.506 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.506 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T12:33:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T12:33:23.507 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.508 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.508 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.508 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.508 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.508 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.508 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.508 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.509 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.509 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.509 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.509 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.509 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T12:33:23.509 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T12:33:23.509 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.510 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.510 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.510 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8", 2024-09-12T12:33:23.510 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.510 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.510 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.510 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.511 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.511 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.511 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.511 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.511 INFO:teuthology.orchestra.run.smithi137.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T12:33:23.511 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T12:33:23.511 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.512 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.512 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.512 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "45", 2024-09-12T12:33:23.512 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.512 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.512 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.513 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.513 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.513 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.513 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.513 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.513 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T12:33:23.513 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.514 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.515 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.515 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.515 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.515 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T12:33:23.515 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T12:33:23.515 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.516 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.516 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.516 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-09-12T12:33:23.516 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.516 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.516 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.517 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.517 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.517 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.517 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.517 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T12:33:23.517 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T12:33:23.518 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.518 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.518 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.518 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.518 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.518 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T12:33:23.519 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T12:33:23.520 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.520 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.520 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.520 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.521 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.521 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.521 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.521 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.521 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.521 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.521 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.522 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T12:33:23.522 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T12:33:23.522 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.522 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.522 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.522 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:23.523 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.523 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.523 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.523 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.523 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.523 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.523 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.524 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.525 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.525 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.525 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.525 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.525 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.525 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T12:33:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T12:33:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:23.526 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.527 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.527 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.527 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.527 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.527 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.527 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "cross_origin_url": { 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cross_origin_url", 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.528 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.529 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.529 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.529 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.529 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.529 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.529 INFO:teuthology.orchestra.run.smithi137.stdout: "crt_file": { 2024-09-12T12:33:23.530 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crt_file", 2024-09-12T12:33:23.530 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.530 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.530 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.530 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.530 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.530 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: "debug": { 2024-09-12T12:33:23.531 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "debug", 2024-09-12T12:33:23.532 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.532 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.532 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.532 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.532 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.532 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable/disable debug options", 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: "jwt_token_ttl": { 2024-09-12T12:33:23.533 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "jwt_token_ttl", 2024-09-12T12:33:23.534 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.534 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.534 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.534 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "28800", 2024-09-12T12:33:23.534 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.534 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.535 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.535 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.535 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.535 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.535 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.536 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-09-12T12:33:23.536 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-09-12T12:33:23.536 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.536 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.536 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.536 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.536 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.537 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.537 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.537 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.537 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.537 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.537 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.538 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.538 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.538 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.538 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.538 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.538 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.539 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.539 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.539 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.539 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.539 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.539 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.539 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.540 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.540 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.540 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.540 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.540 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.540 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.541 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.543 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.543 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.543 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.543 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.543 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.543 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.543 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.544 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.545 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.545 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.545 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.545 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.545 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.545 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.546 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.546 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.546 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.546 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.546 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.546 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "motd": { 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "motd", 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.548 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.549 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.549 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "The message of the day", 2024-09-12T12:33:23.549 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.549 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.549 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.549 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect_resolve_ip_addr": { 2024-09-12T12:33:23.550 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T12:33:23.550 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.550 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.550 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.550 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.550 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.550 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.551 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.551 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.551 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.551 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.551 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.551 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.552 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.553 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.553 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.553 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.553 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.553 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8080", 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.554 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.555 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.555 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.555 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.555 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.555 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-09-12T12:33:23.556 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-09-12T12:33:23.556 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.556 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.556 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.556 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.556 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.556 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.557 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.557 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.557 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.557 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.557 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.558 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl_server_port": { 2024-09-12T12:33:23.558 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl_server_port", 2024-09-12T12:33:23.558 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.558 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.558 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.558 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8443", 2024-09-12T12:33:23.559 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.559 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.559 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.559 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.559 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.560 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.560 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.560 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-09-12T12:33:23.560 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-09-12T12:33:23.560 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.561 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.561 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.561 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "redirect", 2024-09-12T12:33:23.561 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.561 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.561 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.561 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.562 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect" 2024-09-12T12:33:23.562 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.562 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.562 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.562 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.562 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-09-12T12:33:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-09-12T12:33:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-09-12T12:33:23.563 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-09-12T12:33:23.564 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-09-12T12:33:23.564 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.564 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.564 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.564 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.564 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "url_prefix": { 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url_prefix", 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.565 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.566 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.566 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.566 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.566 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.566 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.566 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.567 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_monitoring": { 2024-09-12T12:33:23.568 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_monitoring", 2024-09-12T12:33:23.568 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.568 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.568 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.568 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.568 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.568 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.569 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.569 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "monitor device health metrics", 2024-09-12T12:33:23.569 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.569 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.569 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.569 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.569 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.570 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.571 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.572 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.572 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.572 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.572 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.572 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.572 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.574 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.574 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.574 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.574 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.574 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.574 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.575 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.575 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.575 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.575 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.575 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.576 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.576 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.576 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.576 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.576 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.576 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.576 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.577 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.577 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.577 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.577 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.577 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.577 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.578 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.579 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.579 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.579 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.579 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.579 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.579 INFO:teuthology.orchestra.run.smithi137.stdout: "mark_out_threshold": { 2024-09-12T12:33:23.580 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mark_out_threshold", 2024-09-12T12:33:23.580 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.580 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.580 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.580 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2419200", 2024-09-12T12:33:23.580 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.580 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T12:33:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.581 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_name": { 2024-09-12T12:33:23.582 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_name", 2024-09-12T12:33:23.582 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.582 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.582 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.582 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "device_health_metrics", 2024-09-12T12:33:23.582 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.582 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: "retention_period": { 2024-09-12T12:33:23.583 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retention_period", 2024-09-12T12:33:23.584 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.584 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.584 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.584 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15552000", 2024-09-12T12:33:23.584 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.584 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.584 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain device health metrics", 2024-09-12T12:33:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_frequency": { 2024-09-12T12:33:23.585 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_frequency", 2024-09-12T12:33:23.586 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.586 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.586 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.586 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-09-12T12:33:23.586 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.586 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.586 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: "self_heal": { 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "self_heal", 2024-09-12T12:33:23.587 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.588 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.588 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.588 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.588 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.588 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.588 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.588 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.589 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.590 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.590 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:23.590 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.590 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.590 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.590 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T12:33:23.590 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.591 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.591 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.591 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_threshold": { 2024-09-12T12:33:23.591 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_threshold", 2024-09-12T12:33:23.591 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.591 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.592 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.592 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7257600", 2024-09-12T12:33:23.592 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.592 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.592 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.592 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T12:33:23.593 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.593 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.593 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.593 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.593 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.593 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.594 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "diskprediction_local", 2024-09-12T12:33:23.594 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.594 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.594 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.594 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.594 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.594 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.595 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.595 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.595 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.595 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.595 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.595 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.595 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.596 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.596 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.596 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.596 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.596 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.596 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.596 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.597 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.597 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.597 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.597 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.597 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.597 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.598 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.598 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.598 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.598 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.598 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.598 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.598 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.599 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.600 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.600 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.600 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.600 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.600 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.600 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.601 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.601 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.601 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.601 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.601 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.601 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.602 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.602 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.602 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.602 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.602 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.602 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.602 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.603 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.604 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.604 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.604 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.604 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.604 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.604 INFO:teuthology.orchestra.run.smithi137.stdout: "predict_interval": { 2024-09-12T12:33:23.604 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predict_interval", 2024-09-12T12:33:23.605 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.605 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.605 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.605 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-09-12T12:33:23.605 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.606 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.606 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.606 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.606 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.606 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.606 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.607 INFO:teuthology.orchestra.run.smithi137.stdout: "predictor_model": { 2024-09-12T12:33:23.607 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predictor_model", 2024-09-12T12:33:23.607 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.607 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.607 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.607 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "prophetstor", 2024-09-12T12:33:23.607 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.608 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:23.609 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:23.609 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.609 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.609 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.609 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-09-12T12:33:23.609 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.609 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.610 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.610 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.610 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.610 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.610 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.610 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.610 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.611 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.611 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "influx", 2024-09-12T12:33:23.611 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": false, 2024-09-12T12:33:23.611 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "influxdb python module not found", 2024-09-12T12:33:23.611 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.611 INFO:teuthology.orchestra.run.smithi137.stdout: "batch_size": { 2024-09-12T12:33:23.612 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "batch_size", 2024-09-12T12:33:23.612 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.612 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.612 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.612 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5000", 2024-09-12T12:33:23.612 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.612 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.613 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.613 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T12:33:23.613 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.613 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.613 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "database": { 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "database", 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "ceph", 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.614 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.615 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.615 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T12:33:23.615 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.615 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.615 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.615 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": { 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hostname", 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.616 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.617 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server hostname", 2024-09-12T12:33:23.617 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.617 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.617 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.617 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "30", 2024-09-12T12:33:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "5", 2024-09-12T12:33:23.618 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.619 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.619 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T12:33:23.619 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.619 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.619 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.619 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.620 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.620 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.620 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.620 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.620 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.620 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.620 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.621 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.621 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.621 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.621 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.621 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.621 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.622 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.622 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.622 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.622 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.622 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.622 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.623 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.623 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.623 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.623 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.623 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.623 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.624 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.624 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.624 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.624 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.624 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.624 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.624 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.625 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.625 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.625 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.625 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.625 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.626 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.626 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.626 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.626 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.626 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.626 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.626 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.627 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.627 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.627 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.627 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.627 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.627 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.627 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.628 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.628 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.628 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.628 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.628 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.629 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.629 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.629 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.629 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.629 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.629 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: "password": { 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "password", 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.630 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.631 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.631 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.631 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.631 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.631 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.631 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "password of InfluxDB server user", 2024-09-12T12:33:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: "port": { 2024-09-12T12:33:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "port", 2024-09-12T12:33:23.632 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.633 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.633 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.633 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8086", 2024-09-12T12:33:23.633 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.633 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.633 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.633 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server port", 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.634 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.635 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.635 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "false", 2024-09-12T12:33:23.635 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.635 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.635 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.635 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T12:33:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: "threads": { 2024-09-12T12:33:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threads", 2024-09-12T12:33:23.636 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.637 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.637 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.637 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:23.637 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:23.637 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "32", 2024-09-12T12:33:23.637 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.637 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T12:33:23.638 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.638 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.638 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.638 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.638 INFO:teuthology.orchestra.run.smithi137.stdout: "username": { 2024-09-12T12:33:23.638 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "username", 2024-09-12T12:33:23.638 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.639 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.639 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.639 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.639 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.639 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.639 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.639 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "username of InfluxDB server user", 2024-09-12T12:33:23.640 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.640 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.640 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.640 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.640 INFO:teuthology.orchestra.run.smithi137.stdout: "verify_ssl": { 2024-09-12T12:33:23.640 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "verify_ssl", 2024-09-12T12:33:23.640 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.641 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.641 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.641 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "true", 2024-09-12T12:33:23.641 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.641 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.641 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.641 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T12:33:23.642 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.642 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.642 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.642 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.642 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.642 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.642 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.643 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "insights", 2024-09-12T12:33:23.643 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.643 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.643 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.643 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.643 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.643 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.644 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.644 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.644 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.644 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.644 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.644 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.645 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.645 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.645 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.645 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.645 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.645 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.645 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.646 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.647 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.648 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.649 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.650 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.650 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.650 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.650 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.650 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.650 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.650 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.651 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.651 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.651 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.651 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.651 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.651 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.652 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.652 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.652 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.652 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.652 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.652 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.652 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.653 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.653 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.653 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.653 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.653 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.653 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.654 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.654 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "iostat", 2024-09-12T12:33:23.654 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.654 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.654 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.654 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.654 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.655 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.656 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.656 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.656 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.656 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.656 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.656 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.656 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.657 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.657 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.657 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.657 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.657 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.657 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.657 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.658 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.658 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.658 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.658 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.658 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.658 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.658 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.659 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.660 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.660 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.660 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.660 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.660 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.660 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.660 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.661 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.661 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.661 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.661 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.661 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.661 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.662 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.662 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.662 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.662 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.662 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.662 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.662 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.663 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.664 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.664 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.664 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.664 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.664 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.664 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.665 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "k8sevents", 2024-09-12T12:33:23.665 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.665 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.665 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.665 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_event_retention_days": { 2024-09-12T12:33:23.665 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ceph_event_retention_days", 2024-09-12T12:33:23.665 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.666 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.666 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.666 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7", 2024-09-12T12:33:23.666 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.666 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.666 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.667 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T12:33:23.667 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.667 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.667 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.667 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.667 INFO:teuthology.orchestra.run.smithi137.stdout: "config_check_secs": { 2024-09-12T12:33:23.667 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_check_secs", 2024-09-12T12:33:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "10", 2024-09-12T12:33:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.668 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.669 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T12:33:23.669 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.669 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.669 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.669 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.669 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.669 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.671 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.671 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.671 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.671 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.671 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.671 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.672 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.672 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.672 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.672 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.672 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.672 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.672 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.673 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.674 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.674 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.674 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.674 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.674 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.674 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.675 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.676 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.676 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.676 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.676 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.677 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.677 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.677 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.677 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.677 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.677 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.677 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.678 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.678 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.678 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.678 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.678 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.678 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.678 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.679 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.679 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.679 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.679 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.679 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.679 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.680 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.680 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.680 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.680 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.680 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.680 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.682 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "localpool", 2024-09-12T12:33:23.682 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.682 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.682 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.682 INFO:teuthology.orchestra.run.smithi137.stdout: "failure_domain": { 2024-09-12T12:33:23.682 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "failure_domain", 2024-09-12T12:33:23.682 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.683 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.683 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.683 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "host", 2024-09-12T12:33:23.683 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.683 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.683 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.684 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "failure domain for any created local pool", 2024-09-12T12:33:23.684 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T12:33:23.684 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.685 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.686 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.686 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.686 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.686 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.686 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.686 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.687 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.687 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.687 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.687 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.687 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.687 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.687 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.688 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.688 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.688 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.688 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.688 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.688 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.689 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.689 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.689 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.689 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.689 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.689 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.689 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.690 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.690 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.690 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.690 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.690 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.690 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.691 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.691 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.691 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.691 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.691 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.691 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.692 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.692 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.692 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.692 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.692 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.692 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.692 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.693 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.693 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.693 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.693 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.693 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.693 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.694 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.694 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.694 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.694 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.694 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.694 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.694 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.695 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.695 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.695 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.695 INFO:teuthology.orchestra.run.smithi137.stdout: "min_size": { 2024-09-12T12:33:23.695 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_size", 2024-09-12T12:33:23.695 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.696 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.696 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.696 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.696 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.696 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.696 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.696 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default min_size for any created local pool", 2024-09-12T12:33:23.697 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T12:33:23.697 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.697 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.697 INFO:teuthology.orchestra.run.smithi137.stdout: "num_rep": { 2024-09-12T12:33:23.697 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "num_rep", 2024-09-12T12:33:23.697 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.698 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.698 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.698 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3", 2024-09-12T12:33:23.698 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.698 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.698 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.698 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default replica count for any created local pool", 2024-09-12T12:33:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_num": { 2024-09-12T12:33:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_num", 2024-09-12T12:33:23.699 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.700 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.700 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.700 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "128", 2024-09-12T12:33:23.700 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.700 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.700 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T12:33:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: "prefix": { 2024-09-12T12:33:23.701 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prefix", 2024-09-12T12:33:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.702 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.703 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name prefix for any created local pool", 2024-09-12T12:33:23.703 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.703 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.703 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.703 INFO:teuthology.orchestra.run.smithi137.stdout: "subtree": { 2024-09-12T12:33:23.703 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "subtree", 2024-09-12T12:33:23.704 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.704 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.704 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.704 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "rack", 2024-09-12T12:33:23.704 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.704 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.704 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.706 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mds_autoscaler", 2024-09-12T12:33:23.706 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.706 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.706 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.706 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.706 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.706 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.707 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.708 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.708 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.708 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.708 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.708 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.708 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.708 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.709 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.712 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.713 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.714 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.715 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.716 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirroring", 2024-09-12T12:33:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.717 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.718 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.719 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.720 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.721 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.723 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.724 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.726 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.727 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "nfs", 2024-09-12T12:33:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.728 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.729 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.730 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.731 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.732 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.735 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.736 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.737 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.738 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.739 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "fail_fs": { 2024-09-12T12:33:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "fail_fs", 2024-09-12T12:33:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.740 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T12:33:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.741 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.742 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.743 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.744 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.745 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.746 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.747 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.748 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.749 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.750 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.751 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator": { 2024-09-12T12:33:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-09-12T12:33:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.752 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-09-12T12:33:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "rook", 2024-09-12T12:33:23.753 INFO:teuthology.orchestra.run.smithi137.stdout: "test_orchestrator" 2024-09-12T12:33:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Orchestrator backend", 2024-09-12T12:33:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.754 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_perf_query", 2024-09-12T12:33:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.755 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.756 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.757 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.758 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.759 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.761 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.762 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.764 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.765 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_support", 2024-09-12T12:33:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.766 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.767 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.768 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.769 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.770 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.771 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.772 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.773 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.774 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.775 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.776 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_autoscaler", 2024-09-12T12:33:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.778 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.779 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.780 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.781 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.782 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.783 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.784 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.785 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.786 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.787 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.788 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.789 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "threshold": { 2024-09-12T12:33:23.790 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threshold", 2024-09-12T12:33:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-09-12T12:33:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1.0", 2024-09-12T12:33:23.791 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "scaling threshold", 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "progress", 2024-09-12T12:33:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_pg_recovery_event": { 2024-09-12T12:33:23.793 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_pg_recovery_event", 2024-09-12T12:33:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.794 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T12:33:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-09-12T12:33:23.795 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-09-12T12:33:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.796 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.797 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.798 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.799 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.801 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.802 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.803 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.804 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.805 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.806 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "max_completed_events": { 2024-09-12T12:33:23.807 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_completed_events", 2024-09-12T12:33:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "50", 2024-09-12T12:33:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.808 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "number of past completed events to remember", 2024-09-12T12:33:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-09-12T12:33:23.809 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-09-12T12:33:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-09-12T12:33:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.810 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long the module is going to sleep", 2024-09-12T12:33:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus", 2024-09-12T12:33:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.812 INFO:teuthology.orchestra.run.smithi137.stdout: "cache": { 2024-09-12T12:33:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cache", 2024-09-12T12:33:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.813 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.814 INFO:teuthology.orchestra.run.smithi137.stdout: "exclude_perf_counters": { 2024-09-12T12:33:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "exclude_perf_counters", 2024-09-12T12:33:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.815 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T12:33:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T12:33:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.816 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.817 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.818 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.819 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.820 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.821 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.822 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.823 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.824 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.825 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools": { 2024-09-12T12:33:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools", 2024-09-12T12:33:23.826 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.827 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T12:33:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T12:33:23.828 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "300", 2024-09-12T12:33:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.829 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_interval": { 2024-09-12T12:33:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_interval", 2024-09-12T12:33:23.830 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15.0", 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.831 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.832 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-09-12T12:33:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.833 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T12:33:23.834 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.834 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.834 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.834 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.834 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:23.834 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:23.834 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.835 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.835 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.835 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "9283", 2024-09-12T12:33:23.835 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.835 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.835 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.835 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T12:33:23.836 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.836 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.836 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.836 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.836 INFO:teuthology.orchestra.run.smithi137.stdout: "stale_cache_strategy": { 2024-09-12T12:33:23.836 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stale_cache_strategy", 2024-09-12T12:33:23.836 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.837 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.837 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.837 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "log", 2024-09-12T12:33:23.837 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.837 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.837 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.837 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-09-12T12:33:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-09-12T12:33:23.838 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "default", 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "default", 2024-09-12T12:33:23.839 INFO:teuthology.orchestra.run.smithi137.stdout: "error" 2024-09-12T12:33:23.840 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.840 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.840 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.840 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.840 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.841 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-09-12T12:33:23.841 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-09-12T12:33:23.841 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.841 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.841 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.841 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-09-12T12:33:23.841 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-09-12T12:33:23.842 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-09-12T12:33:23.842 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.842 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.842 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.842 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.842 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.842 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.843 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.843 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.843 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-09-12T12:33:23.843 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.843 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.843 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.844 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.844 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.844 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.844 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.844 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.844 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_snap_create": { 2024-09-12T12:33:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_snap_create", 2024-09-12T12:33:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-09-12T12:33:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "mirror_snapshot_schedule": { 2024-09-12T12:33:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T12:33:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "trash_purge_schedule": { 2024-09-12T12:33:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "trash_purge_schedule", 2024-09-12T12:33:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "restful", 2024-09-12T12:33:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_auth": { 2024-09-12T12:33:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_auth", 2024-09-12T12:33:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-09-12T12:33:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-09-12T12:33:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-09-12T12:33:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-09-12T12:33:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-09-12T12:33:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rgw", 2024-09-12T12:33:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rook", 2024-09-12T12:33:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "storage_class": { 2024-09-12T12:33:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "storage_class", 2024-09-12T12:33:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "local", 2024-09-12T12:33:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T12:33:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "selftest", 2024-09-12T12:33:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "roption1": { 2024-09-12T12:33:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption1", 2024-09-12T12:33:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "roption2": { 2024-09-12T12:33:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption2", 2024-09-12T12:33:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "xyz", 2024-09-12T12:33:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption1": { 2024-09-12T12:33:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption1", 2024-09-12T12:33:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption2": { 2024-09-12T12:33:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption2", 2024-09-12T12:33:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption3": { 2024-09-12T12:33:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption3", 2024-09-12T12:33:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-09-12T12:33:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption4": { 2024-09-12T12:33:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption4", 2024-09-12T12:33:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption5": { 2024-09-12T12:33:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption5", 2024-09-12T12:33:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption6": { 2024-09-12T12:33:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption6", 2024-09-12T12:33:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption7": { 2024-09-12T12:33:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption7", 2024-09-12T12:33:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-09-12T12:33:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "42", 2024-09-12T12:33:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "testkey": { 2024-09-12T12:33:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testkey", 2024-09-12T12:33:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: "testlkey": { 2024-09-12T12:33:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testlkey", 2024-09-12T12:33:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.935 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.935 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.935 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.935 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.935 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.935 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.935 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.936 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.936 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.936 INFO:teuthology.orchestra.run.smithi137.stdout: "testnewline": { 2024-09-12T12:33:23.936 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testnewline", 2024-09-12T12:33:23.936 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.937 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.938 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.938 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.938 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.938 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.939 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-09-12T12:33:23.939 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.939 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.939 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.939 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_m_granularity": { 2024-09-12T12:33:23.940 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_m_granularity", 2024-09-12T12:33:23.940 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.940 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.940 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.940 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.940 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.940 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.941 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.941 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T12:33:23.941 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.941 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.941 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.942 INFO:teuthology.orchestra.run.smithi137.stdout: "dump_on_update": { 2024-09-12T12:33:23.942 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dump_on_update", 2024-09-12T12:33:23.942 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.942 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.942 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.942 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.942 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.943 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.943 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.943 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "dump database to debug log on update", 2024-09-12T12:33:23.943 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.943 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.943 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.944 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.944 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.944 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.944 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.944 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.944 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.945 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.945 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.945 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.945 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.945 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.945 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.945 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.947 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.947 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.947 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.947 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.947 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.947 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.947 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.948 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.948 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.948 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.948 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.948 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.948 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.949 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.949 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.949 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.949 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.949 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.949 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.949 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.950 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.950 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.950 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.950 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.950 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.950 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.950 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.951 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.952 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.952 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.952 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.952 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.952 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.952 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.952 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.953 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.953 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.953 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.953 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.953 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.953 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.954 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.954 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.954 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stats", 2024-09-12T12:33:23.954 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.954 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.954 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.955 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.955 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.955 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.955 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.955 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.955 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.955 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.956 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.956 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.956 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.956 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.956 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.956 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.956 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.957 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.957 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.957 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.957 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.957 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.957 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.958 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.958 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.958 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.958 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.958 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.958 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.959 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.959 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.959 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.959 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.959 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.959 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.959 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.960 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.960 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.960 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.960 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.960 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.960 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.961 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.961 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.961 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.961 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.961 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.961 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.961 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.962 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.962 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.962 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.962 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.962 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.962 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.962 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.963 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.963 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.963 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.963 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.963 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.964 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.964 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.964 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.964 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.964 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.964 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.965 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.965 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.965 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.965 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.965 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.965 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "status", 2024-09-12T12:33:23.966 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.966 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.966 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.966 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.966 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.966 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.966 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.967 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.967 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.967 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.967 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.967 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.967 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.967 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.968 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.968 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.968 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.968 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.968 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.968 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.968 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.969 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.969 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.969 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.969 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.969 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.969 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.970 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.971 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.972 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.972 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.972 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.972 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.972 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.972 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.972 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.973 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.974 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.974 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.974 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.974 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.974 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.974 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.975 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.975 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.975 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.975 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.975 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.975 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.975 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.976 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.976 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.976 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.976 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.976 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.976 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telegraf", 2024-09-12T12:33:23.977 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.977 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.977 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.977 INFO:teuthology.orchestra.run.smithi137.stdout: "address": { 2024-09-12T12:33:23.977 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "address", 2024-09-12T12:33:23.977 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.978 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.978 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.978 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T12:33:23.978 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.978 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.978 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.978 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.979 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.979 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.979 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.979 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:23.979 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:23.979 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:23.980 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.980 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.980 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15", 2024-09-12T12:33:23.980 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.980 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.980 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.980 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.981 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.981 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.981 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.981 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:23.981 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:23.982 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.982 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.982 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.982 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:23.982 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.982 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.982 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.983 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.983 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.983 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.983 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.983 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.983 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.983 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.984 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.984 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.984 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.984 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.984 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:23.984 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:23.985 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.985 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.985 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.985 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.985 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.985 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.986 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.986 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.986 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.986 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.986 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.987 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:23.987 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:23.987 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:23.987 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.987 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.987 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:23.988 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.988 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.988 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:23.988 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:23.988 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:23.988 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:23.988 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:23.989 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:23.989 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:23.989 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:23.989 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.989 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.989 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.989 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.990 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:23.990 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:23.990 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.990 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.990 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:23.990 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.991 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.991 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.991 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.991 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:23.991 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.991 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.991 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.992 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.992 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:23.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.992 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:23.992 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telemetry", 2024-09-12T12:33:23.992 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:23.992 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:23.993 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:23.993 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_basic": { 2024-09-12T12:33:23.993 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_basic", 2024-09-12T12:33:23.993 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.993 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.993 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.993 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.994 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.994 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.994 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.994 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T12:33:23.994 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.994 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.994 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_crash": { 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_crash", 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.995 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.996 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.996 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.996 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T12:33:23.996 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.996 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.996 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.997 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_device": { 2024-09-12T12:33:23.997 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_device", 2024-09-12T12:33:23.997 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.997 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.997 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.997 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:23.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:23.999 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_ident": { 2024-09-12T12:33:23.999 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_ident", 2024-09-12T12:33:23.999 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:23.999 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:23.999 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:23.999 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:23.999 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.000 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.000 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.000 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T12:33:24.000 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.000 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.000 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.001 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_perf": { 2024-09-12T12:33:24.001 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_perf", 2024-09-12T12:33:24.001 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.001 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.001 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.001 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.002 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.002 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.002 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.002 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T12:33:24.002 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.002 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.002 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.003 INFO:teuthology.orchestra.run.smithi137.stdout: "contact": { 2024-09-12T12:33:24.003 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "contact", 2024-09-12T12:33:24.003 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.003 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.003 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.003 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.004 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.004 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.004 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.004 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.004 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.004 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.004 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.005 INFO:teuthology.orchestra.run.smithi137.stdout: "description": { 2024-09-12T12:33:24.005 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "description", 2024-09-12T12:33:24.005 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.005 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.005 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.005 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.006 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.006 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.006 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.006 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.006 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.006 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.006 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.007 INFO:teuthology.orchestra.run.smithi137.stdout: "device_url": { 2024-09-12T12:33:24.007 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_url", 2024-09-12T12:33:24.007 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.007 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.007 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.007 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T12:33:24.008 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.008 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.008 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.008 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.008 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.008 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.008 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.009 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-09-12T12:33:24.009 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-09-12T12:33:24.009 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.009 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.009 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.009 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.010 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.010 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.010 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.010 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.010 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.010 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.010 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "24", 2024-09-12T12:33:24.011 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "8", 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.012 INFO:teuthology.orchestra.run.smithi137.stdout: "last_opt_revision": { 2024-09-12T12:33:24.013 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "last_opt_revision", 2024-09-12T12:33:24.013 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:24.013 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.013 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.013 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1", 2024-09-12T12:33:24.013 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.013 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.014 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.014 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.014 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.014 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.014 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.014 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard": { 2024-09-12T12:33:24.015 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard", 2024-09-12T12:33:24.015 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.015 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.015 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.015 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.015 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.015 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.016 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.016 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.016 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.016 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.016 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.017 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard_description": { 2024-09-12T12:33:24.017 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard_description", 2024-09-12T12:33:24.017 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.017 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.017 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.017 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.017 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.018 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.018 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.018 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.018 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.018 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.018 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.019 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:24.019 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:24.019 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.019 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.019 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.019 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.020 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.020 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.020 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.020 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.020 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.020 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.020 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.021 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.021 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.021 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.021 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.021 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.021 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.021 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.022 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:24.022 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:24.022 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.022 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.022 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.023 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.024 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:24.024 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:24.024 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.024 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.024 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.024 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:24.025 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.025 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.025 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.025 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.025 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.025 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.025 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.026 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.026 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.026 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.026 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.026 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.026 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.026 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.027 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:24.027 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:24.027 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.027 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.027 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.027 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.028 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.028 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.028 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.028 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.028 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.028 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.029 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.029 INFO:teuthology.orchestra.run.smithi137.stdout: "organization": { 2024-09-12T12:33:24.029 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "organization", 2024-09-12T12:33:24.029 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.029 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.030 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.030 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.030 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.030 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.030 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.030 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.030 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.031 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.031 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.031 INFO:teuthology.orchestra.run.smithi137.stdout: "proxy": { 2024-09-12T12:33:24.031 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "proxy", 2024-09-12T12:33:24.031 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.031 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.032 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.032 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.032 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.032 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.032 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.032 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: "url": { 2024-09-12T12:33:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url", 2024-09-12T12:33:24.033 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.034 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.034 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.034 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T12:33:24.034 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.034 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.034 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.034 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.035 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.035 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.035 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.036 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "test_orchestrator", 2024-09-12T12:33:24.036 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:24.036 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:24.036 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:24.036 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:24.036 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:24.036 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.037 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.037 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.037 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.037 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.037 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.037 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.037 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.038 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.038 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.038 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.038 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.038 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.038 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.039 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.039 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.039 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.039 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.039 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:24.039 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:24.040 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.040 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.040 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.040 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.040 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.040 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.041 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.041 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.041 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.041 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.041 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.041 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.042 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.043 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.044 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.044 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.044 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.044 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:24.044 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:24.045 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.045 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.045 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.045 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.045 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.045 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.045 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.046 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.046 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.046 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.046 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.046 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.046 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.047 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-09-12T12:33:24.047 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:24.047 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:24.047 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:24.047 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:24.048 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:24.048 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.048 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.048 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.048 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.048 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.048 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.049 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.049 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.049 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.049 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.049 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.049 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.050 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.050 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.050 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.050 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.050 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.050 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.051 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:24.051 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:24.051 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.051 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.051 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.051 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.052 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.052 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.052 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.052 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.052 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.052 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.052 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.053 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:24.053 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:24.053 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.053 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.053 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.053 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:24.054 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.054 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.054 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.054 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.054 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.054 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.054 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.055 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.055 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.055 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.055 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.055 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.055 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.055 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.056 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:24.056 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:24.056 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.056 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.056 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.056 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.057 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.057 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.057 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.057 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.057 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.057 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.057 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_clones": { 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_clones", 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4", 2024-09-12T12:33:24.058 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.059 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.059 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.059 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T12:33:24.059 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.059 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.059 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.060 INFO:teuthology.orchestra.run.smithi137.stdout: "periodic_async_work": { 2024-09-12T12:33:24.060 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "periodic_async_work", 2024-09-12T12:33:24.060 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.060 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.060 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.060 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Periodically check for async work", 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.062 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_delay": { 2024-09-12T12:33:24.062 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_delay", 2024-09-12T12:33:24.062 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:24.062 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.062 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.062 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-09-12T12:33:24.062 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.063 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.063 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.063 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T12:33:24.063 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.063 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.063 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.064 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_no_wait": { 2024-09-12T12:33:24.064 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T12:33:24.064 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.064 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.064 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.064 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-09-12T12:33:24.064 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.065 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.065 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.065 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T12:33:24.065 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.065 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.065 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.067 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.067 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.067 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix", 2024-09-12T12:33:24.067 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-09-12T12:33:24.067 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "discovery_interval": { 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "discovery_interval", 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "100", 2024-09-12T12:33:24.068 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.069 INFO:teuthology.orchestra.run.smithi137.stdout: "identifier": { 2024-09-12T12:33:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "identifier", 2024-09-12T12:33:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-09-12T12:33:24.071 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.072 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.073 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.074 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.076 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-09-12T12:33:24.077 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-09-12T12:33:24.078 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.079 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-09-12T12:33:24.080 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_host": { 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_host", 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.082 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_port": { 2024-09-12T12:33:24.083 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_port", 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10051", 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.084 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_sender": { 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_sender", 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-09-12T12:33:24.085 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-09-12T12:33:24.086 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: "services": { 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: "dashboard": "https://172.21.15.137:8443/" 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.087 INFO:teuthology.orchestra.run.smithi137.stdout: "always_on_modules": { 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "octopus": [ 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-09-12T12:33:24.088 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: "pacific": [ 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-09-12T12:33:24.089 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.090 INFO:teuthology.orchestra.run.smithi137.stdout: "quincy": [ 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-09-12T12:33:24.091 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: "reef": [ 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-09-12T12:33:24.092 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-09-12T12:33:24.093 INFO:teuthology.orchestra.run.smithi137.stdout: "squid": [ 2024-09-12T12:33:24.094 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-09-12T12:33:24.094 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-09-12T12:33:24.094 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-09-12T12:33:24.094 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-09-12T12:33:24.094 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-09-12T12:33:24.094 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-09-12T12:33:24.094 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: "last_failure_osd_epoch": 85, 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: "active_clients": [ 2024-09-12T12:33:24.095 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 3744227359 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.096 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "libcephsqlite", 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-09-12T12:33:24.097 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 2650674252 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.098 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 331467788 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-09-12T12:33:24.099 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 1538744155 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.100 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 225128983 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 2005502638 2024-09-12T12:33:24.102 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.103 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.103 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.103 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-09-12T12:33:24.103 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:33:24.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:21 smithi171 ceph-mon[35098]: from='client.17895 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:33:24.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:21 smithi171 ceph-mon[35098]: pgmap v640: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.9 MiB/s wr, 461 op/s 2024-09-12T12:33:24.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:21 smithi171 ceph-mon[35098]: from='client.17901 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:33:24.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:21 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/884305808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.105 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:21 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/515472396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.106 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:21 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1684177156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.106 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:22 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/4030338575' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T12:33:24.106 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:23 smithi171 ceph-mon[35098]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 342 op/s 2024-09-12T12:33:24.106 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:23 smithi171 ceph-mon[35098]: osdmap e120: 12 total, 12 up, 12 in 2024-09-12T12:33:24.106 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:21 smithi189 ceph-mon[35475]: from='client.17895 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:33:24.106 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:21 smithi189 ceph-mon[35475]: pgmap v640: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.9 MiB/s wr, 461 op/s 2024-09-12T12:33:24.106 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:21 smithi189 ceph-mon[35475]: from='client.17901 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:33:24.107 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:21 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/884305808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.107 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:21 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/515472396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.107 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:21 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1684177156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.107 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:22 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/4030338575' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T12:33:24.107 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:23 smithi189 ceph-mon[35475]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 342 op/s 2024-09-12T12:33:24.107 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:23 smithi189 ceph-mon[35475]: osdmap e120: 12 total, 12 up, 12 in 2024-09-12T12:33:24.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8eb40a2d00 msgr2=0x7f8eb40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:24.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8eb40a2d00 0x7f8eb40a51c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8006010 tx=0x7f8eb8005a20 comp rx=0 tx=0).stop 2024-09-12T12:33:24.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81003a0 msgr2=0x7f8ec8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:24.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81003a0 0x7f8ec8198440 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400e9f0 tx=0x7f8ec400edf0 comp rx=0 tx=0).stop 2024-09-12T12:33:24.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 shutdown_connections 2024-09-12T12:33:24.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f8eb40a2d00 0x7f8eb40a51c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:24.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec819cd00 0x7f8ec819d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:24.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ec81054f0 0x7f8ec8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:24.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 --2- 172.21.15.137:0/4030338575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ec81003a0 0x7f8ec8198440 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:24.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 >> 172.21.15.137:0/4030338575 conn(0x7f8ec80771e0 msgr2=0x7f8ec81035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:24.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 shutdown_connections 2024-09-12T12:33:24.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:21.611+0000 7f8ed0265640 1 -- 172.21.15.137:0/4030338575 wait complete. 2024-09-12T12:33:24.115 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T12:33:24.115 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T12:33:24.116 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:21 smithi137 ceph-mon[30950]: from='client.17895 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:33:24.116 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:21 smithi137 ceph-mon[30950]: pgmap v640: 97 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 80 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.9 MiB/s wr, 461 op/s 2024-09-12T12:33:24.116 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:21 smithi137 ceph-mon[30950]: from='client.17901 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T12:33:24.116 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:21 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/884305808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.117 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:21 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/515472396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.117 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:21 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1684177156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T12:33:24.117 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:22 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/4030338575' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T12:33:24.117 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:23 smithi137 ceph-mon[30950]: pgmap v641: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 342 op/s 2024-09-12T12:33:24.117 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:23 smithi137 ceph-mon[30950]: osdmap e120: 12 total, 12 up, 12 in 2024-09-12T12:33:24.409 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:24.945 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:33:24.945 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 368, 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 358, 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1484944, 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 221216, 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.054707559, 2024-09-12T12:33:24.946 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.022037426, 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.043170965, 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 357, 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.016407045, 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000045958 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.947 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1484944, 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 221216 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.948 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3, 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 275, 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 480, 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 284512, 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.018697225, 2024-09-12T12:33:24.949 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.013468535, 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002263168, 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 272, 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.017673769, 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000064977 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.950 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 284512 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:24.951 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 4, 2024-09-12T12:33:24.952 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 4, 2024-09-12T12:33:24.952 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3024, 2024-09-12T12:33:24.952 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 448, 2024-09-12T12:33:24.952 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-09-12T12:33:24.952 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-09-12T12:33:24.952 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.002233384, 2024-09-12T12:33:24.952 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000040995, 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002154273, 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.004033747, 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001344582 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:24.953 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-12T12:33:24.954 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:24.955 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:24.956 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:24.957 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:24.958 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:24.958 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:24.958 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:24.958 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:24.958 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:24.959 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:24.959 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:24.959 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:24.959 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:24.959 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-12T12:33:24.959 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148714, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 30, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:24.960 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-09-12T12:33:24.961 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:24.962 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:24.962 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:24.962 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-12T12:33:24.962 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-09-12T12:33:24.962 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-09-12T12:33:24.962 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:24.962 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-12T12:33:24.963 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 371, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1433000, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 371, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 371, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1433000, 2024-09-12T12:33:24.964 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-12T12:33:24.965 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:24.965 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:24.965 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:24.965 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.965 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:24.965 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:33:25.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:25 smithi137 ceph-mon[30950]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 342 op/s 2024-09-12T12:33:25.659 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-12T12:33:25.659 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-12T12:33:25.659 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-12T12:33:25.659 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-12T12:33:25.659 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-12T12:33:25.660 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-12T12:33:25.660 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-12T12:33:25.660 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T12:33:25.660 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T12:33:25.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:25 smithi189 ceph-mon[35475]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 342 op/s 2024-09-12T12:33:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:25.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:25 smithi171 ceph-mon[35098]: pgmap v643: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.9 MiB/s wr, 342 op/s 2024-09-12T12:33:26.471 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:33:26.471 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:26.471 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 377, 2024-09-12T12:33:26.471 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 365, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1489328, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 224864, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.058443598, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.023912909, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.045473007, 2024-09-12T12:33:26.472 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 365, 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.020491592, 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000056141 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:26.473 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:26.474 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:26.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.474 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1489328, 2024-09-12T12:33:26.474 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 224864 2024-09-12T12:33:26.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.474 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:26.474 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 279, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 480, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 285216, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.019443271, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.014328928, 2024-09-12T12:33:26.475 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002156454, 2024-09-12T12:33:26.476 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:26.476 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:26.476 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 276, 2024-09-12T12:33:26.476 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.022183944, 2024-09-12T12:33:26.476 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000080376 2024-09-12T12:33:26.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.476 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 285216 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:26.477 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-09-12T12:33:26.478 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-09-12T12:33:26.478 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-09-12T12:33:26.478 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-09-12T12:33:26.478 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-09-12T12:33:26.478 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-09-12T12:33:26.478 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.000320667, 2024-09-12T12:33:26.478 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000239746, 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:26.479 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-12T12:33:26.480 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:26.481 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:26.482 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:26.483 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:26.483 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:26.483 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:26.483 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:26.483 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:26.483 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:26.483 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:26.484 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:26.484 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:26.484 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:26.484 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:26.484 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:26.484 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:26.484 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:26.485 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:26.485 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:26.485 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:26.485 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:26.485 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-12T12:33:26.485 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:26.485 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:26.486 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:26.486 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:26.486 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148714, 2024-09-12T12:33:26.486 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 30, 2024-09-12T12:33:26.486 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:26.486 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:26.487 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-12T12:33:26.487 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-12T12:33:26.487 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:26.487 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:26.487 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:26.487 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:26.487 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-09-12T12:33:26.488 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-09-12T12:33:26.488 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:26.488 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:26.488 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:26.488 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-12T12:33:26.488 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-09-12T12:33:26.488 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-09-12T12:33:26.489 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:26.489 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-12T12:33:26.489 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:26.489 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-12T12:33:26.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:26.489 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 376, 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1433650, 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 376, 2024-09-12T12:33:26.490 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-12T12:33:26.491 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-12T12:33:26.491 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 376, 2024-09-12T12:33:26.491 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1433650, 2024-09-12T12:33:26.491 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-12T12:33:26.491 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:26.491 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:26.491 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:26.492 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:26.492 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:26.492 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:33:26.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:26 smithi137 ceph-mon[30950]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-09-12T12:33:26.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:26 smithi189 ceph-mon[35475]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-09-12T12:33:26.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:26 smithi171 ceph-mon[35098]: pgmap v644: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-09-12T12:33:27.235 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T12:33:27.235 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T12:33:27.236 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-12T12:33:27.236 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-12T12:33:27.236 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-12T12:33:27.236 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-12T12:33:27.236 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-12T12:33:27.236 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T12:33:27.237 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T12:33:27.494 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:28.053 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:33:28.053 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:28.053 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-09-12T12:33:28.053 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-09-12T12:33:28.053 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.001222386, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000285139, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:28.054 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:28.055 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3, 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 276, 2024-09-12T12:33:28.056 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 480, 2024-09-12T12:33:28.057 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 284688, 2024-09-12T12:33:28.057 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:28.057 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-09-12T12:33:28.057 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.018781434, 2024-09-12T12:33:28.057 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.013714433, 2024-09-12T12:33:28.057 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002176011, 2024-09-12T12:33:28.057 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:28.058 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:28.058 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 273, 2024-09-12T12:33:28.058 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.017008544, 2024-09-12T12:33:28.058 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000062302 2024-09-12T12:33:28.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.058 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:28.058 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 284688 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:28.059 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 374, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 362, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1488512, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 222944, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.054279726, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.022753155, 2024-09-12T12:33:28.060 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.042993971, 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 362, 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.021254626, 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000058714 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:28.061 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1488512, 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 222944 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-12T12:33:28.062 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:28.063 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:28.064 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:28.065 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:28.066 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:28.067 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:28.067 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:28.067 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-12T12:33:28.067 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:28.067 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:28.067 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:28.067 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 149745, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 31, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:28.068 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:28.069 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:28.069 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:28.069 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-09-12T12:33:28.069 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-09-12T12:33:28.069 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:28.069 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:28.069 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-12T12:33:28.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 373, 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1433260, 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:28.071 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 373, 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 373, 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1433260, 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:28.072 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:28.073 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:28.073 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:28.073 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:33:28.785 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-12T12:33:28.786 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-12T12:33:28.786 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-12T12:33:28.786 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-12T12:33:28.786 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-12T12:33:28.786 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-12T12:33:28.786 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-12T12:33:28.786 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T12:33:28.787 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T12:33:29.031 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:29.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:28 smithi137 ceph-mon[30950]: pgmap v645: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-09-12T12:33:29.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:28 smithi189 ceph-mon[35475]: pgmap v645: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-09-12T12:33:29.199 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:28 smithi171 ceph-mon[35098]: pgmap v645: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.4 MiB/s wr, 288 op/s 2024-09-12T12:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-09-12T12:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.001399017, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T12:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000235434, 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 379, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 367, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1489872, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 226144, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.057009069, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.022517796, 2024-09-12T12:33:29.552 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.046261655, 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 367, 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.020960975, 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000057114 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1489872, 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 226144 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 281, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 480, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 285568, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.021001752, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.015586299, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002331538, 2024-09-12T12:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 278, 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.019653277, 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000070695 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 285568 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-09-12T12:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-09-12T12:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148794, 2024-09-12T12:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 32, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:29.563 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-09-12T12:33:29.564 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-09-12T12:33:29.565 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 378, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1433910, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 378, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-09-12T12:33:29.566 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 378, 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1433910, 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-09-12T12:33:29.567 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-09-12T12:33:30.291 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T12:33:30.291 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T12:33:30.291 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-12T12:33:30.291 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-12T12:33:30.292 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-12T12:33:30.292 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-12T12:33:30.292 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-12T12:33:30.292 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T12:33:30.292 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T12:33:30.640 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:33:30.970 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:30 smithi171 ceph-mon[35098]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.5 MiB/s wr, 284 op/s 2024-09-12T12:33:31.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:30 smithi137 ceph-mon[30950]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.5 MiB/s wr, 284 op/s 2024-09-12T12:33:31.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:30 smithi189 ceph-mon[35475]: pgmap v646: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.5 MiB/s wr, 284 op/s 2024-09-12T12:33:31.721 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 58316, 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 58081, 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 15344143, 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 423382402, 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 261, 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-09-12T12:33:31.722 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 5.247649281, 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 2.313276974, 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.951074827, 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 1.629928303, 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 58075, 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 21.308295984, 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000366909 2024-09-12T12:33:31.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 638, 2024-09-12T12:33:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000079207, 2024-09-12T12:33:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000124 2024-09-12T12:33:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 139424, 2024-09-12T12:33:31.724 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 35168 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 749982, 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 923950, 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 216646869, 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 773966089, 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 10, 2024-09-12T12:33:31.725 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-09-12T12:33:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 34.586133587, 2024-09-12T12:33:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 14.486382231, 2024-09-12T12:33:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 22.956349874, 2024-09-12T12:33:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 1.576181542, 2024-09-12T12:33:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 923951, 2024-09-12T12:33:31.726 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 56.047415466, 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000060660 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1492691, 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.207843699, 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000139 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.727 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 659968 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 260011, 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 320982, 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 74910919, 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 291975214, 2024-09-12T12:33:31.728 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 20, 2024-09-12T12:33:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 7, 2024-09-12T12:33:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 14.946266955, 2024-09-12T12:33:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 6.352359757, 2024-09-12T12:33:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 8.451056131, 2024-09-12T12:33:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 1.226718154, 2024-09-12T12:33:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:31.729 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 320982, 2024-09-12T12:33:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 25.033487535, 2024-09-12T12:33:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000077990 2024-09-12T12:33:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 502383, 2024-09-12T12:33:31.730 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.073384644, 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000146 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 2315216, 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 450448 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-09-12T12:33:31.731 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-12T12:33:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-09-12T12:33:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-12T12:33:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-12T12:33:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 145740, 2024-09-12T12:33:31.732 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 122.829671785, 2024-09-12T12:33:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000842799 2024-09-12T12:33:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-09-12T12:33:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-12T12:33:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-12T12:33:31.733 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 55915, 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 7.040129779, 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000125907 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 300561, 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 268169, 2024-09-12T12:33:31.734 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-09-12T12:33:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 268169, 2024-09-12T12:33:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1838.805446513, 2024-09-12T12:33:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.006856890 2024-09-12T12:33:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-09-12T12:33:31.735 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 32392, 2024-09-12T12:33:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 20, 2024-09-12T12:33:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-09-12T12:33:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 4067, 2024-09-12T12:33:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 21, 2024-09-12T12:33:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 21, 2024-09-12T12:33:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 3103, 2024-09-12T12:33:31.736 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 1403, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 1678, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 22, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 1099, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 2783, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 396, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 396, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 13, 2024-09-12T12:33:31.737 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 472995, 2024-09-12T12:33:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 289018, 2024-09-12T12:33:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 32350, 2024-09-12T12:33:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 138, 2024-09-12T12:33:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 7, 2024-09-12T12:33:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-09-12T12:33:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 2366, 2024-09-12T12:33:31.738 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 236063, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 3, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 13, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 1877, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 25, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 746, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-09-12T12:33:31.739 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-09-12T12:33:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 695, 2024-09-12T12:33:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 211884559, 2024-09-12T12:33:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 6, 2024-09-12T12:33:31.740 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T12:33:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-09-12T12:33:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T12:33:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-09-12T12:33:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-09-12T12:33:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T12:33:31.741 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T12:33:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-09-12T12:33:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 32, 2024-09-12T12:33:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 46500, 2024-09-12T12:33:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 123383, 2024-09-12T12:33:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T12:33:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T12:33:31.742 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 115596, 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 27100, 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 13344, 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 174232 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 283, 2024-09-12T12:33:31.743 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 130, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 84, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 184, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 1364, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 182, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 76, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 33480, 2024-09-12T12:33:31.744 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 33298, 2024-09-12T12:33:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-09-12T12:33:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-09-12T12:33:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-09-12T12:33:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-09-12T12:33:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-09-12T12:33:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-09-12T12:33:31.745 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 18, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 3543, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 16, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 42, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-09-12T12:33:31.746 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 193016, 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 192072, 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 192072, 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 944, 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-09-12T12:33:31.747 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 1629, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 341, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 339, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 339, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 2, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 1, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 3, 2024-09-12T12:33:31.748 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 806907199, 2024-09-12T12:33:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 809529048, 2024-09-12T12:33:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2024-09-12T12:33:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-09-12T12:33:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 76605, 2024-09-12T12:33:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 383.078344512, 2024-09-12T12:33:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.005000696 2024-09-12T12:33:31.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 2016, 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 37168, 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 35152, 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 263, 2024-09-12T12:33:31.750 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 546, 2024-09-12T12:33:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 283, 2024-09-12T12:33:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 3110, 2024-09-12T12:33:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 71830, 2024-09-12T12:33:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 68720, 2024-09-12T12:33:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 306, 2024-09-12T12:33:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 37677, 2024-09-12T12:33:31.751 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 37371, 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 94300, 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 231712 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 420911, 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 44681, 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 300561, 2024-09-12T12:33:31.752 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 5198, 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 89255, 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 34211, 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 403.507263271, 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.011794664 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-09-12T12:33:31.753 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 36479, 2024-09-12T12:33:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 242.350933356, 2024-09-12T12:33:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.006643573 2024-09-12T12:33:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-09-12T12:33:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.754 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-09-12T12:33:31.755 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 74027, 2024-09-12T12:33:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 152.924077590, 2024-09-12T12:33:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002065787 2024-09-12T12:33:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-09-12T12:33:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.756 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-09-12T12:33:31.757 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-09-12T12:33:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.758 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-09-12T12:33:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.759 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-09-12T12:33:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 20, 2024-09-12T12:33:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.006883221, 2024-09-12T12:33:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000344161 2024-09-12T12:33:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-09-12T12:33:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 119, 2024-09-12T12:33:31.760 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.650332678, 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.005464980 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.761 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 20, 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 7.787331030, 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.389366551 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 16, 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.024063896, 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001503993 2024-09-12T12:33:31.762 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-09-12T12:33:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 79052, 2024-09-12T12:33:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 273.685579508, 2024-09-12T12:33:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.003462095 2024-09-12T12:33:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-09-12T12:33:31.763 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 8868, 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 328.543092257, 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.037048161 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.764 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1, 2024-09-12T12:33:31.765 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.005135486, 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.005135486 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 14, 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 11.170559247, 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.797897089 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.766 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-09-12T12:33:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-09-12T12:33:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 95, 2024-09-12T12:33:31.767 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 50.018025675, 2024-09-12T12:33:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.526505533 2024-09-12T12:33:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-09-12T12:33:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.768 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-09-12T12:33:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1760, 2024-09-12T12:33:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2.078410497, 2024-09-12T12:33:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001180915 2024-09-12T12:33:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-09-12T12:33:31.769 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-09-12T12:33:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 7, 2024-09-12T12:33:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.018329179, 2024-09-12T12:33:31.770 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002618454 2024-09-12T12:33:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-09-12T12:33:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.771 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-09-12T12:33:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 33480, 2024-09-12T12:33:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 366.035429622, 2024-09-12T12:33:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.010932957 2024-09-12T12:33:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-09-12T12:33:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-09-12T12:33:31.772 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-09-12T12:33:31.773 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 5, 2024-09-12T12:33:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 3, 2024-09-12T12:33:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-09-12T12:33:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-09-12T12:33:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 29230, 2024-09-12T12:33:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 14615, 2024-09-12T12:33:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 1190, 2024-09-12T12:33:31.774 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-09-12T12:33:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-12T12:33:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 480, 2024-09-12T12:33:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 480, 2024-09-12T12:33:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:31.775 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:31.776 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:31.777 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:31.778 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:31.779 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 3082017, 2024-09-12T12:33:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 2215, 2024-09-12T12:33:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 176, 2024-09-12T12:33:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 2, 2024-09-12T12:33:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-12T12:33:31.780 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8032, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 177, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:31.781 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 8704156, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 134187, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:31.782 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 137318, 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 843901255, 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 37, 2024-09-12T12:33:31.783 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 137318, 2024-09-12T12:33:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-09-12T12:33:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 137318, 2024-09-12T12:33:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 2232.109028820, 2024-09-12T12:33:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.016255035 2024-09-12T12:33:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-09-12T12:33:31.784 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 137318, 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 184850 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 137318, 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 981, 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 136337, 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-09-12T12:33:31.785 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 3747, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 18447, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 961, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 77566, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 881, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-09-12T12:33:31.786 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 21, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 34335, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 7, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 36894, 2024-09-12T12:33:31.787 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-09-12T12:33:31.788 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-09-12T12:33:31.789 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-09-12T12:33:31.790 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 120, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 38, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 12, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-09-12T12:33:31.791 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 9441, 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 40, 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 2510 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-09-12T12:33:31.792 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-09-12T12:33:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-09-12T12:33:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-09-12T12:33:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 3593 2024-09-12T12:33:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-09-12T12:33:31.793 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 10, 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 33298, 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 33298, 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.794 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 1068305, 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 210645416, 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 1068305, 2024-09-12T12:33:31.795 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 1068305, 2024-09-12T12:33:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 210645416, 2024-09-12T12:33:31.796 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-09-12T12:33:31.797 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 137318, 2024-09-12T12:33:31.798 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 847269601, 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 137318, 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 847269601, 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.799 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:31.800 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 137318, 2024-09-12T12:33:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 137318, 2024-09-12T12:33:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 137318, 2024-09-12T12:33:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 137318, 2024-09-12T12:33:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.801 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:31.802 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 33298, 2024-09-12T12:33:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 3368666, 2024-09-12T12:33:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 33298, 2024-09-12T12:33:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 961, 2024-09-12T12:33:31.803 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 3368666, 2024-09-12T12:33:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:31.804 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x561f186f8960": { 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 28376, 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 193016, 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 805334744, 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 193016, 2024-09-12T12:33:31.805 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 76605, 2024-09-12T12:33:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 805306368, 2024-09-12T12:33:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:31.806 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.807 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:31.807 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-12T12:33:32.982 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:32 smithi171 ceph-mon[35098]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 248 op/s 2024-09-12T12:33:32.984 INFO:tasks.check_counter:Daemon mds.b mds.exported=13 2024-09-12T12:33:32.984 INFO:tasks.check_counter:Daemon mds.b mds.imported=25 2024-09-12T12:33:32.984 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=283 2024-09-12T12:33:32.985 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=130 2024-09-12T12:33:32.985 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-12T12:33:32.985 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=20 2024-09-12T12:33:32.985 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=14 2024-09-12T12:33:32.985 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T12:33:32.985 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T12:33:33.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:32 smithi137 ceph-mon[30950]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 248 op/s 2024-09-12T12:33:33.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:32 smithi189 ceph-mon[35475]: pgmap v647: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 248 op/s 2024-09-12T12:33:33.254 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:33:33.922 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-12T12:33:33.922 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 4, 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 4, 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 3024, 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 448, 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.003143017, 2024-09-12T12:33:33.923 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000043134, 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.002216767, 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3, 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.004172100, 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001390700 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.924 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:33.925 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:33.925 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:33.925 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:33.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.925 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T12:33:33.925 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T12:33:33.925 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 3, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 280, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 480, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 285392, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.014795707, 2024-09-12T12:33:33.926 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.009958646, 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001981287, 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 277, 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.020224072, 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000073011 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:33.927 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 285392 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:33.928 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 374, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 364, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1486576, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 225056, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.052488410, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.021342420, 2024-09-12T12:33:33.929 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.045054890, 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 363, 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.016581523, 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000045679 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:33.930 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1486576, 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 225056 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-12T12:33:33.931 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-09-12T12:33:33.932 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-12T12:33:33.932 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.932 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-12T12:33:33.932 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:33.932 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:33.932 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:33.932 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:33.933 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:33.933 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:33.933 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:33.936 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:33.937 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:33.938 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:33.938 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:33.938 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:33.938 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:33.938 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:33.938 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:33.938 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148682, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 29, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:33.939 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-09-12T12:33:33.940 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:33.941 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 377, 2024-09-12T12:33:33.942 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1433780, 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 377, 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 377, 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1433780, 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:33.943 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:33.944 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:33.944 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:33.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:33.944 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:33.944 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-12T12:33:34.807 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-09-12T12:33:34.807 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-09-12T12:33:34.807 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-09-12T12:33:34.807 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-09-12T12:33:34.807 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-12T12:33:34.807 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-12T12:33:34.807 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-12T12:33:34.808 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T12:33:34.808 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T12:33:35.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:34 smithi137 ceph-mon[30950]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 227 op/s 2024-09-12T12:33:35.072 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:33:35.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:34 smithi189 ceph-mon[35475]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 227 op/s 2024-09-12T12:33:35.193 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:34 smithi171 ceph-mon[35098]: pgmap v648: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 227 op/s 2024-09-12T12:33:35.672 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-12T12:33:35.672 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:35.672 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 2899, 2024-09-12T12:33:35.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2901, 2024-09-12T12:33:35.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 189599482, 2024-09-12T12:33:35.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 921889, 2024-09-12T12:33:35.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-09-12T12:33:35.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-09-12T12:33:35.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.319414855, 2024-09-12T12:33:35.673 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.073081881, 2024-09-12T12:33:35.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.158928903, 2024-09-12T12:33:35.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.067938468, 2024-09-12T12:33:35.674 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:35.674 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2899, 2024-09-12T12:33:35.674 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.096646631, 2024-09-12T12:33:35.674 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000033337 2024-09-12T12:33:35.674 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.675 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:35.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.675 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.675 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.675 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.675 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:35.676 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:35.676 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.676 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:35.676 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1055, 2024-09-12T12:33:35.676 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1329, 2024-09-12T12:33:35.676 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 183944694, 2024-09-12T12:33:35.676 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1076686, 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 10, 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 7, 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.183988224, 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.029926414, 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.122897171, 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.023006091, 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:35.677 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1327, 2024-09-12T12:33:35.678 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.047978056, 2024-09-12T12:33:35.678 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000036155 2024-09-12T12:33:35.678 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.678 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:35.678 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.678 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.678 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.679 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-09-12T12:33:35.679 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 743360 2024-09-12T12:33:35.680 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.680 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:35.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1833, 2024-09-12T12:33:35.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1789, 2024-09-12T12:33:35.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 194984644, 2024-09-12T12:33:35.680 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 671730, 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.252732307, 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.047612051, 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.176926245, 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.036040450, 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:35.681 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1788, 2024-09-12T12:33:35.682 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.056395710, 2024-09-12T12:33:35.682 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000031541 2024-09-12T12:33:35.682 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.682 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:35.682 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.682 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.682 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1662752, 2024-09-12T12:33:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 231936 2024-09-12T12:33:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-12T12:33:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-09-12T12:33:35.683 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-12T12:33:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-09-12T12:33:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-12T12:33:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-12T12:33:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 8672, 2024-09-12T12:33:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.184793153, 2024-09-12T12:33:35.684 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000021309 2024-09-12T12:33:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-09-12T12:33:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-12T12:33:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-12T12:33:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.685 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-09-12T12:33:35.686 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-09-12T12:33:35.687 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-09-12T12:33:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-09-12T12:33:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-09-12T12:33:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-09-12T12:33:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-09-12T12:33:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-09-12T12:33:35.688 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 23640, 2024-09-12T12:33:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 2798, 2024-09-12T12:33:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 20842, 2024-09-12T12:33:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-09-12T12:33:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 19643, 2024-09-12T12:33:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-09-12T12:33:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-09-12T12:33:35.689 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 6, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-09-12T12:33:35.690 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-09-12T12:33:35.691 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-09-12T12:33:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-09-12T12:33:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-09-12T12:33:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-09-12T12:33:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-09-12T12:33:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T12:33:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-09-12T12:33:35.692 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T12:33:35.693 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-09-12T12:33:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T12:33:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T12:33:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T12:33:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-09-12T12:33:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-09-12T12:33:35.694 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-09-12T12:33:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-09-12T12:33:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-09-12T12:33:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-09-12T12:33:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-09-12T12:33:35.695 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-09-12T12:33:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-09-12T12:33:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-09-12T12:33:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-09-12T12:33:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-09-12T12:33:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-09-12T12:33:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-09-12T12:33:35.696 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-09-12T12:33:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-09-12T12:33:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-09-12T12:33:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-09-12T12:33:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-09-12T12:33:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-09-12T12:33:35.697 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-09-12T12:33:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-09-12T12:33:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-09-12T12:33:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-09-12T12:33:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T12:33:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-09-12T12:33:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-09-12T12:33:35.698 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-09-12T12:33:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-09-12T12:33:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0 2024-09-12T12:33:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-09-12T12:33:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-09-12T12:33:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-09-12T12:33:35.699 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-09-12T12:33:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-09-12T12:33:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 139729, 2024-09-12T12:33:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-09-12T12:33:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-09-12T12:33:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-09-12T12:33:35.700 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-09-12T12:33:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-09-12T12:33:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 135, 2024-09-12T12:33:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 13, 2024-09-12T12:33:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-09-12T12:33:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-09-12T12:33:35.701 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 69941494, 2024-09-12T12:33:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 569522446, 2024-09-12T12:33:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 569511323, 2024-09-12T12:33:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-09-12T12:33:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.702 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 139729 2024-09-12T12:33:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-09-12T12:33:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 2104, 2024-09-12T12:33:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 23764, 2024-09-12T12:33:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 21660, 2024-09-12T12:33:35.703 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 122, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 215, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 93, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 23640, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 24207, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 567, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-09-12T12:33:35.704 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-09-12T12:33:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 82652, 2024-09-12T12:33:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 215328 2024-09-12T12:33:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-09-12T12:33:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-09-12T12:33:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.706 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-09-12T12:33:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-09-12T12:33:35.707 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-09-12T12:33:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.708 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-09-12T12:33:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.709 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-09-12T12:33:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-09-12T12:33:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.710 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-09-12T12:33:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.711 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-09-12T12:33:35.712 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-09-12T12:33:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.713 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-09-12T12:33:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.714 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-09-12T12:33:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-09-12T12:33:35.715 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.716 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-09-12T12:33:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-09-12T12:33:35.717 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-09-12T12:33:35.718 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-09-12T12:33:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.719 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-09-12T12:33:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.720 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-09-12T12:33:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.721 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-09-12T12:33:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-09-12T12:33:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.722 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-09-12T12:33:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.723 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-09-12T12:33:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-09-12T12:33:35.724 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-09-12T12:33:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.725 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-09-12T12:33:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.726 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-09-12T12:33:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-09-12T12:33:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-09-12T12:33:35.727 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-09-12T12:33:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-09-12T12:33:35.728 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-09-12T12:33:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 3, 2024-09-12T12:33:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 1, 2024-09-12T12:33:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 3, 2024-09-12T12:33:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-09-12T12:33:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-09-12T12:33:35.729 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-09-12T12:33:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 3121, 2024-09-12T12:33:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-09-12T12:33:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-12T12:33:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:35.730 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:35.731 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:35.732 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:35.733 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:35.734 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-12T12:33:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 2290778, 2024-09-12T12:33:35.735 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 2497, 2024-09-12T12:33:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 88, 2024-09-12T12:33:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 1, 2024-09-12T12:33:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-12T12:33:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-12T12:33:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:35.736 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 8032, 2024-09-12T12:33:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 177, 2024-09-12T12:33:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:35.737 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-12T12:33:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 22103075, 2024-09-12T12:33:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 169262, 2024-09-12T12:33:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-12T12:33:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-12T12:33:35.738 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-09-12T12:33:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-09-12T12:33:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-09-12T12:33:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 5330, 2024-09-12T12:33:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-09-12T12:33:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 3, 2024-09-12T12:33:35.739 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 5330, 2024-09-12T12:33:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-09-12T12:33:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5330, 2024-09-12T12:33:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 7.087853895, 2024-09-12T12:33:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001329803 2024-09-12T12:33:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.740 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-09-12T12:33:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-09-12T12:33:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5330, 2024-09-12T12:33:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 5331 2024-09-12T12:33:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 5330, 2024-09-12T12:33:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 5330, 2024-09-12T12:33:35.741 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-09-12T12:33:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-09-12T12:33:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-09-12T12:33:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 3123, 2024-09-12T12:33:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-09-12T12:33:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 2205, 2024-09-12T12:33:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-09-12T12:33:35.742 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-09-12T12:33:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-09-12T12:33:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-09-12T12:33:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-09-12T12:33:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-09-12T12:33:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-09-12T12:33:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-09-12T12:33:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-09-12T12:33:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-09-12T12:33:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-09-12T12:33:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-09-12T12:33:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-09-12T12:33:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-09-12T12:33:35.744 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-09-12T12:33:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-09-12T12:33:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-09-12T12:33:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-09-12T12:33:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T12:33:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-09-12T12:33:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-09-12T12:33:35.745 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-09-12T12:33:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-09-12T12:33:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-09-12T12:33:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-09-12T12:33:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-09-12T12:33:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-09-12T12:33:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-09-12T12:33:35.746 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-09-12T12:33:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-09-12T12:33:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-09-12T12:33:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-09-12T12:33:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-09-12T12:33:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-09-12T12:33:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-09-12T12:33:35.747 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-09-12T12:33:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-09-12T12:33:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-09-12T12:33:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 120, 2024-09-12T12:33:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-09-12T12:33:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 38, 2024-09-12T12:33:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 11, 2024-09-12T12:33:35.748 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2024-09-12T12:33:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 1, 2024-09-12T12:33:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-09-12T12:33:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-09-12T12:33:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-09-12T12:33:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-09-12T12:33:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.749 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-09-12T12:33:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-09-12T12:33:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-09-12T12:33:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-09-12T12:33:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-09-12T12:33:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "osd-slow-ops": { 2024-09-12T12:33:35.750 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-09-12T12:33:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-09-12T12:33:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-09-12T12:33:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T12:33:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-09-12T12:33:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-09-12T12:33:35.751 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-09-12T12:33:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-09-12T12:33:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-09-12T12:33:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-12T12:33:35.752 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 5783, 2024-09-12T12:33:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 567982526, 2024-09-12T12:33:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 5783, 2024-09-12T12:33:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:35.753 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 5783, 2024-09-12T12:33:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 567982526, 2024-09-12T12:33:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.754 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-09-12T12:33:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-12T12:33:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:35.755 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-12T12:33:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-12T12:33:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 5330, 2024-09-12T12:33:35.756 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 565394829, 2024-09-12T12:33:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 5330, 2024-09-12T12:33:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 565394829, 2024-09-12T12:33:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:35.757 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-09-12T12:33:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:35.758 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-09-12T12:33:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-12T12:33:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-12T12:33:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 5330, 2024-09-12T12:33:35.759 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 5330, 2024-09-12T12:33:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 5330, 2024-09-12T12:33:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 5330, 2024-09-12T12:33:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.760 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-09-12T12:33:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-12T12:33:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-12T12:33:35.761 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-12T12:33:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-12T12:33:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-12T12:33:35.762 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x5639ac3cab40": { 2024-09-12T12:33:35.763 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:35.764 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-12T12:33:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-12T12:33:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:35.765 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.766 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:35.766 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-12T12:33:36.504 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T12:33:36.505 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T12:33:36.505 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-12T12:33:36.505 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-09-12T12:33:36.505 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-09-12T12:33:36.505 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-12T12:33:36.505 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T12:33:36.505 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T12:33:36.506 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T12:33:36.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:36 smithi137 ceph-mon[30950]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 260 op/s 2024-09-12T12:33:36.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:36 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:33:36.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:36 smithi171 ceph-mon[35098]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 260 op/s 2024-09-12T12:33:36.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:36 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:33:37.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:36 smithi189 ceph-mon[35475]: pgmap v649: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 260 op/s 2024-09-12T12:33:37.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:36 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T12:33:37.111 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T12:33:37.961 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-12T12:33:37.961 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:37.961 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 382, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 370, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 1490688, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 228064, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 4, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 0, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.057809520, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.022312805, 2024-09-12T12:33:37.962 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.048286985, 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 370, 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.021111879, 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000057059 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:37.963 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1490688, 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 228064 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:37.964 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 3, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 283, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 480, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 285920, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.015467166, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.010792497, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.001916359, 2024-09-12T12:33:37.965 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 280, 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.017814986, 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000063624 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:37.966 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 285920 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 0, 2024-09-12T12:33:37.967 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 0, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 0, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 2, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 1, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.000197088, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.000117284, 2024-09-12T12:33:37.968 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:37.969 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 0, 2024-09-12T12:33:37.970 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:37.971 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:37.972 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:37.973 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:37.974 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 148682, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 29, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:37.975 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:37.976 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-12T12:33:37.976 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-12T12:33:37.976 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:37.976 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:37.976 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:37.976 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:37.976 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 0, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 0, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 928, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 2, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:37.977 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-12T12:33:37.978 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 381, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 1434300, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 381, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 381, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 1434300, 2024-09-12T12:33:37.979 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-12T12:33:37.980 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-12T12:33:37.980 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-12T12:33:37.980 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-12T12:33:37.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:37.980 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-12T12:33:37.980 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-12T12:33:38.666 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.666 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.666 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 3.7 MiB/s wr, 147 op/s 2024-09-12T12:33:38.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:33:38.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:33:38.667 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.668 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:38 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.693 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-12T12:33:38.693 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-12T12:33:38.693 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-12T12:33:38.693 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-12T12:33:38.694 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-12T12:33:38.694 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-12T12:33:38.694 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-12T12:33:38.694 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T12:33:38.694 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T12:33:38.783 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.783 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.783 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 3.7 MiB/s wr, 147 op/s 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.784 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:38 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: pgmap v650: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 3.7 MiB/s wr, 147 op/s 2024-09-12T12:33:38.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:38.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi137 to 176.0M 2024-09-12T12:33:38.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi137 to 184563814: error parsing value: Value '184563814' is below minimum 939524096 2024-09-12T12:33:38.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:38.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:38 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.007 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:33:39.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.670 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:33:39.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:33:39.671 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:33:39.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:33:39.672 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:39 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.700 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-12T12:33:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 452469, 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 576986, 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 143823341, 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 559582738, 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 14, 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 25.343116353, 2024-09-12T12:33:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 11.220782109, 2024-09-12T12:33:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 16.342948126, 2024-09-12T12:33:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.923272048, 2024-09-12T12:33:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 576984, 2024-09-12T12:33:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 34.514852630, 2024-09-12T12:33:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000059819 2024-09-12T12:33:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 911678, 2024-09-12T12:33:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.163844328, 2024-09-12T12:33:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000179 2024-09-12T12:33:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-12T12:33:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 736 2024-09-12T12:33:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 33921, 2024-09-12T12:33:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 33954, 2024-09-12T12:33:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 8519692, 2024-09-12T12:33:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 264034253, 2024-09-12T12:33:39.704 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 16, 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 8, 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 3.669957932, 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 1.575875102, 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.567308608, 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 1.350262523, 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 33952, 2024-09-12T12:33:39.705 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 2.458242448, 2024-09-12T12:33:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000072403 2024-09-12T12:33:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 31, 2024-09-12T12:33:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000004946, 2024-09-12T12:33:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000159 2024-09-12T12:33:39.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 6752, 2024-09-12T12:33:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 134080 2024-09-12T12:33:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 320384, 2024-09-12T12:33:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 286208, 2024-09-12T12:33:39.707 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 99669638, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 251729418, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 15.739048208, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 6.231052183, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 10.473257051, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 1.037478728, 2024-09-12T12:33:39.708 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 286207, 2024-09-12T12:33:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 11.982995160, 2024-09-12T12:33:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000041868 2024-09-12T12:33:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 552522, 2024-09-12T12:33:39.709 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.086954429, 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000157 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1670528, 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 767728 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-09-12T12:33:39.710 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-12T12:33:39.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.711 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-09-12T12:33:39.711 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:39.711 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:39.711 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 86732, 2024-09-12T12:33:39.712 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 83.527463168, 2024-09-12T12:33:39.712 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000963052 2024-09-12T12:33:39.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.712 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-09-12T12:33:39.712 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:39.712 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:39.713 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 50917, 2024-09-12T12:33:39.713 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 6.432153361, 2024-09-12T12:33:39.713 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000126326 2024-09-12T12:33:39.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.713 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-09-12T12:33:39.713 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 231199, 2024-09-12T12:33:39.714 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 198261, 2024-09-12T12:33:39.714 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-09-12T12:33:39.714 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 198261, 2024-09-12T12:33:39.714 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1077.928097764, 2024-09-12T12:33:39.714 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.005436914 2024-09-12T12:33:39.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.714 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-09-12T12:33:39.715 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 32938, 2024-09-12T12:33:39.715 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 1, 2024-09-12T12:33:39.715 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-09-12T12:33:39.715 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 138, 2024-09-12T12:33:39.715 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 14, 2024-09-12T12:33:39.715 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 14, 2024-09-12T12:33:39.715 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 3105, 2024-09-12T12:33:39.716 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 966, 2024-09-12T12:33:39.716 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 2059, 2024-09-12T12:33:39.716 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 80, 2024-09-12T12:33:39.716 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 1601, 2024-09-12T12:33:39.716 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 284, 2024-09-12T12:33:39.716 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 260, 2024-09-12T12:33:39.717 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 260, 2024-09-12T12:33:39.717 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 12, 2024-09-12T12:33:39.717 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 418499, 2024-09-12T12:33:39.717 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 196700, 2024-09-12T12:33:39.717 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 32926, 2024-09-12T12:33:39.718 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 135, 2024-09-12T12:33:39.718 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 1, 2024-09-12T12:33:39.718 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-09-12T12:33:39.718 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 4429, 2024-09-12T12:33:39.718 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 31235, 2024-09-12T12:33:39.718 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-09-12T12:33:39.718 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 16, 2024-09-12T12:33:39.719 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 196, 2024-09-12T12:33:39.719 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 7, 2024-09-12T12:33:39.719 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 732, 2024-09-12T12:33:39.719 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-09-12T12:33:39.719 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-09-12T12:33:39.719 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-09-12T12:33:39.720 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 703, 2024-09-12T12:33:39.720 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 211987629, 2024-09-12T12:33:39.720 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-09-12T12:33:39.720 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T12:33:39.720 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-09-12T12:33:39.720 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T12:33:39.720 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 28, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 11758, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 114016, 2024-09-12T12:33:39.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 67701, 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 23586, 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 7945, 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 170838 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.722 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 446, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 909, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 514, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 811, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 604, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 89, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-09-12T12:33:39.723 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 31423, 2024-09-12T12:33:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 31335, 2024-09-12T12:33:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-09-12T12:33:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-09-12T12:33:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-09-12T12:33:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-09-12T12:33:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-09-12T12:33:39.724 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 18, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 1983, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 23, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 28, 2024-09-12T12:33:39.725 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-09-12T12:33:39.726 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-09-12T12:33:39.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.726 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-09-12T12:33:39.726 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-09-12T12:33:39.726 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 163910, 2024-09-12T12:33:39.726 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 79394, 2024-09-12T12:33:39.726 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 73741, 2024-09-12T12:33:39.727 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 90169, 2024-09-12T12:33:39.727 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-09-12T12:33:39.727 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 5653, 2024-09-12T12:33:39.727 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 227, 2024-09-12T12:33:39.727 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 99, 2024-09-12T12:33:39.727 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 93, 2024-09-12T12:33:39.727 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 134, 2024-09-12T12:33:39.728 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 8, 2024-09-12T12:33:39.728 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-09-12T12:33:39.728 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 6, 2024-09-12T12:33:39.728 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 136110416, 2024-09-12T12:33:39.728 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 629796572, 2024-09-12T12:33:39.728 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 46510, 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 232.872827952, 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.005006941 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.729 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-09-12T12:33:39.730 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 1742, 2024-09-12T12:33:39.730 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 33281, 2024-09-12T12:33:39.730 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 31539, 2024-09-12T12:33:39.730 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 224, 2024-09-12T12:33:39.730 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 651, 2024-09-12T12:33:39.730 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 427, 2024-09-12T12:33:39.731 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 3105, 2024-09-12T12:33:39.731 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 52167, 2024-09-12T12:33:39.731 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 49062, 2024-09-12T12:33:39.731 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 259, 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 34446, 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 34187, 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 85320, 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 223520 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 363232, 2024-09-12T12:33:39.732 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 156, 2024-09-12T12:33:39.733 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 231199, 2024-09-12T12:33:39.733 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 4011, 2024-09-12T12:33:39.733 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 35921, 2024-09-12T12:33:39.733 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-09-12T12:33:39.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 31428, 2024-09-12T12:33:39.733 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 366.413858376, 2024-09-12T12:33:39.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.011658834 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 8489, 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 84.591764228, 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.009964867 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-09-12T12:33:39.734 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 51513, 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 117.000940690, 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.002271289 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.736 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.737 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.738 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.739 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-09-12T12:33:39.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.740 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.740 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-09-12T12:33:39.740 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.740 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.741 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-09-12T12:33:39.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.741 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-09-12T12:33:39.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.743 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.743 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-09-12T12:33:39.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.743 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 69411, 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 153.542637327, 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.002212079 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-09-12T12:33:39.744 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 5419, 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 108.670706816, 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.020053645 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.746 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.747 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 98, 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 24.716308078, 2024-09-12T12:33:39.748 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.252207225 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 480, 2024-09-12T12:33:39.749 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.976777639, 2024-09-12T12:33:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.002034953 2024-09-12T12:33:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-09-12T12:33:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.751 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-09-12T12:33:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 31423, 2024-09-12T12:33:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 222.015104610, 2024-09-12T12:33:39.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.007065369 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 3, 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 1, 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-09-12T12:33:39.754 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 25682, 2024-09-12T12:33:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 25682, 2024-09-12T12:33:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 1226, 2024-09-12T12:33:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-09-12T12:33:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-12T12:33:39.755 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:39.756 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:39.757 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:39.758 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:39.759 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-12T12:33:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:39.760 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 429043, 2024-09-12T12:33:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 1099, 2024-09-12T12:33:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 264, 2024-09-12T12:33:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 3, 2024-09-12T12:33:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-12T12:33:39.761 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8032, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 177, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:39.762 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-12T12:33:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 7048322, 2024-09-12T12:33:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 72530, 2024-09-12T12:33:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-12T12:33:39.763 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 81910, 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 632972233, 2024-09-12T12:33:39.764 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 31, 2024-09-12T12:33:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 81910, 2024-09-12T12:33:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-09-12T12:33:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 81910, 2024-09-12T12:33:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 420.189467787, 2024-09-12T12:33:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.005129892 2024-09-12T12:33:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.765 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-09-12T12:33:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-09-12T12:33:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 81910, 2024-09-12T12:33:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 83434 2024-09-12T12:33:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 81910, 2024-09-12T12:33:39.766 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 1113, 2024-09-12T12:33:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 80797, 2024-09-12T12:33:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-09-12T12:33:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-09-12T12:33:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 1, 2024-09-12T12:33:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 21, 2024-09-12T12:33:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 1112, 2024-09-12T12:33:39.767 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 47622, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 609, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 1, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 14, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 31596, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-09-12T12:33:39.768 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-09-12T12:33:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-09-12T12:33:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-09-12T12:33:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 42, 2024-09-12T12:33:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-09-12T12:33:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-09-12T12:33:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-09-12T12:33:39.769 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-09-12T12:33:39.770 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-09-12T12:33:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-09-12T12:33:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-09-12T12:33:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-09-12T12:33:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-09-12T12:33:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-09-12T12:33:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-09-12T12:33:39.771 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-09-12T12:33:39.772 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-09-12T12:33:39.772 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-09-12T12:33:39.772 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-09-12T12:33:39.772 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-09-12T12:33:39.772 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-09-12T12:33:39.772 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-09-12T12:33:39.773 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-09-12T12:33:39.773 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 120, 2024-09-12T12:33:39.773 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-09-12T12:33:39.773 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 38, 2024-09-12T12:33:39.773 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 12, 2024-09-12T12:33:39.773 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 12, 2024-09-12T12:33:39.773 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-09-12T12:33:39.774 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-09-12T12:33:39.774 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 1767, 2024-09-12T12:33:39.774 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-09-12T12:33:39.774 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 646 2024-09-12T12:33:39.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.774 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-09-12T12:33:39.774 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 2022 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.775 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 10, 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 31335, 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 31335, 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-09-12T12:33:39.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.777 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:39.777 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:39.777 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:39.777 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:39.777 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 806770, 2024-09-12T12:33:39.777 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 179375419, 2024-09-12T12:33:39.777 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:39.778 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 806770, 2024-09-12T12:33:39.778 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:39.778 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:39.778 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 806770, 2024-09-12T12:33:39.778 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 179375419, 2024-09-12T12:33:39.779 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:39.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.779 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.779 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:39.780 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 81910, 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 636141484, 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 81910, 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 636141484, 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.782 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-09-12T12:33:39.782 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:39.782 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-09-12T12:33:39.782 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:39.782 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:39.782 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:39.783 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:39.783 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:39.783 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 81910, 2024-09-12T12:33:39.783 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 81910, 2024-09-12T12:33:39.783 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 81910, 2024-09-12T12:33:39.783 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 81910, 2024-09-12T12:33:39.783 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:39.784 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:39.785 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:39.785 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 31335, 2024-09-12T12:33:39.785 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 3169267, 2024-09-12T12:33:39.785 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:39.785 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 31335, 2024-09-12T12:33:39.785 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:39.785 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1112, 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 3169267, 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5629dcc4e960": { 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 1961, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 163910, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 625602268, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 163910, 2024-09-12T12:33:39.787 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 46510, 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 625600307, 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:39.788 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:39.789 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-12T12:33:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:33:39.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:33:39.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:33:39.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:33:39.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:33:39.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:33:39.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:39 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi189 to 176.0M 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi189 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T12:33:39.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: Adjusting osd_memory_target on smithi171 to 176.0M 2024-09-12T12:33:39.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: Unable to set osd_memory_target on smithi171 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-12T12:33:39.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T12:33:39.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T12:33:39.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:39 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' 2024-09-12T12:33:40.773 INFO:tasks.check_counter:Daemon mds.c mds.exported=16 2024-09-12T12:33:40.773 INFO:tasks.check_counter:Daemon mds.c mds.imported=7 2024-09-12T12:33:40.773 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=446 2024-09-12T12:33:40.773 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=909 2024-09-12T12:33:40.773 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-12T12:33:40.773 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-12T12:33:40.773 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T12:33:40.774 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T12:33:40.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T12:33:40.796 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:40 smithi189 ceph-mon[35475]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 3.7 MiB/s wr, 148 op/s 2024-09-12T12:33:40.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:40 smithi137 ceph-mon[30950]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 3.7 MiB/s wr, 148 op/s 2024-09-12T12:33:40.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:40 smithi171 ceph-mon[35098]: pgmap v651: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 3.7 MiB/s wr, 148 op/s 2024-09-12T12:33:41.038 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:33:41.749 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-12T12:33:41.750 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:41.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 172, 2024-09-12T12:33:41.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 174, 2024-09-12T12:33:41.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 29105146, 2024-09-12T12:33:41.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 54702, 2024-09-12T12:33:41.750 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-09-12T12:33:41.751 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-09-12T12:33:41.751 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.031767015, 2024-09-12T12:33:41.751 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.004136302, 2024-09-12T12:33:41.751 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.021012340, 2024-09-12T12:33:41.751 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.004195117, 2024-09-12T12:33:41.751 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:41.751 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 172, 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.011021703, 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000064079 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.753 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:41.753 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:41.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.753 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:41.753 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 250, 2024-09-12T12:33:41.753 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 278, 2024-09-12T12:33:41.753 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 29438845, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 214171, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 6, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.045863102, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.007205781, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.029453191, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.007029929, 2024-09-12T12:33:41.754 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:41.755 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 276, 2024-09-12T12:33:41.755 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.022549973, 2024-09-12T12:33:41.755 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000081702 2024-09-12T12:33:41.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.755 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:41.755 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.755 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.756 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.756 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T12:33:41.756 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 134928 2024-09-12T12:33:41.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.756 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:41.756 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 196, 2024-09-12T12:33:41.757 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 193, 2024-09-12T12:33:41.757 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 29574025, 2024-09-12T12:33:41.757 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 70046, 2024-09-12T12:33:41.757 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-09-12T12:33:41.757 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-09-12T12:33:41.757 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.036162607, 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.004878477, 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.037621829, 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.003414500, 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 193, 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.017340844, 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000089848 2024-09-12T12:33:41.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.759 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:41.759 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.759 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.759 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.759 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 174064, 2024-09-12T12:33:41.759 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 24880 2024-09-12T12:33:41.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.760 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-12T12:33:41.760 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-09-12T12:33:41.760 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-12T12:33:41.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.760 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-09-12T12:33:41.760 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:41.761 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:41.761 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 894, 2024-09-12T12:33:41.761 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.061984746, 2024-09-12T12:33:41.761 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000069334 2024-09-12T12:33:41.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.761 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-09-12T12:33:41.762 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:41.762 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:41.762 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.762 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-09-12T12:33:41.763 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-09-12T12:33:41.764 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 5280, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 3247, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 2033, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 658, 2024-09-12T12:33:41.765 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-09-12T12:33:41.766 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-09-12T12:33:41.766 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-09-12T12:33:41.766 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 11, 2024-09-12T12:33:41.766 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-09-12T12:33:41.766 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-09-12T12:33:41.766 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-09-12T12:33:41.766 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-09-12T12:33:41.767 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-09-12T12:33:41.767 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-09-12T12:33:41.767 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-09-12T12:33:41.767 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-09-12T12:33:41.767 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-09-12T12:33:41.768 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 703, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 211987629, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-09-12T12:33:41.769 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T12:33:41.770 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T12:33:41.770 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-09-12T12:33:41.770 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-09-12T12:33:41.770 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T12:33:41.770 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-09-12T12:33:41.770 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T12:33:41.771 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T12:33:41.771 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T12:33:41.771 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-09-12T12:33:41.771 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-09-12T12:33:41.771 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-09-12T12:33:41.771 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-09-12T12:33:41.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.772 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-09-12T12:33:41.772 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-09-12T12:33:41.772 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-09-12T12:33:41.772 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-09-12T12:33:41.772 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-09-12T12:33:41.772 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-09-12T12:33:41.773 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-09-12T12:33:41.773 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-09-12T12:33:41.773 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-09-12T12:33:41.773 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-09-12T12:33:41.773 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-09-12T12:33:41.773 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-09-12T12:33:41.773 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-09-12T12:33:41.774 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-09-12T12:33:41.774 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-09-12T12:33:41.774 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-09-12T12:33:41.774 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-09-12T12:33:41.774 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-09-12T12:33:41.774 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-09-12T12:33:41.774 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-09-12T12:33:41.775 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T12:33:41.775 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-09-12T12:33:41.775 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-09-12T12:33:41.775 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-09-12T12:33:41.775 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-09-12T12:33:41.775 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-09-12T12:33:41.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.776 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-09-12T12:33:41.776 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-09-12T12:33:41.776 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-09-12T12:33:41.776 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-09-12T12:33:41.777 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-09-12T12:33:41.777 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 20108, 2024-09-12T12:33:41.777 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 9, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-09-12T12:33:41.778 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 806907199, 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 809766623, 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 809758168, 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.779 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 20108 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 772, 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 5053, 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 4281, 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 106, 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 144, 2024-09-12T12:33:41.780 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 38, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 5280, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 5280, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 56488, 2024-09-12T12:33:41.781 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215328 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-09-12T12:33:41.782 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-09-12T12:33:41.783 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.783 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.783 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.783 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-09-12T12:33:41.783 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.783 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.784 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-09-12T12:33:41.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.784 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.784 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.785 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-09-12T12:33:41.786 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.787 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.787 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.787 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-09-12T12:33:41.787 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.787 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.787 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-09-12T12:33:41.788 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.790 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.791 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.792 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-09-12T12:33:41.793 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.795 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-09-12T12:33:41.796 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.798 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.799 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-09-12T12:33:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-09-12T12:33:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.800 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.801 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 0, 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 0, 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-09-12T12:33:41.802 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 2493, 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:41.803 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:41.804 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:41.805 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:41.806 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:41.807 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 259567, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 1083, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:41.808 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8032, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 177, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:41.809 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 5921464, 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 54341, 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.810 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 563, 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 26, 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 4, 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 563, 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-09-12T12:33:41.811 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 563, 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1.169976386, 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.002078110 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 563, 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 564 2024-09-12T12:33:41.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 563, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 563, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 316, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-09-12T12:33:41.813 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 244, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-09-12T12:33:41.814 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 2, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-09-12T12:33:41.815 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-09-12T12:33:41.816 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-09-12T12:33:41.817 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 120, 2024-09-12T12:33:41.818 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 4, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 9, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 9, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-09-12T12:33:41.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.820 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-09-12T12:33:41.820 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-09-12T12:33:41.820 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-09-12T12:33:41.820 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-09-12T12:33:41.820 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-09-12T12:33:41.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.820 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-09-12T12:33:41.821 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 614, 2024-09-12T12:33:41.822 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 88057201, 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 614, 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 614, 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 88057201, 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:41.823 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:41.824 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 563, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 87783861, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 563, 2024-09-12T12:33:41.825 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 87783861, 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-09-12T12:33:41.826 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 563, 2024-09-12T12:33:41.827 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 563, 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 563, 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 563, 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:41.829 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.830 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x557330cee640": { 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:41.831 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:41.832 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:41.833 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:41.833 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:41.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:41.833 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-12T12:33:43.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:42 smithi189 ceph-mon[35475]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.6 MiB/s wr, 153 op/s 2024-09-12T12:33:43.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:42 smithi137 ceph-mon[30950]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.6 MiB/s wr, 153 op/s 2024-09-12T12:33:43.076 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T12:33:43.076 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T12:33:43.076 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-12T12:33:43.077 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-12T12:33:43.077 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-12T12:33:43.077 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-12T12:33:43.077 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T12:33:43.077 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T12:33:43.077 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T12:33:43.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:42 smithi171 ceph-mon[35098]: pgmap v652: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 3.6 MiB/s wr, 153 op/s 2024-09-12T12:33:43.380 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:33:44.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:43 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:33:44.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:43 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:33:44.145 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-12T12:33:44.145 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:44.146 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 393119, 2024-09-12T12:33:44.146 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 344931, 2024-09-12T12:33:44.146 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 111327594, 2024-09-12T12:33:44.146 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 258769342, 2024-09-12T12:33:44.146 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 10, 2024-09-12T12:33:44.147 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-09-12T12:33:44.147 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 20.075549719, 2024-09-12T12:33:44.147 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 8.597947976, 2024-09-12T12:33:44.147 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 13.755825272, 2024-09-12T12:33:44.147 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.816351638, 2024-09-12T12:33:44.147 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:44.148 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 344931, 2024-09-12T12:33:44.148 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 23.049534747, 2024-09-12T12:33:44.148 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000066823 2024-09-12T12:33:44.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.148 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:44.149 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 686340, 2024-09-12T12:33:44.149 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.105381085, 2024-09-12T12:33:44.149 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000153 2024-09-12T12:33:44.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.149 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:44.150 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:44.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.150 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:44.150 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 22189, 2024-09-12T12:33:44.150 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 22463, 2024-09-12T12:33:44.151 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 5622016, 2024-09-12T12:33:44.151 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 250962912, 2024-09-12T12:33:44.151 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 15, 2024-09-12T12:33:44.151 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 7, 2024-09-12T12:33:44.151 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 2.622055018, 2024-09-12T12:33:44.152 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 1.246374545, 2024-09-12T12:33:44.152 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.405552864, 2024-09-12T12:33:44.152 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.843278978, 2024-09-12T12:33:44.152 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:44.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 22461, 2024-09-12T12:33:44.153 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 2.354022585, 2024-09-12T12:33:44.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000104804 2024-09-12T12:33:44.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.153 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:44.153 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 12, 2024-09-12T12:33:44.153 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000001335, 2024-09-12T12:33:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000111 2024-09-12T12:33:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 4272, 2024-09-12T12:33:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 657616 2024-09-12T12:33:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 331546, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 439241, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 108021911, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 550874168, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 10, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 7, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 18.545325441, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 8.030236564, 2024-09-12T12:33:44.155 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 13.491019220, 2024-09-12T12:33:44.156 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.695580167, 2024-09-12T12:33:44.156 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:44.156 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 439238, 2024-09-12T12:33:44.156 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 35.572700969, 2024-09-12T12:33:44.156 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000080987 2024-09-12T12:33:44.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.157 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:44.157 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 677489, 2024-09-12T12:33:44.157 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.124396799, 2024-09-12T12:33:44.157 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000183 2024-09-12T12:33:44.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.158 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1680464, 2024-09-12T12:33:44.158 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 249488 2024-09-12T12:33:44.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.158 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-12T12:33:44.158 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-09-12T12:33:44.159 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-12T12:33:44.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.159 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-09-12T12:33:44.159 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:44.159 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:44.160 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 52983, 2024-09-12T12:33:44.160 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 80.121652312, 2024-09-12T12:33:44.160 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001512214 2024-09-12T12:33:44.160 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.161 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-09-12T12:33:44.161 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:44.161 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:44.161 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 34180, 2024-09-12T12:33:44.161 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 7.650486278, 2024-09-12T12:33:44.162 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000223829 2024-09-12T12:33:44.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.162 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-09-12T12:33:44.162 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 219940, 2024-09-12T12:33:44.163 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 188448, 2024-09-12T12:33:44.163 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-09-12T12:33:44.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 188448, 2024-09-12T12:33:44.163 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1250.986899930, 2024-09-12T12:33:44.163 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.006638366 2024-09-12T12:33:44.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.164 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-09-12T12:33:44.164 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 31489, 2024-09-12T12:33:44.164 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 11, 2024-09-12T12:33:44.165 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-09-12T12:33:44.165 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 155, 2024-09-12T12:33:44.165 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 15, 2024-09-12T12:33:44.165 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 9, 2024-09-12T12:33:44.165 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 3894, 2024-09-12T12:33:44.166 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 1437, 2024-09-12T12:33:44.166 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 2430, 2024-09-12T12:33:44.166 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 27, 2024-09-12T12:33:44.166 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 1729, 2024-09-12T12:33:44.167 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 755, 2024-09-12T12:33:44.167 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 288, 2024-09-12T12:33:44.167 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 288, 2024-09-12T12:33:44.168 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 15, 2024-09-12T12:33:44.168 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 324701, 2024-09-12T12:33:44.168 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 173539, 2024-09-12T12:33:44.168 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 31474, 2024-09-12T12:33:44.168 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 276, 2024-09-12T12:33:44.169 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 5, 2024-09-12T12:33:44.172 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-09-12T12:33:44.172 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 2296, 2024-09-12T12:33:44.172 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 30245, 2024-09-12T12:33:44.173 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-09-12T12:33:44.173 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 10, 2024-09-12T12:33:44.173 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 562, 2024-09-12T12:33:44.173 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 11, 2024-09-12T12:33:44.173 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 1418, 2024-09-12T12:33:44.173 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 703, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 211987629, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 6, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T12:33:44.174 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 19, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 27895, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 103851, 2024-09-12T12:33:44.175 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T12:33:44.176 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T12:33:44.176 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T12:33:44.176 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 78655, 2024-09-12T12:33:44.176 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 20575, 2024-09-12T12:33:44.176 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 10103, 2024-09-12T12:33:44.176 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 148516 2024-09-12T12:33:44.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.177 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-09-12T12:33:44.177 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 306, 2024-09-12T12:33:44.177 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 1348, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 754, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 1355, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 382, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 16, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 21406, 2024-09-12T12:33:44.178 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 21390, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 18, 2024-09-12T12:33:44.179 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 2211, 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 23, 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 24, 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-09-12T12:33:44.180 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 140029, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 55728, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 49161, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 90868, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 6567, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 194, 2024-09-12T12:33:44.181 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 66, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 58, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 136, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 8, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 8, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 69941494, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 570924282, 2024-09-12T12:33:44.182 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 30208, 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 253.201438569, 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.008381933 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.183 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 2274, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 24236, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 21962, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 248, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 932, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 684, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 3894, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 48060, 2024-09-12T12:33:44.184 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 44166, 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 288, 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 25653, 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 25365, 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 91044, 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 223520 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-09-12T12:33:44.185 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 297537, 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 24, 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 219940, 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 3133, 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 8853, 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 21373, 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 391.676855973, 2024-09-12T12:33:44.186 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.018325778 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3365, 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 40.179990245, 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.011940561 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-09-12T12:33:44.187 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.188 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-09-12T12:33:44.189 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 47848, 2024-09-12T12:33:44.189 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 202.204588152, 2024-09-12T12:33:44.189 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.004225977 2024-09-12T12:33:44.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.189 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-09-12T12:33:44.189 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.189 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000162440, 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000081220 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-09-12T12:33:44.190 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.192 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-09-12T12:33:44.192 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.193 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.193 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.193 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-09-12T12:33:44.193 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.193 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.193 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-09-12T12:33:44.194 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.195 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.195 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.195 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-09-12T12:33:44.195 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.195 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.195 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-09-12T12:33:44.196 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 89947, 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 321.257051829, 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003571626 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3860, 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 32.602590172, 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.008446266 2024-09-12T12:33:44.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.198 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.199 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-09-12T12:33:44.200 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.200 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.200 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.200 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-09-12T12:33:44.200 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.200 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.201 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.201 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-09-12T12:33:44.201 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 99, 2024-09-12T12:33:44.201 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 95.741299601, 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.967083834 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.202 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 548, 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1.702947647, 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003107568 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.203 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.204 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-09-12T12:33:44.204 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.204 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.204 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.204 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 21406, 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 165.621413871, 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.007737149 2024-09-12T12:33:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-09-12T12:33:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-09-12T12:33:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 3, 2024-09-12T12:33:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 1, 2024-09-12T12:33:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 2, 2024-09-12T12:33:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-09-12T12:33:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 27610, 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 13805, 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 1198, 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 240, 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 240, 2024-09-12T12:33:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-12T12:33:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 343494, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 1487, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 264, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 3, 2024-09-12T12:33:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8032, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 177, 2024-09-12T12:33:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 8973989, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 84295, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-12T12:33:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-12T12:33:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-09-12T12:33:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-09-12T12:33:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-09-12T12:33:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 54639, 2024-09-12T12:33:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 572886641, 2024-09-12T12:33:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 28, 2024-09-12T12:33:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 54639, 2024-09-12T12:33:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-09-12T12:33:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 54639, 2024-09-12T12:33:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 404.073823869, 2024-09-12T12:33:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.007395337 2024-09-12T12:33:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-09-12T12:33:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-09-12T12:33:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 54639, 2024-09-12T12:33:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 56013 2024-09-12T12:33:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 54639, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 730, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 53909, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 15, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 21, 2024-09-12T12:33:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 719, 2024-09-12T12:33:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 30927, 2024-09-12T12:33:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 412, 2024-09-12T12:33:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-09-12T12:33:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-09-12T12:33:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-09-12T12:33:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 15, 2024-09-12T12:33:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 21600, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 7, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 42, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-09-12T12:33:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-09-12T12:33:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-09-12T12:33:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 120, 2024-09-12T12:33:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 38, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 12, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 12, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 1790, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 22, 2024-09-12T12:33:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 443 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 2248 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 10, 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 10, 2024-09-12T12:33:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 21390, 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 21390, 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 746850, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 157727099, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 746850, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 746850, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 157727099, 2024-09-12T12:33:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-09-12T12:33:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 54639, 2024-09-12T12:33:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 575051631, 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 54639, 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 575051631, 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 54639, 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 54639, 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 54639, 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 54639, 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 21390, 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2165166, 2024-09-12T12:33:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 21390, 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 719, 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2165166, 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x558c74c4e960": { 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 14950, 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 140029, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 566729978, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 140029, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 30208, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 566715028, 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:44.239 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-12T12:33:44.239 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:43 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:33:44.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:44 smithi189 ceph-mon[35475]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-09-12T12:33:44.927 INFO:tasks.check_counter:Daemon mds.i mds.exported=10 2024-09-12T12:33:44.928 INFO:tasks.check_counter:Daemon mds.i mds.imported=11 2024-09-12T12:33:44.928 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=306 2024-09-12T12:33:44.928 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1348 2024-09-12T12:33:44.928 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-12T12:33:44.928 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-12T12:33:44.928 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T12:33:44.929 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T12:33:44.929 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T12:33:45.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:44 smithi137 ceph-mon[30950]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-09-12T12:33:45.177 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T12:33:45.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:44 smithi171 ceph-mon[35098]: pgmap v653: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-09-12T12:33:45.731 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-12T12:33:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T12:33:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1327, 2024-09-12T12:33:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1329, 2024-09-12T12:33:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 226839155, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 422050, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 8, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.240265531, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.029057388, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.168449397, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.032399072, 2024-09-12T12:33:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1327, 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.047561361, 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000035841 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1750, 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2030, 2024-09-12T12:33:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 256353445, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 1213909, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 9, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 7, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.276608990, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.051651470, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.171054837, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.040647180, 2024-09-12T12:33:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2028, 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.083186506, 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000041018 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 659312 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 2675, 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2631, 2024-09-12T12:33:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 146443825, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 941740, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.336140979, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.080331487, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.194932097, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.062285074, 2024-09-12T12:33:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2630, 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.106870140, 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000040635 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1669008, 2024-09-12T12:33:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 236416 2024-09-12T12:33:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-09-12T12:33:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-09-12T12:33:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 8619, 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.226619214, 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000026292 2024-09-12T12:33:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-09-12T12:33:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-09-12T12:33:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-09-12T12:33:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-09-12T12:33:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-09-12T12:33:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-09-12T12:33:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-09-12T12:33:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 33249, 2024-09-12T12:33:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 9956, 2024-09-12T12:33:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 23293, 2024-09-12T12:33:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 19026, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 7, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-09-12T12:33:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-09-12T12:33:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-09-12T12:33:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-09-12T12:33:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-09-12T12:33:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-09-12T12:33:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-09-12T12:33:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-09-12T12:33:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 0, 2024-09-12T12:33:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 0, 2024-09-12T12:33:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 0, 2024-09-12T12:33:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T12:33:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-09-12T12:33:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T12:33:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-09-12T12:33:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-09-12T12:33:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T12:33:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-09-12T12:33:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-09-12T12:33:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-09-12T12:33:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-09-12T12:33:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-09-12T12:33:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-09-12T12:33:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-09-12T12:33:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-09-12T12:33:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-09-12T12:33:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_path": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_quiesce_inode": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-09-12T12:33:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-09-12T12:33:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-09-12T12:33:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-09-12T12:33:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-09-12T12:33:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-09-12T12:33:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-09-12T12:33:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 163987, 2024-09-12T12:33:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-09-12T12:33:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-09-12T12:33:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-09-12T12:33:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-09-12T12:33:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-09-12T12:33:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 134, 2024-09-12T12:33:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 15, 2024-09-12T12:33:45.757 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-09-12T12:33:45.757 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-09-12T12:33:45.757 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 136110416, 2024-09-12T12:33:45.757 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 630630623, 2024-09-12T12:33:45.757 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 630623752, 2024-09-12T12:33:45.757 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-09-12T12:33:45.758 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.758 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.758 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.758 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 163987 2024-09-12T12:33:45.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.758 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 1656, 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 33171, 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 31515, 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 84, 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 184, 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 100, 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 33249, 2024-09-12T12:33:45.759 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 33483, 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 234, 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 86136, 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215328 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.760 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-09-12T12:33:45.761 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-09-12T12:33:45.761 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-09-12T12:33:45.761 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-09-12T12:33:45.761 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-09-12T12:33:45.761 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-09-12T12:33:45.761 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-09-12T12:33:45.761 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.763 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-09-12T12:33:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-09-12T12:33:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-09-12T12:33:45.765 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.767 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-09-12T12:33:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-09-12T12:33:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-09-12T12:33:45.769 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-09-12T12:33:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.770 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.771 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.772 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.773 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.774 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.775 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-09-12T12:33:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-09-12T12:33:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.776 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.777 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.778 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-09-12T12:33:45.779 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.780 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.780 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.780 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-09-12T12:33:45.780 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-09-12T12:33:45.780 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-09-12T12:33:45.780 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 1, 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-09-12T12:33:45.781 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 3, 2024-09-12T12:33:45.782 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 3, 2024-09-12T12:33:45.782 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-09-12T12:33:45.782 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-09-12T12:33:45.782 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-09-12T12:33:45.782 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 5044, 2024-09-12T12:33:45.782 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-09-12T12:33:45.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-09-12T12:33:45.783 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T12:33:45.784 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T12:33:45.785 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T12:33:45.786 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-09-12T12:33:45.786 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-09-12T12:33:45.786 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-09-12T12:33:45.786 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T12:33:45.786 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T12:33:45.786 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-09-12T12:33:45.786 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 311013, 2024-09-12T12:33:45.787 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 1975, 2024-09-12T12:33:45.788 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-09-12T12:33:45.788 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-09-12T12:33:45.788 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-09-12T12:33:45.788 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-09-12T12:33:45.788 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-09-12T12:33:45.788 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-09-12T12:33:45.788 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8032, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 177, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-09-12T12:33:45.789 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 27726507, 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 184790, 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-09-12T12:33:45.790 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 5289, 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 21, 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 3, 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 5289, 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 5289, 2024-09-12T12:33:45.791 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 7.291851633, 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.001378682 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 5289, 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 5290 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.792 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 5289, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 5289, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 3026, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 2261, 2024-09-12T12:33:45.793 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-09-12T12:33:45.794 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 1, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-09-12T12:33:45.795 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T12:33:45.796 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-09-12T12:33:45.796 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-09-12T12:33:45.796 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-09-12T12:33:45.796 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-09-12T12:33:45.796 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-09-12T12:33:45.796 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-09-12T12:33:45.796 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-09-12T12:33:45.797 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 120, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 38, 2024-09-12T12:33:45.798 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 11, 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 11, 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.799 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-09-12T12:33:45.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.801 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-09-12T12:33:45.801 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-09-12T12:33:45.801 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T12:33:45.801 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-09-12T12:33:45.801 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-09-12T12:33:45.801 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-09-12T12:33:45.801 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 5748, 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 629092915, 2024-09-12T12:33:45.802 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 5748, 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 5748, 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 629092915, 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.803 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.804 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.804 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-09-12T12:33:45.804 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:45.804 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-09-12T12:33:45.804 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:45.805 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:45.805 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:45.805 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:45.805 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:45.805 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 5289, 2024-09-12T12:33:45.805 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 626505785, 2024-09-12T12:33:45.806 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 5289, 2024-09-12T12:33:45.806 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 626505785, 2024-09-12T12:33:45.806 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:45.806 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.806 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:45.807 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:45.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:45.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:45.808 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:45.808 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 5289, 2024-09-12T12:33:45.808 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 5289, 2024-09-12T12:33:45.808 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 5289, 2024-09-12T12:33:45.808 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 5289, 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-09-12T12:33:45.809 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:45.810 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:45.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:45.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:45.810 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:45.812 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:45.812 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:45.812 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:45.812 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:45.812 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-09-12T12:33:45.813 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-09-12T12:33:45.813 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:45.813 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.813 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.813 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.813 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-12T12:33:45.814 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x563ac9a24b40": { 2024-09-12T12:33:45.814 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-09-12T12:33:45.814 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-09-12T12:33:45.814 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-09-12T12:33:45.814 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-09-12T12:33:45.815 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-09-12T12:33:45.815 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-09-12T12:33:45.815 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-09-12T12:33:45.815 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-09-12T12:33:45.815 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-09-12T12:33:45.815 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-09-12T12:33:45.815 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-09-12T12:33:45.816 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-09-12T12:33:45.816 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-09-12T12:33:45.816 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-09-12T12:33:45.816 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-09-12T12:33:45.816 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.816 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-12T12:33:45.816 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-12T12:33:46.529 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T12:33:46.530 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T12:33:46.530 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-12T12:33:46.530 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-12T12:33:46.530 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-12T12:33:46.530 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-12T12:33:46.531 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-12T12:33:46.531 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T12:33:46.544 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T12:33:46.545 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:33:46.545 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:33:46.566 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-09-12T12:33:46.567 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T12:33:46.626 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T12:33:46.967 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:47.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:46 smithi137 ceph-mon[30950]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.5 MiB/s wr, 110 op/s 2024-09-12T12:33:47.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:46 smithi189 ceph-mon[35475]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.5 MiB/s wr, 110 op/s 2024-09-12T12:33:47.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:46 smithi171 ceph-mon[35098]: pgmap v654: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.5 MiB/s wr, 110 op/s 2024-09-12T12:33:47.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 -- 172.21.15.137:0/406718722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 msgr2=0x7f623010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:47.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 --2- 172.21.15.137:0/406718722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f623010ea90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6228009ca0 tx=0x7f622802a8c0 comp rx=0 tx=0).stop 2024-09-12T12:33:47.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 -- 172.21.15.137:0/406718722 shutdown_connections 2024-09-12T12:33:47.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 --2- 172.21.15.137:0/406718722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f623010ea90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:47.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 --2- 172.21.15.137:0/406718722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6230102790 0x7f62301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:47.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 --2- 172.21.15.137:0/406718722 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6230101de0 0x7f62301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:47.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 -- 172.21.15.137:0/406718722 >> 172.21.15.137:0/406718722 conn(0x7f62300fd880 msgr2=0x7f62300ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:47.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 -- 172.21.15.137:0/406718722 shutdown_connections 2024-09-12T12:33:47.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.846+0000 7f6235c5c640 1 -- 172.21.15.137:0/406718722 wait complete. 2024-09-12T12:33:47.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.847+0000 7f6235c5c640 1 Processor -- start 2024-09-12T12:33:47.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.847+0000 7f6235c5c640 1 -- start start 2024-09-12T12:33:47.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f6235c5c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6230101de0 0x7f62300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:47.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f6235c5c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6230102790 0x7f6230074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:47.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f6235c5c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f6230074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:47.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f6235c5c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6230111f70 con 0x7f6230106f10 2024-09-12T12:33:47.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f6235c5c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6230111df0 con 0x7f6230102790 2024-09-12T12:33:47.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f6235c5c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62301120f0 con 0x7f6230101de0 2024-09-12T12:33:47.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f6230074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:47.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f6230074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51736/0 (socket says 172.21.15.137:51736) 2024-09-12T12:33:47.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 -- 172.21.15.137:0/336495572 learned_addr learned my addr 172.21.15.137:0/336495572 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:47.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622effd640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6230102790 0x7f6230074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:47.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622f7fe640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6230101de0 0x7f62300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:47.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 -- 172.21.15.137:0/336495572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6230101de0 msgr2=0x7f62300744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:47.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6230101de0 0x7f62300744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 -- 172.21.15.137:0/336495572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6230102790 msgr2=0x7f6230074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6230102790 0x7f6230074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622ffff640 1 -- 172.21.15.137:0/336495572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62301a6850 con 0x7f6230106f10 2024-09-12T12:33:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622effd640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6230102790 0x7f6230074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:33:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.848+0000 7f622f7fe640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6230101de0 0x7f62300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:33:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.849+0000 7f622ffff640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f6230074f70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6228031a20 tx=0x7f6228031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:47.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.849+0000 7f622cff9640 1 -- 172.21.15.137:0/336495572 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228003c30 con 0x7f6230106f10 2024-09-12T12:33:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.849+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62301a6ae0 con 0x7f6230106f10 2024-09-12T12:33:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.849+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62301a7070 con 0x7f6230106f10 2024-09-12T12:33:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.849+0000 7f622cff9640 1 -- 172.21.15.137:0/336495572 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6228003dd0 con 0x7f6230106f10 2024-09-12T12:33:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.849+0000 7f622cff9640 1 -- 172.21.15.137:0/336495572 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622802fba0 con 0x7f6230106f10 2024-09-12T12:33:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.850+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62301021c0 con 0x7f6230106f10 2024-09-12T12:33:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.852+0000 7f622cff9640 1 -- 172.21.15.137:0/336495572 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f62280046c0 con 0x7f6230106f10 2024-09-12T12:33:47.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.853+0000 7f622cff9640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62180a2df0 0x7f62180a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:47.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.853+0000 7f622f7fe640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62180a2df0 0x7f62180a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:47.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.853+0000 7f622cff9640 1 -- 172.21.15.137:0/336495572 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8484+0+0 (secure 0 0 0) 0x7f62280f0750 con 0x7f6230106f10 2024-09-12T12:33:47.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.853+0000 7f622f7fe640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62180a2df0 0x7f62180a52b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6220005d00 tx=0x7f6220009550 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:47.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:47.856+0000 7f622cff9640 1 -- 172.21.15.137:0/336495572 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62280bd4d0 con 0x7f6230106f10 2024-09-12T12:33:48.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.039+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f623002d7c0 con 0x7f6230106f10 2024-09-12T12:33:48.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.040+0000 7f622cff9640 1 -- 172.21.15.137:0/336495572 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v120) ==== 81+0+958 (secure 0 0 0) 0x7f62280bcdd0 con 0x7f6230106f10 2024-09-12T12:33:48.042 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/4099842792 2024-09-13T12:13:13.350302+0000 2024-09-12T12:33:48.042 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/123979974 2024-09-13T12:13:13.350302+0000 2024-09-12T12:33:48.042 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2550482799 2024-09-13T12:13:13.350302+0000 2024-09-12T12:33:48.042 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1305616495 2024-09-13T12:03:52.204874+0000 2024-09-12T12:33:48.043 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2039555152 2024-09-13T12:13:13.350302+0000 2024-09-12T12:33:48.043 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2994676901 2024-09-13T12:03:52.204874+0000 2024-09-12T12:33:48.043 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1104802518 2024-09-13T12:03:52.204874+0000 2024-09-12T12:33:48.043 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/612748114 2024-09-13T12:04:15.038962+0000 2024-09-12T12:33:48.044 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2160224100 2024-09-13T12:04:15.038962+0000 2024-09-12T12:33:48.044 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1596672143 2024-09-13T12:04:15.038962+0000 2024-09-12T12:33:48.044 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1789953576 2024-09-13T12:13:13.350302+0000 2024-09-12T12:33:48.045 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1104802518 2024-09-13T12:03:52.204874+0000 2024-09-12T12:33:48.045 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/936885966 2024-09-13T12:03:52.204874+0000 2024-09-12T12:33:48.045 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1596672143 2024-09-13T12:04:15.038962+0000 2024-09-12T12:33:48.045 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/228183714 2024-09-13T12:04:15.038962+0000 2024-09-12T12:33:48.046 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1789953576 2024-09-13T12:13:13.350302+0000 2024-09-12T12:33:48.047 INFO:teuthology.orchestra.run.smithi137.stderr:listed 16 entries 2024-09-12T12:33:48.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62180a2df0 msgr2=0x7f62180a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:48.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62180a2df0 0x7f62180a52b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6220005d00 tx=0x7f6220009550 comp rx=0 tx=0).stop 2024-09-12T12:33:48.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 msgr2=0x7f6230074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:48.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f6230074f70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f6228031a20 tx=0x7f6228031a50 comp rx=0 tx=0).stop 2024-09-12T12:33:48.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 shutdown_connections 2024-09-12T12:33:48.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f62180a2df0 0x7f62180a52b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:48.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6230106f10 0x7f6230074f70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:48.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6230102790 0x7f6230074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:48.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 --2- 172.21.15.137:0/336495572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6230101de0 0x7f62300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:48.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 >> 172.21.15.137:0/336495572 conn(0x7f62300fd880 msgr2=0x7f62300feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:48.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 shutdown_connections 2024-09-12T12:33:48.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:48.042+0000 7f6235c5c640 1 -- 172.21.15.137:0/336495572 wait complete. 2024-09-12T12:33:48.810 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T12:33:48.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:33:48.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T12:33:49.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:33:49.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:33:49.001 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T12:33:49.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:48 smithi137 ceph-mon[30950]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-12T12:33:49.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:48 smithi137 ceph-mon[30950]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:33:49.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:48 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/336495572' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T12:33:49.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:33:49.062 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-09-12T12:33:49.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:48 smithi189 ceph-mon[35475]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-12T12:33:49.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:48 smithi189 ceph-mon[35475]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:33:49.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:48 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/336495572' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T12:33:49.109 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T12:33:49.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:33:49.110 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T12:33:49.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T12:33:49.177 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link delete ceph-brx 2024-09-12T12:33:49.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:48 smithi171 ceph-mon[35098]: pgmap v655: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-12T12:33:49.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:48 smithi171 ceph-mon[35098]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T12:33:49.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:48 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/336495572' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T12:33:49.279 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T12:33:49.290 INFO:tasks.cephadm:Teardown begin 2024-09-12T12:33:49.290 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T12:33:49.291 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:33:49.608 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:50.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 -- 172.21.15.137:0/2390104214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 msgr2=0x7fb738101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:50.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 --2- 172.21.15.137:0/2390104214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb738101200 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb728009900 tx=0x7fb72802a480 comp rx=0 tx=0).stop 2024-09-12T12:33:50.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 -- 172.21.15.137:0/2390104214 shutdown_connections 2024-09-12T12:33:50.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 --2- 172.21.15.137:0/2390104214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb738102ef0 0x7fb73810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 --2- 172.21.15.137:0/2390104214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb738102570 0x7fb7381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 --2- 172.21.15.137:0/2390104214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb738101200 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 -- 172.21.15.137:0/2390104214 >> 172.21.15.137:0/2390104214 conn(0x7fb7380fc7d0 msgr2=0x7fb7380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:50.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 -- 172.21.15.137:0/2390104214 shutdown_connections 2024-09-12T12:33:50.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 -- 172.21.15.137:0/2390104214 wait complete. 2024-09-12T12:33:50.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.135+0000 7fb73f156640 1 Processor -- start 2024-09-12T12:33:50.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73f156640 1 -- start start 2024-09-12T12:33:50.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73f156640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb7381984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73f156640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb738102570 0x7fb738198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73f156640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb738102ef0 0x7fb73819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73f156640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb738110fc0 con 0x7fb738100e20 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73f156640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb738110e40 con 0x7fb738102570 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73f156640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb738111140 con 0x7fb738102ef0 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb7381984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb7381984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60032/0 (socket says 172.21.15.137:60032) 2024-09-12T12:33:50.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 -- 172.21.15.137:0/928487491 learned_addr learned my addr 172.21.15.137:0/928487491 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73d6f3640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb738102ef0 0x7fb73819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb737fff640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb738102570 0x7fb738198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 -- 172.21.15.137:0/928487491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb738102ef0 msgr2=0x7fb73819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb738102ef0 0x7fb73819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 -- 172.21.15.137:0/928487491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb738102570 msgr2=0x7fb738198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb738102570 0x7fb738198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 -- 172.21.15.137:0/928487491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73819d4a0 con 0x7fb738100e20 2024-09-12T12:33:50.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb737fff640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb738102570 0x7fb738198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:50.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.136+0000 7fb73cef2640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb7381984f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb728006010 tx=0x7fb728003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:50.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.137+0000 7fb735ffb640 1 -- 172.21.15.137:0/928487491 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72803f070 con 0x7fb738100e20 2024-09-12T12:33:50.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.137+0000 7fb735ffb640 1 -- 172.21.15.137:0/928487491 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb728031c30 con 0x7fb738100e20 2024-09-12T12:33:50.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.137+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73819d730 con 0x7fb738100e20 2024-09-12T12:33:50.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.137+0000 7fb735ffb640 1 -- 172.21.15.137:0/928487491 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728035030 con 0x7fb738100e20 2024-09-12T12:33:50.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.137+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7381a5070 con 0x7fb738100e20 2024-09-12T12:33:50.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.138+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb738101200 con 0x7fb738100e20 2024-09-12T12:33:50.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.139+0000 7fb735ffb640 1 -- 172.21.15.137:0/928487491 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb728042070 con 0x7fb738100e20 2024-09-12T12:33:50.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.140+0000 7fb735ffb640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb7200a2cf0 0x7fb7200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:50.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.140+0000 7fb737fff640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb7200a2cf0 0x7fb7200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:50.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.140+0000 7fb735ffb640 1 -- 172.21.15.137:0/928487491 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8484+0+0 (secure 0 0 0) 0x7fb7280efca0 con 0x7fb738100e20 2024-09-12T12:33:50.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.140+0000 7fb737fff640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb7200a2cf0 0x7fb7200a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7380ff060 tx=0x7fb72c009570 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:50.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.145+0000 7fb735ffb640 1 -- 172.21.15.137:0/928487491 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb728031de0 con 0x7fb738100e20 2024-09-12T12:33:50.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.349+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb73810a0a0 con 0x7fb738100e20 2024-09-12T12:33:50.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.350+0000 7fb735ffb640 1 -- 172.21.15.137:0/928487491 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11940 (secure 0 0 0) 0x7fb7280dae20 con 0x7fb738100e20 2024-09-12T12:33:50.351 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:33:50.351 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":57,"btime":"2024-09-12T12:33:47:759390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:33:47.020947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26861":{"gid":26861,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/4049352466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4049352466},{"type":"v1","addr":"172.21.15.189:6835","nonce":4049352466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:33:50.351 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 57 2024-09-12T12:33:50.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb7200a2cf0 msgr2=0x7fb7200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:50.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb7200a2cf0 0x7fb7200a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb7380ff060 tx=0x7fb72c009570 comp rx=0 tx=0).stop 2024-09-12T12:33:50.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 msgr2=0x7fb7381984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:50.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb7381984f0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb728006010 tx=0x7fb728003c10 comp rx=0 tx=0).stop 2024-09-12T12:33:50.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 shutdown_connections 2024-09-12T12:33:50.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fb7200a2cf0 0x7fb7200a51b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb738102ef0 0x7fb73819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb738102570 0x7fb738198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 --2- 172.21.15.137:0/928487491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb738100e20 0x7fb7381984f0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:50.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 >> 172.21.15.137:0/928487491 conn(0x7fb7380fc7d0 msgr2=0x7fb73806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:50.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 shutdown_connections 2024-09-12T12:33:50.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:50.352+0000 7fb73f156640 1 -- 172.21.15.137:0/928487491 wait complete. 2024-09-12T12:33:51.018 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:50 smithi137 ceph-mon[30950]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-12T12:33:51.019 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:50 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/928487491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:51.047 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T12:33:51.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:50 smithi189 ceph-mon[35475]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-12T12:33:51.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:50 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/928487491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:51.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:50 smithi171 ceph-mon[35098]: pgmap v656: 97 pgs: 97 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-12T12:33:51.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:50 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/928487491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:51.293 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:51.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 -- 172.21.15.137:0/3814930581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c041082a0 msgr2=0x7f6c0410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:51.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3814930581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c041082a0 0x7f6c0410ec40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc009cd0 tx=0x7f6bfc02a8e0 comp rx=0 tx=0).stop 2024-09-12T12:33:51.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 -- 172.21.15.137:0/3814930581 shutdown_connections 2024-09-12T12:33:51.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3814930581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c041082a0 0x7f6c0410ec40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3814930581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c041036e0 0x7f6c04107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3814930581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c04102d30 0x7f6c04103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 -- 172.21.15.137:0/3814930581 >> 172.21.15.137:0/3814930581 conn(0x7f6c040fc7d0 msgr2=0x7f6c040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 -- 172.21.15.137:0/3814930581 shutdown_connections 2024-09-12T12:33:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.811+0000 7f6c0933c640 1 -- 172.21.15.137:0/3814930581 wait complete. 2024-09-12T12:33:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.812+0000 7f6c0933c640 1 Processor -- start 2024-09-12T12:33:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.812+0000 7f6c0933c640 1 -- start start 2024-09-12T12:33:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c04102d30 0x7f6c041985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c041036e0 0x7f6c04198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c041082a0 0x7f6c0419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04112100 con 0x7f6c04102d30 2024-09-12T12:33:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04111f80 con 0x7f6c041082a0 2024-09-12T12:33:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04112280 con 0x7f6c041036e0 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c04102d30 0x7f6c041985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c04102d30 0x7f6c041985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60052/0 (socket says 172.21.15.137:60052) 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 -- 172.21.15.137:0/3690461712 learned_addr learned my addr 172.21.15.137:0/3690461712 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 -- 172.21.15.137:0/3690461712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c041036e0 msgr2=0x7f6c04198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c027fc640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c041036e0 0x7f6c04198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c037fe640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c041082a0 0x7f6c0419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c041036e0 0x7f6c04198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 -- 172.21.15.137:0/3690461712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c041082a0 msgr2=0x7f6c0419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c041082a0 0x7f6c0419cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 -- 172.21.15.137:0/3690461712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0419d580 con 0x7f6c04102d30 2024-09-12T12:33:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c02ffd640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c04102d30 0x7f6c041985d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400b4a0 tx=0x7f6bf400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6bfbfff640 1 -- 172.21.15.137:0/3690461712 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4014070 con 0x7f6c04102d30 2024-09-12T12:33:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6bfbfff640 1 -- 172.21.15.137:0/3690461712 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bf4010af0 con 0x7f6c04102d30 2024-09-12T12:33:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0419d870 con 0x7f6c04102d30 2024-09-12T12:33:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6bfbfff640 1 -- 172.21.15.137:0/3690461712 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4017d10 con 0x7f6c04102d30 2024-09-12T12:33:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.813+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c041a5150 con 0x7f6c04102d30 2024-09-12T12:33:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.815+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c0406a850 con 0x7f6c04102d30 2024-09-12T12:33:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.816+0000 7f6bfbfff640 1 -- 172.21.15.137:0/3690461712 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6bf4017070 con 0x7f6c04102d30 2024-09-12T12:33:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.816+0000 7f6bfbfff640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6bf00a2cf0 0x7f6bf00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.816+0000 7f6c027fc640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6bf00a2cf0 0x7f6bf00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.816+0000 7f6bfbfff640 1 -- 172.21.15.137:0/3690461712 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8484+0+0 (secure 0 0 0) 0x7f6bf40ce7a0 con 0x7f6c04102d30 2024-09-12T12:33:51.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.817+0000 7f6c027fc640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6bf00a2cf0 0x7f6bf00a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6bec005fd0 tx=0x7f6bec005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:51.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:51.820+0000 7f6bfbfff640 1 -- 172.21.15.137:0/3690461712 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bf409b4f0 con 0x7f6c04102d30 2024-09-12T12:33:52.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.018+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c04103110 con 0x7f6c04102d30 2024-09-12T12:33:52.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.019+0000 7f6bfbfff640 1 -- 172.21.15.137:0/3690461712 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11940 (secure 0 0 0) 0x7f6bf409adf0 con 0x7f6c04102d30 2024-09-12T12:33:52.019 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:33:52.019 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":57,"btime":"2024-09-12T12:33:47:759390+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/2258495281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":2258495281},{"type":"v1","addr":"172.21.15.137:6835","nonce":2258495281}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/1845807788","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":1845807788},{"type":"v1","addr":"172.21.15.137:6837","nonce":1845807788}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3700003405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3700003405},{"type":"v1","addr":"172.21.15.137:6839","nonce":3700003405}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2375268827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2375268827},{"type":"v1","addr":"172.21.15.137:6841","nonce":2375268827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1998961324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1998961324},{"type":"v1","addr":"172.21.15.171:6833","nonce":1998961324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/971785585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":971785585},{"type":"v1","addr":"172.21.15.171:6835","nonce":971785585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T12:11:13.196665+0000","modified":"2024-09-12T12:33:47.020947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24476,"mds_1":24475,"mds_2":24454},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/4107944214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":4107944214},{"type":"v1","addr":"172.21.15.189:6833","nonce":4107944214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/1673181827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1673181827},{"type":"v1","addr":"172.21.15.171:6837","nonce":1673181827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.189:6837/2107760146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2107760146},{"type":"v1","addr":"172.21.15.189:6837","nonce":2107760146}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24475":{"gid":24475,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.189:6839/4067701297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":4067701297},{"type":"v1","addr":"172.21.15.189:6839","nonce":4067701297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24476":{"gid":24476,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.171:6839/1705495422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":1705495422},{"type":"v1","addr":"172.21.15.171:6839","nonce":1705495422}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26861":{"gid":26861,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/4049352466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":4049352466},{"type":"v1","addr":"172.21.15.189:6835","nonce":4049352466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24476,"qdb_cluster":[24476,24475,24454]},"id":1}]} 2024-09-12T12:33:52.020 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 57 2024-09-12T12:33:52.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6bf00a2cf0 msgr2=0x7f6bf00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:52.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6bf00a2cf0 0x7f6bf00a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6bec005fd0 tx=0x7f6bec005850 comp rx=0 tx=0).stop 2024-09-12T12:33:52.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c04102d30 msgr2=0x7f6c041985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:52.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c04102d30 0x7f6c041985d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6bf400b4a0 tx=0x7f6bf400b960 comp rx=0 tx=0).stop 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 shutdown_connections 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f6bf00a2cf0 0x7f6bf00a51b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6c041082a0 0x7f6c0419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c041036e0 0x7f6c04198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 --2- 172.21.15.137:0/3690461712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c04102d30 0x7f6c041985d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.021+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 >> 172.21.15.137:0/3690461712 conn(0x7f6c040fc7d0 msgr2=0x7f6c04105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.022+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 shutdown_connections 2024-09-12T12:33:52.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:52.022+0000 7f6c0933c640 1 -- 172.21.15.137:0/3690461712 wait complete. 2024-09-12T12:33:52.722 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T12:33:52.970 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:53.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:52 smithi189 ceph-mon[35475]: pgmap v657: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.4 MiB/s wr, 82 op/s 2024-09-12T12:33:53.036 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:52 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/3690461712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:53.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:52 smithi137 ceph-mon[30950]: pgmap v657: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.4 MiB/s wr, 82 op/s 2024-09-12T12:33:53.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:52 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/3690461712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:53.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:52 smithi171 ceph-mon[35098]: pgmap v657: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 2.4 MiB/s wr, 82 op/s 2024-09-12T12:33:53.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:52 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/3690461712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T12:33:53.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 -- 172.21.15.137:0/2048275198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070101420 msgr2=0x7fd07010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:53.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 --2- 172.21.15.137:0/2048275198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070101420 0x7fd07010d950 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fd068009cd0 tx=0x7fd06802a8e0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 -- 172.21.15.137:0/2048275198 shutdown_connections 2024-09-12T12:33:53.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 --2- 172.21.15.137:0/2048275198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070101420 0x7fd07010d950 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 --2- 172.21.15.137:0/2048275198 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd070100a80 0x7fd070100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 --2- 172.21.15.137:0/2048275198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd070106a80 0x7fd070106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 -- 172.21.15.137:0/2048275198 >> 172.21.15.137:0/2048275198 conn(0x7fd0700fc7a0 msgr2=0x7fd0700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:53.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 -- 172.21.15.137:0/2048275198 shutdown_connections 2024-09-12T12:33:53.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.603+0000 7fd075962640 1 -- 172.21.15.137:0/2048275198 wait complete. 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.604+0000 7fd075962640 1 Processor -- start 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.604+0000 7fd075962640 1 -- start start 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070100a80 0x7fd070198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd070101420 0x7fd0701989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd070106a80 0x7fd07019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd070110e80 con 0x7fd070100a80 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd070110d00 con 0x7fd070101420 2024-09-12T12:33:53.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd070111000 con 0x7fd070106a80 2024-09-12T12:33:53.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070100a80 0x7fd070198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:53.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070100a80 0x7fd070198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60070/0 (socket says 172.21.15.137:60070) 2024-09-12T12:33:53.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 -- 172.21.15.137:0/1077436569 learned_addr learned my addr 172.21.15.137:0/1077436569 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:53.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06f7fe640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd070106a80 0x7fd07019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:53.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd0667fc640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd070101420 0x7fd0701989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:53.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 -- 172.21.15.137:0/1077436569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd070106a80 msgr2=0x7fd07019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:53.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd070106a80 0x7fd07019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 -- 172.21.15.137:0/1077436569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd070101420 msgr2=0x7fd0701989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd070101420 0x7fd0701989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 -- 172.21.15.137:0/1077436569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07019d440 con 0x7fd070100a80 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd0667fc640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd070101420 0x7fd0701989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06f7fe640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd070106a80 0x7fd07019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd06effd640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070100a80 0x7fd070198490 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd06000d770 tx=0x7fd06000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd074960640 1 -- 172.21.15.137:0/1077436569 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060017070 con 0x7fd070100a80 2024-09-12T12:33:53.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07019d730 con 0x7fd070100a80 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd074960640 1 -- 172.21.15.137:0/1077436569 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd060005500 con 0x7fd070100a80 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd074960640 1 -- 172.21.15.137:0/1077436569 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd060004e00 con 0x7fd070100a80 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.605+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0701a5010 con 0x7fd070100a80 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.606+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd070100ee0 con 0x7fd070100a80 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.607+0000 7fd074960640 1 -- 172.21.15.137:0/1077436569 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd060004fa0 con 0x7fd070100a80 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.608+0000 7fd074960640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd05c0a2cf0 0x7fd05c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.608+0000 7fd0667fc640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd05c0a2cf0 0x7fd05c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:53.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.608+0000 7fd074960640 1 -- 172.21.15.137:0/1077436569 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8484+0+0 (secure 0 0 0) 0x7fd0600cef00 con 0x7fd070100a80 2024-09-12T12:33:53.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.609+0000 7fd0667fc640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd05c0a2cf0 0x7fd05c0a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0580058c0 tx=0x7fd058005850 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:53.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.612+0000 7fd074960640 1 -- 172.21.15.137:0/1077436569 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd06009bb40 con 0x7fd070100a80 2024-09-12T12:33:53.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.782+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd07010a000 con 0x7fd070100a80 2024-09-12T12:33:53.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.783+0000 7fd074960640 1 -- 172.21.15.137:0/1077436569 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+23211 (secure 0 0 0) 0x7fd06009b440 con 0x7fd070100a80 2024-09-12T12:33:53.783 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-09-12T12:33:53.784 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":120,"fsid":"dd66268a-70fe-11ef-bcea-c7b262605968","created":"2024-09-12T12:03:22.696612+0000","modified":"2024-09-12T12:33:22.751291+0000","last_up_change":"2024-09-12T12:10:19.792374+0000","last_in_change":"2024-09-12T12:09:56.336373+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T12:06:29.335398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T12:11:08.337441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T12:11:10.709230+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"45d33c8c-abf3-4322-aeb0-28f4141abb52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6803","nonce":655478089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6805","nonce":655478089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6809","nonce":655478089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":655478089},{"type":"v1","addr":"172.21.15.137:6807","nonce":655478089}]},"public_addr":"172.21.15.137:6803/655478089","cluster_addr":"172.21.15.137:6805/655478089","heartbeat_back_addr":"172.21.15.137:6809/655478089","heartbeat_front_addr":"172.21.15.137:6807/655478089","state":["exists","up"]},{"osd":1,"uuid":"a1a7f639-1cce-4a38-b5b2-9e6035281396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6801","nonce":1953204063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6803","nonce":1953204063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6807","nonce":1953204063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1953204063},{"type":"v1","addr":"172.21.15.171:6805","nonce":1953204063}]},"public_addr":"172.21.15.171:6801/1953204063","cluster_addr":"172.21.15.171:6803/1953204063","heartbeat_back_addr":"172.21.15.171:6807/1953204063","heartbeat_front_addr":"172.21.15.171:6805/1953204063","state":["exists","up"]},{"osd":2,"uuid":"845d927b-63b2-4eba-961c-222c975d8779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6801","nonce":2022208006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6803","nonce":2022208006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6807","nonce":2022208006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2022208006},{"type":"v1","addr":"172.21.15.189:6805","nonce":2022208006}]},"public_addr":"172.21.15.189:6801/2022208006","cluster_addr":"172.21.15.189:6803/2022208006","heartbeat_back_addr":"172.21.15.189:6807/2022208006","heartbeat_front_addr":"172.21.15.189:6805/2022208006","state":["exists","up"]},{"osd":3,"uuid":"825bb9bf-6e25-4933-b0fc-4be4950fac0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6811","nonce":1732422087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6813","nonce":1732422087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6817","nonce":1732422087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":1732422087},{"type":"v1","addr":"172.21.15.137:6815","nonce":1732422087}]},"public_addr":"172.21.15.137:6811/1732422087","cluster_addr":"172.21.15.137:6813/1732422087","heartbeat_back_addr":"172.21.15.137:6817/1732422087","heartbeat_front_addr":"172.21.15.137:6815/1732422087","state":["exists","up"]},{"osd":4,"uuid":"8d266b7f-3149-4edd-b3d3-15c3ba62ce7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6809","nonce":1838766520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6811","nonce":1838766520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6815","nonce":1838766520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1838766520},{"type":"v1","addr":"172.21.15.171:6813","nonce":1838766520}]},"public_addr":"172.21.15.171:6809/1838766520","cluster_addr":"172.21.15.171:6811/1838766520","heartbeat_back_addr":"172.21.15.171:6815/1838766520","heartbeat_front_addr":"172.21.15.171:6813/1838766520","state":["exists","up"]},{"osd":5,"uuid":"80eab3c2-6d42-4a43-a151-b8e92c20dda1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6809","nonce":3684989511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6811","nonce":3684989511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6815","nonce":3684989511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3684989511},{"type":"v1","addr":"172.21.15.189:6813","nonce":3684989511}]},"public_addr":"172.21.15.189:6809/3684989511","cluster_addr":"172.21.15.189:6811/3684989511","heartbeat_back_addr":"172.21.15.189:6815/3684989511","heartbeat_front_addr":"172.21.15.189:6813/3684989511","state":["exists","up"]},{"osd":6,"uuid":"4971246b-c619-43a1-a798-e0007f23cab0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6819","nonce":2955523379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6821","nonce":2955523379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6825","nonce":2955523379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2955523379},{"type":"v1","addr":"172.21.15.137:6823","nonce":2955523379}]},"public_addr":"172.21.15.137:6819/2955523379","cluster_addr":"172.21.15.137:6821/2955523379","heartbeat_back_addr":"172.21.15.137:6825/2955523379","heartbeat_front_addr":"172.21.15.137:6823/2955523379","state":["exists","up"]},{"osd":7,"uuid":"e16ef0a1-dcb9-45df-bde2-22fe86d53f55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6817","nonce":665345582}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6819","nonce":665345582}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6823","nonce":665345582}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":665345582},{"type":"v1","addr":"172.21.15.171:6821","nonce":665345582}]},"public_addr":"172.21.15.171:6817/665345582","cluster_addr":"172.21.15.171:6819/665345582","heartbeat_back_addr":"172.21.15.171:6823/665345582","heartbeat_front_addr":"172.21.15.171:6821/665345582","state":["exists","up"]},{"osd":8,"uuid":"88d4c0f4-ec90-4c43-a8da-53e18fcd1984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6817","nonce":1497312216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6819","nonce":1497312216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6823","nonce":1497312216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1497312216},{"type":"v1","addr":"172.21.15.189:6821","nonce":1497312216}]},"public_addr":"172.21.15.189:6817/1497312216","cluster_addr":"172.21.15.189:6819/1497312216","heartbeat_back_addr":"172.21.15.189:6823/1497312216","heartbeat_front_addr":"172.21.15.189:6821/1497312216","state":["exists","up"]},{"osd":9,"uuid":"e6098dac-95f6-45c0-8e0c-2299eb98ca05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6827","nonce":4232440350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6829","nonce":4232440350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6833","nonce":4232440350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":4232440350},{"type":"v1","addr":"172.21.15.137:6831","nonce":4232440350}]},"public_addr":"172.21.15.137:6827/4232440350","cluster_addr":"172.21.15.137:6829/4232440350","heartbeat_back_addr":"172.21.15.137:6833/4232440350","heartbeat_front_addr":"172.21.15.137:6831/4232440350","state":["exists","up"]},{"osd":10,"uuid":"579cd8cf-bf21-4438-9476-72080c6e193a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6825","nonce":124645600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6827","nonce":124645600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6831","nonce":124645600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":124645600},{"type":"v1","addr":"172.21.15.171:6829","nonce":124645600}]},"public_addr":"172.21.15.171:6825/124645600","cluster_addr":"172.21.15.171:6827/124645600","heartbeat_back_addr":"172.21.15.171:6831/124645600","heartbeat_front_addr":"172.21.15.171:6829/124645600","state":["exists","up"]},{"osd":11,"uuid":"26715dfc-c88e-42be-a958-ffdb37376a62","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6825","nonce":2395385355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6827","nonce":2395385355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6831","nonce":2395385355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":2395385355},{"type":"v1","addr":"172.21.15.189:6829","nonce":2395385355}]},"public_addr":"172.21.15.189:6825/2395385355","cluster_addr":"172.21.15.189:6827/2395385355","heartbeat_back_addr":"172.21.15.189:6831/2395385355","heartbeat_front_addr":"172.21.15.189:6829/2395385355","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:05:35.028547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:00.446751+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:26.164303+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:06:52.754312+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:17.880117+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:07:43.519473+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:09.190839+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:08:34.969136+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:00.575042+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:27.091477+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:09:52.085428+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T12:10:18.172569+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:0/4099842792":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/123979974":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/2550482799":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/1305616495":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/2039555152":"2024-09-13T12:13:13.350302+0000","172.21.15.137:0/2994676901":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6801/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/612748114":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/2160224100":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6801/1789953576":"2024-09-13T12:13:13.350302+0000","172.21.15.137:6800/1104802518":"2024-09-13T12:03:52.204874+0000","172.21.15.137:0/936885966":"2024-09-13T12:03:52.204874+0000","172.21.15.137:6801/1596672143":"2024-09-13T12:04:15.038962+0000","172.21.15.137:0/228183714":"2024-09-13T12:04:15.038962+0000","172.21.15.137:6800/1789953576":"2024-09-13T12:13:13.350302+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T12:33:53.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd05c0a2cf0 msgr2=0x7fd05c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:53.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd05c0a2cf0 0x7fd05c0a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd0580058c0 tx=0x7fd058005850 comp rx=0 tx=0).stop 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070100a80 msgr2=0x7fd070198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070100a80 0x7fd070198490 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd06000d770 tx=0x7fd06000dc30 comp rx=0 tx=0).stop 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 shutdown_connections 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fd05c0a2cf0 0x7fd05c0a51b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd070106a80 0x7fd07019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd070101420 0x7fd0701989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 --2- 172.21.15.137:0/1077436569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd070100a80 0x7fd070198490 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:53.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 >> 172.21.15.137:0/1077436569 conn(0x7fd0700fc7a0 msgr2=0x7fd07006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 shutdown_connections 2024-09-12T12:33:53.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:53.786+0000 7fd075962640 1 -- 172.21.15.137:0/1077436569 wait complete. 2024-09-12T12:33:54.499 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T12:33:54.743 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:55.026 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:54 smithi137 ceph-mon[30950]: pgmap v658: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-09-12T12:33:55.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:54 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1077436569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:33:55.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:54 smithi189 ceph-mon[35475]: pgmap v658: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-09-12T12:33:55.091 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:54 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1077436569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:33:55.200 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:54 smithi171 ceph-mon[35098]: pgmap v658: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 2.4 MiB/s wr, 72 op/s 2024-09-12T12:33:55.201 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:54 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1077436569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T12:33:55.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 -- 172.21.15.137:0/1958282685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798100e20 msgr2=0x7f1798101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:55.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 --2- 172.21.15.137:0/1958282685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798100e20 0x7f1798101200 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f178c009900 tx=0x7f178c02a480 comp rx=0 tx=0).stop 2024-09-12T12:33:55.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 -- 172.21.15.137:0/1958282685 shutdown_connections 2024-09-12T12:33:55.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 --2- 172.21.15.137:0/1958282685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1798102ef0 0x7f179810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 --2- 172.21.15.137:0/1958282685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1798102570 0x7f17981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 --2- 172.21.15.137:0/1958282685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798100e20 0x7f1798101200 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 -- 172.21.15.137:0/1958282685 >> 172.21.15.137:0/1958282685 conn(0x7f17980fc7d0 msgr2=0x7f17980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:55.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 -- 172.21.15.137:0/1958282685 shutdown_connections 2024-09-12T12:33:55.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.271+0000 7f179f542640 1 -- 172.21.15.137:0/1958282685 wait complete. 2024-09-12T12:33:55.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 Processor -- start 2024-09-12T12:33:55.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 -- start start 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1798100e20 0x7f1798198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1798102570 0x7f17981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798102ef0 0x7f179819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1798110f80 con 0x7f1798102ef0 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1798110e00 con 0x7f1798100e20 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179f542640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1798111100 con 0x7f1798102570 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798102ef0 0x7f179819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:55.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798102ef0 0x7f179819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60084/0 (socket says 172.21.15.137:60084) 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 -- 172.21.15.137:0/1796845142 learned_addr learned my addr 172.21.15.137:0/1796845142 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179d2de640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1798100e20 0x7f1798198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 -- 172.21.15.137:0/1796845142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1798102570 msgr2=0x7f17981989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1798102570 0x7f17981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 -- 172.21.15.137:0/1796845142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1798100e20 msgr2=0x7f1798198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1798100e20 0x7f1798198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179dadf640 1 -- 172.21.15.137:0/1796845142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179819d410 con 0x7f1798102ef0 2024-09-12T12:33:55.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.272+0000 7f179d2de640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1798100e20 0x7f1798198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:33:55.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.273+0000 7f179dadf640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798102ef0 0x7f179819cd30 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f179000cad0 tx=0x7f179000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:55.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.273+0000 7f178a7fc640 1 -- 172.21.15.137:0/1796845142 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179000ae20 con 0x7f1798102ef0 2024-09-12T12:33:55.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.273+0000 7f178a7fc640 1 -- 172.21.15.137:0/1796845142 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1790005020 con 0x7f1798102ef0 2024-09-12T12:33:55.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.273+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179819d700 con 0x7f1798102ef0 2024-09-12T12:33:55.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.273+0000 7f178a7fc640 1 -- 172.21.15.137:0/1796845142 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179000f6f0 con 0x7f1798102ef0 2024-09-12T12:33:55.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.273+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17981a4fe0 con 0x7f1798102ef0 2024-09-12T12:33:55.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.274+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1798101200 con 0x7f1798102ef0 2024-09-12T12:33:55.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.275+0000 7f178a7fc640 1 -- 172.21.15.137:0/1796845142 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f17900051d0 con 0x7f1798102ef0 2024-09-12T12:33:55.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.276+0000 7f178a7fc640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17840a2cf0 0x7f17840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:55.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.276+0000 7f179d2de640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17840a2cf0 0x7f17840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:55.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.276+0000 7f178a7fc640 1 -- 172.21.15.137:0/1796845142 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8484+0+0 (secure 0 0 0) 0x7f17900cf850 con 0x7f1798102ef0 2024-09-12T12:33:55.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.276+0000 7f179d2de640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17840a2cf0 0x7f17840a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f178c005ab0 tx=0x7f178c005a20 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:55.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.279+0000 7f178a7fc640 1 -- 172.21.15.137:0/1796845142 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1790014030 con 0x7f1798102ef0 2024-09-12T12:33:55.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.480+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f179810a0a0 con 0x7f1798102ef0 2024-09-12T12:33:55.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.483+0000 7f178a7fc640 1 -- 172.21.15.137:0/1796845142 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7f179009c490 con 0x7f1798102ef0 2024-09-12T12:33:55.482 INFO:teuthology.orchestra.run.smithi137.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T12:33:55.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17840a2cf0 msgr2=0x7f17840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:55.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17840a2cf0 0x7f17840a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f178c005ab0 tx=0x7f178c005a20 comp rx=0 tx=0).stop 2024-09-12T12:33:55.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798102ef0 msgr2=0x7f179819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:55.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798102ef0 0x7f179819cd30 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f179000cad0 tx=0x7f179000cf90 comp rx=0 tx=0).stop 2024-09-12T12:33:55.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 shutdown_connections 2024-09-12T12:33:55.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f17840a2cf0 0x7f17840a51b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1798102ef0 0x7f179819cd30 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1798102570 0x7f17981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 --2- 172.21.15.137:0/1796845142 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1798100e20 0x7f1798198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:55.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.486+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 >> 172.21.15.137:0/1796845142 conn(0x7f17980fc7d0 msgr2=0x7f179806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:55.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.487+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 shutdown_connections 2024-09-12T12:33:55.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:55.487+0000 7f179f542640 1 -- 172.21.15.137:0/1796845142 wait complete. 2024-09-12T12:33:55.776 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:55 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-12T12:33:55.481+0000 7f7d53ec5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:33:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1796845142' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T12:33:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mon[35475]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T12:33:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mon[35475]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:33:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1796845142' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T12:33:55.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mon[35475]: osdmap e121: 12 total, 12 up, 12 in 2024-09-12T12:33:55.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mon[35475]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T12:33:55.842 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c[65837]: ignoring --setuser ceph since I am not root 2024-09-12T12:33:55.842 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c[65837]: ignoring --setgroup ceph since I am not root 2024-09-12T12:33:55.842 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mds[65841]: -- 172.21.15.189:0/2992094686 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e24077860 con 0x563e24055400 2024-09-12T12:33:55.842 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c[65837]: starting mds.c at 2024-09-12T12:33:55.843 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setuser ceph since I am not root 2024-09-12T12:33:55.843 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: ignoring --setgroup ceph since I am not root 2024-09-12T12:33:55.843 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mds[65540]: -- 172.21.15.189:0/2032190416 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d5738d860 con 0x558d5736a800 2024-09-12T12:33:55.843 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: starting mds.f at 2024-09-12T12:33:55.844 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i[66137]: ignoring --setuser ceph since I am not root 2024-09-12T12:33:55.844 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i[66137]: ignoring --setgroup ceph since I am not root 2024-09-12T12:33:55.844 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mds[66141]: -- 172.21.15.189:0/2312647846 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555654701860 con 0x5556546df000 2024-09-12T12:33:55.844 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i[66137]: starting mds.i at 2024-09-12T12:33:55.844 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l[65234]: ignoring --setuser ceph since I am not root 2024-09-12T12:33:55.844 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l[65234]: ignoring --setgroup ceph since I am not root 2024-09-12T12:33:55.845 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-mds[65238]: -- 172.21.15.189:0/2572936029 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d27706bc20 con 0x55d277049400 2024-09-12T12:33:55.845 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:33:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l[65234]: starting mds.l at 2024-09-12T12:33:55.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1796845142' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T12:33:55.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mon[35098]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T12:33:55.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mon[35098]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:33:55.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1796845142' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T12:33:55.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mon[35098]: osdmap e121: 12 total, 12 up, 12 in 2024-09-12T12:33:55.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mon[35098]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T12:33:55.951 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h[64643]: ignoring --setuser ceph since I am not root 2024-09-12T12:33:55.951 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h[64643]: ignoring --setgroup ceph since I am not root 2024-09-12T12:33:55.951 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mds[64647]: -- 172.21.15.171:0/1857606863 <== mon.2 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe839df860 con 0x55fe839bd400 2024-09-12T12:33:55.952 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h[64643]: starting mds.h at 2024-09-12T12:33:55.952 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: ignoring --setuser ceph since I am not root 2024-09-12T12:33:55.952 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: ignoring --setgroup ceph since I am not root 2024-09-12T12:33:55.952 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-mds[64962]: -- 172.21.15.171:0/2276323240 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fda04a5c20 con 0x55fda0483000 2024-09-12T12:33:55.952 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: starting mds.b at 2024-09-12T12:33:55.952 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:33:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: 2024-09-12T12:33:55.563+0000 7f615bc90640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6836/0: (98) Address already in use 2024-09-12T12:33:56.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:55 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1796845142' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T12:33:56.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:55 smithi137 ceph-mon[30950]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T12:33:56.049 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:55 smithi137 ceph-mon[30950]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T12:33:56.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:55 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1796845142' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T12:33:56.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:55 smithi137 ceph-mon[30950]: osdmap e121: 12 total, 12 up, 12 in 2024-09-12T12:33:56.050 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:55 smithi137 ceph-mon[30950]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-12T12:33:56.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T12:33:56.404 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:56.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 -- 172.21.15.137:0/1538705970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5184074620 msgr2=0x7f5184074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:56.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 --2- 172.21.15.137:0/1538705970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5184074620 0x7f5184074a00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5174009900 tx=0x7f517402a480 comp rx=0 tx=0).stop 2024-09-12T12:33:56.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 -- 172.21.15.137:0/1538705970 shutdown_connections 2024-09-12T12:33:56.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 --2- 172.21.15.137:0/1538705970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f518410e3c0 0x7f5184110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:56.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 --2- 172.21.15.137:0/1538705970 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5184074f40 0x7f518410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:56.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 --2- 172.21.15.137:0/1538705970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5184074620 0x7f5184074a00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:56.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 -- 172.21.15.137:0/1538705970 >> 172.21.15.137:0/1538705970 conn(0x7f51840fd8c0 msgr2=0x7f51840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:56.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 -- 172.21.15.137:0/1538705970 shutdown_connections 2024-09-12T12:33:56.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 -- 172.21.15.137:0/1538705970 wait complete. 2024-09-12T12:33:56.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 Processor -- start 2024-09-12T12:33:56.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.933+0000 7f518a1dd640 1 -- start start 2024-09-12T12:33:56.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f518a1dd640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5184074620 0x7f518419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:56.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f518a1dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5184074f40 0x7f518419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:56.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f518a1dd640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f518410e3c0 0x7f518419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:56.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f518a1dd640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5184114170 con 0x7f518410e3c0 2024-09-12T12:33:56.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f518a1dd640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5184113ff0 con 0x7f5184074620 2024-09-12T12:33:56.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f518a1dd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f51841142f0 con 0x7f5184074f40 2024-09-12T12:33:56.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f518410e3c0 0x7f518419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f518410e3c0 0x7f518419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60100/0 (socket says 172.21.15.137:60100) 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 -- 172.21.15.137:0/2356093373 learned_addr learned my addr 172.21.15.137:0/2356093373 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f51837fe640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5184074620 0x7f518419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f517affd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5184074f40 0x7f518419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 -- 172.21.15.137:0/2356093373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5184074f40 msgr2=0x7f518419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5184074f40 0x7f518419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 -- 172.21.15.137:0/2356093373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5184074620 msgr2=0x7f518419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:56.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5184074620 0x7f518419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 -- 172.21.15.137:0/2356093373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518419f640 con 0x7f518410e3c0 2024-09-12T12:33:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f517affd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5184074f40 0x7f518419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T12:33:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.934+0000 7f5183fff640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f518410e3c0 0x7f518419ef60 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f517c00cad0 tx=0x7f517c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.935+0000 7f51817fa640 1 -- 172.21.15.137:0/2356093373 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c00ae20 con 0x7f518410e3c0 2024-09-12T12:33:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.935+0000 7f51817fa640 1 -- 172.21.15.137:0/2356093373 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f517c005020 con 0x7f518410e3c0 2024-09-12T12:33:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.935+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518419f930 con 0x7f518410e3c0 2024-09-12T12:33:56.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.935+0000 7f51817fa640 1 -- 172.21.15.137:0/2356093373 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517c00f6f0 con 0x7f518410e3c0 2024-09-12T12:33:56.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.935+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51841a7210 con 0x7f518410e3c0 2024-09-12T12:33:56.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.936+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5184074a00 con 0x7f518410e3c0 2024-09-12T12:33:56.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.937+0000 7f51817fa640 1 -- 172.21.15.137:0/2356093373 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f517c0051d0 con 0x7f518410e3c0 2024-09-12T12:33:56.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.937+0000 7f51817fa640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f51700a2d70 0x7f51700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:56.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.938+0000 7f51837fe640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f51700a2d70 0x7f51700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:56.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.938+0000 7f51817fa640 1 -- 172.21.15.137:0/2356093373 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8698+0+0 (secure 0 0 0) 0x7f517c0cf530 con 0x7f518410e3c0 2024-09-12T12:33:56.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.938+0000 7f51837fe640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f51700a2d70 0x7f51700a5230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51740096c0 tx=0x7f5174031490 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:56.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:56.942+0000 7f51817fa640 1 -- 172.21.15.137:0/2356093373 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f517c014030 con 0x7f518410e3c0 2024-09-12T12:33:57.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.142+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f51841095a0 con 0x7f518410e3c0 2024-09-12T12:33:57.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.496+0000 7f51817fa640 1 -- 172.21.15.137:0/2356093373 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f517c09c0a0 con 0x7f518410e3c0 2024-09-12T12:33:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f51700a2d70 msgr2=0x7f51700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f51700a2d70 0x7f51700a5230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51740096c0 tx=0x7f5174031490 comp rx=0 tx=0).stop 2024-09-12T12:33:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f518410e3c0 msgr2=0x7f518419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f518410e3c0 0x7f518419ef60 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f517c00cad0 tx=0x7f517c00cf90 comp rx=0 tx=0).stop 2024-09-12T12:33:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 shutdown_connections 2024-09-12T12:33:57.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f51700a2d70 0x7f51700a5230 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:57.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f518410e3c0 0x7f518419ef60 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:57.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5184074f40 0x7f518419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:57.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 --2- 172.21.15.137:0/2356093373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5184074620 0x7f518419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:57.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 >> 172.21.15.137:0/2356093373 conn(0x7f51840fd8c0 msgr2=0x7f51840ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:57.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 shutdown_connections 2024-09-12T12:33:57.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:57.499+0000 7f518a1dd640 1 -- 172.21.15.137:0/2356093373 wait complete. 2024-09-12T12:33:57.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-12T12:33:57.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6832/3145771703,v1:172.21.15.189:6834/3145771703] up:boot 2024-09-12T12:33:57.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6833/1912370919,v1:172.21.15.189:6835/1912370919] up:boot 2024-09-12T12:33:57.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6836/573050978,v1:172.21.15.189:6838/573050978] up:boot 2024-09-12T12:33:57.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.189:6837/1496846684,v1:172.21.15.189:6839/1496846684] up:boot 2024-09-12T12:33:57.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.171:6836/919921968,v1:172.21.15.171:6838/919921968] up:boot 2024-09-12T12:33:57.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: mds.? [v2:172.21.15.171:6837/2286601386,v1:172.21.15.171:6839/2286601386] up:boot 2024-09-12T12:33:57.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T12:33:57.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:33:57.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:33:57.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:33:57.801 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:33:57.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:33:57.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:33:57.802 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:57 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2356093373' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:33:57.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-12T12:33:57.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6832/3145771703,v1:172.21.15.189:6834/3145771703] up:boot 2024-09-12T12:33:57.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6833/1912370919,v1:172.21.15.189:6835/1912370919] up:boot 2024-09-12T12:33:57.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6836/573050978,v1:172.21.15.189:6838/573050978] up:boot 2024-09-12T12:33:57.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.189:6837/1496846684,v1:172.21.15.189:6839/1496846684] up:boot 2024-09-12T12:33:57.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.171:6836/919921968,v1:172.21.15.171:6838/919921968] up:boot 2024-09-12T12:33:57.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: mds.? [v2:172.21.15.171:6837/2286601386,v1:172.21.15.171:6839/2286601386] up:boot 2024-09-12T12:33:57.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T12:33:57.842 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:33:57.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:33:57.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:33:57.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:33:57.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:33:57.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:33:57.843 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:57 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2356093373' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:33:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: pgmap v660: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-12T12:33:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6832/3145771703,v1:172.21.15.189:6834/3145771703] up:boot 2024-09-12T12:33:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6833/1912370919,v1:172.21.15.189:6835/1912370919] up:boot 2024-09-12T12:33:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6836/573050978,v1:172.21.15.189:6838/573050978] up:boot 2024-09-12T12:33:57.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.189:6837/1496846684,v1:172.21.15.189:6839/1496846684] up:boot 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.171:6836/919921968,v1:172.21.15.171:6838/919921968] up:boot 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: mds.? [v2:172.21.15.171:6837/2286601386,v1:172.21.15.171:6839/2286601386] up:boot 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T12:33:57.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T12:33:57.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: from='mgr.15171 172.21.15.137:0/3863560253' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T12:33:57.952 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:57 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2356093373' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T12:33:58.226 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T12:33:58.472 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:33:58.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:58 smithi137 ceph-mon[30950]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T12:33:58.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:58 smithi137 ceph-mon[30950]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T12:33:58.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:58 smithi137 ceph-mon[30950]: Cluster is now healthy 2024-09-12T12:33:58.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:58 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/2356093373' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T12:33:58.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:58 smithi137 ceph-mon[30950]: fsmap 12 up:standby 2024-09-12T12:33:58.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:58 smithi189 ceph-mon[35475]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T12:33:58.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:58 smithi189 ceph-mon[35475]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T12:33:58.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:58 smithi189 ceph-mon[35475]: Cluster is now healthy 2024-09-12T12:33:58.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:58 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/2356093373' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T12:33:58.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:58 smithi189 ceph-mon[35475]: fsmap 12 up:standby 2024-09-12T12:33:58.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:58 smithi171 ceph-mon[35098]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T12:33:58.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:58 smithi171 ceph-mon[35098]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T12:33:58.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:58 smithi171 ceph-mon[35098]: Cluster is now healthy 2024-09-12T12:33:58.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:58 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/2356093373' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T12:33:58.951 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:58 smithi171 ceph-mon[35098]: fsmap 12 up:standby 2024-09-12T12:33:59.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/472412120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8100a80 msgr2=0x7f4cd8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:59.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/472412120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8100a80 0x7f4cd8100ee0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009930 tx=0x7f4cd402a420 comp rx=0 tx=0).stop 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/472412120 shutdown_connections 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/472412120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cd8101420 0x7f4cd810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/472412120 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8100a80 0x7f4cd8100ee0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/472412120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cd8106a80 0x7f4cd8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/472412120 >> 172.21.15.137:0/472412120 conn(0x7f4cd80fc7a0 msgr2=0x7f4cd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/472412120 shutdown_connections 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.076+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/472412120 wait complete. 2024-09-12T12:33:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 Processor -- start 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 -- start start 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cd8100a80 0x7f4cd8197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cd8101420 0x7f4cd81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8106a80 0x7f4cd819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8110e40 con 0x7f4cd8106a80 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8110cc0 con 0x7f4cd8101420 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.077+0000 7f4cdfd7d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8110fc0 con 0x7f4cd8100a80 2024-09-12T12:33:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8106a80 0x7f4cd819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cddb19640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cd8100a80 0x7f4cd8197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8106a80 0x7f4cd819bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60114/0 (socket says 172.21.15.137:60114) 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 -- 172.21.15.137:0/57749496 learned_addr learned my addr 172.21.15.137:0/57749496 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cdd318640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cd8101420 0x7f4cd81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 -- 172.21.15.137:0/57749496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cd8100a80 msgr2=0x7f4cd8197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cd8100a80 0x7f4cd8197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 -- 172.21.15.137:0/57749496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cd8101420 msgr2=0x7f4cd81978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:59.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cd8101420 0x7f4cd81978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 -- 172.21.15.137:0/57749496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd819c330 con 0x7f4cd8106a80 2024-09-12T12:33:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cddb19640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cd8100a80 0x7f4cd8197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T12:33:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cde31a640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8106a80 0x7f4cd819bc50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000eca0 tx=0x7f4cd000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4ccaffd640 1 -- 172.21.15.137:0/57749496 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0004b20 con 0x7f4cd8106a80 2024-09-12T12:33:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4ccaffd640 1 -- 172.21.15.137:0/57749496 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd0004cc0 con 0x7f4cd8106a80 2024-09-12T12:33:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd819c620 con 0x7f4cd8106a80 2024-09-12T12:33:59.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4ccaffd640 1 -- 172.21.15.137:0/57749496 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0017650 con 0x7f4cd8106a80 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.078+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd806a790 con 0x7f4cd8106a80 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.079+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd81021c0 con 0x7f4cd8106a80 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.081+0000 7f4ccaffd640 1 -- 172.21.15.137:0/57749496 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4cd0005000 con 0x7f4cd8106a80 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.081+0000 7f4ccaffd640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4cc40a2cf0 0x7f4cc40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.081+0000 7f4cddb19640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4cc40a2cf0 0x7f4cc40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.081+0000 7f4ccaffd640 1 -- 172.21.15.137:0/57749496 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8698+0+0 (secure 0 0 0) 0x7f4cd00d17c0 con 0x7f4cd8106a80 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.082+0000 7f4cddb19640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4cc40a2cf0 0x7f4cc40a51b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009f40 tx=0x7f4ccc005f90 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:33:59.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.085+0000 7f4ccaffd640 1 -- 172.21.15.137:0/57749496 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cd009e330 con 0x7f4cd8106a80 2024-09-12T12:33:59.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.251+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4cd8100ee0 con 0x7f4cd8106a80 2024-09-12T12:33:59.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.503+0000 7f4ccaffd640 1 -- 172.21.15.137:0/57749496 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7f4cd8100ee0 con 0x7f4cd8106a80 2024-09-12T12:33:59.503 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_metadata' removed 2024-09-12T12:33:59.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4cc40a2cf0 msgr2=0x7f4cc40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:59.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4cc40a2cf0 0x7f4cc40a51b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009f40 tx=0x7f4ccc005f90 comp rx=0 tx=0).stop 2024-09-12T12:33:59.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8106a80 msgr2=0x7f4cd819bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:33:59.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8106a80 0x7f4cd819bc50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4cd000eca0 tx=0x7f4cd000b690 comp rx=0 tx=0).stop 2024-09-12T12:33:59.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 shutdown_connections 2024-09-12T12:33:59.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7f4cc40a2cf0 0x7f4cc40a51b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4cd8106a80 0x7f4cd819bc50 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cd8101420 0x7f4cd81978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 --2- 172.21.15.137:0/57749496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cd8100a80 0x7f4cd8197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:33:59.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.506+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 >> 172.21.15.137:0/57749496 conn(0x7f4cd80fc7a0 msgr2=0x7f4cd80fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:33:59.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.507+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 shutdown_connections 2024-09-12T12:33:59.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:33:59.510+0000 7f4cdfd7d640 1 -- 172.21.15.137:0/57749496 wait complete. 2024-09-12T12:33:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:59 smithi137 ceph-mon[30950]: pgmap v661: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-12T12:33:59.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:33:59 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/57749496' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T12:33:59.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:59 smithi189 ceph-mon[35475]: pgmap v661: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-12T12:33:59.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:33:59 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/57749496' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T12:33:59.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:59 smithi171 ceph-mon[35098]: pgmap v661: 97 pgs: 97 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-09-12T12:33:59.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:33:59 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/57749496' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T12:34:00.258 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid dd66268a-70fe-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T12:34:00.515 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/57749496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T12:34:00.515 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-mon[30950]: osdmap e122: 12 total, 12 up, 12 in 2024-09-12T12:34:00.516 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:34:00.383+0000 7f4f29363640 -1 client.15210: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-12T12:34:00.516 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:34:00.384+0000 7f4f29363640 -1 client.15210: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-12T12:34:00.516 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Exception in thread Thread-25: 2024-09-12T12:34:00.516 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: Traceback (most recent call last): 2024-09-12T12:34:00.516 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-12T12:34:00.517 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self.run() 2024-09-12T12:34:00.517 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-12T12:34:00.517 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self.function(*self.args, **self.kwargs) 2024-09-12T12:34:00.517 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-12T12:34:00.517 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: self.prune_snapshots(sched) 2024-09-12T12:34:00.517 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-12T12:34:00.581 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T12:34:00.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:00 smithi137 ceph-mon[30950]: pgmap v663: 65 pgs: 65 active+clean; 349 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:34:00.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:00 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/57749496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T12:34:00.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:00 smithi189 ceph-mon[35475]: osdmap e122: 12 total, 12 up, 12 in 2024-09-12T12:34:00.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:00 smithi189 ceph-mon[35475]: pgmap v663: 65 pgs: 65 active+clean; 349 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:34:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:00 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/57749496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T12:34:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:00 smithi171 ceph-mon[35098]: osdmap e122: 12 total, 12 up, 12 in 2024-09-12T12:34:00.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:00 smithi171 ceph-mon[35098]: pgmap v663: 65 pgs: 65 active+clean; 349 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:34:01.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- 172.21.15.137:0/3882992434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa274102d40 msgr2=0x7fa274103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:34:01.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 --2- 172.21.15.137:0/3882992434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa274102d40 0x7fa274103120 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fa270009900 tx=0x7fa27002a480 comp rx=0 tx=0).stop 2024-09-12T12:34:01.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- 172.21.15.137:0/3882992434 shutdown_connections 2024-09-12T12:34:01.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 --2- 172.21.15.137:0/3882992434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27410c170 0x7fa27410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 --2- 172.21.15.137:0/3882992434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa274103660 0x7fa27410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 --2- 172.21.15.137:0/3882992434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa274102d40 0x7fa274103120 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- 172.21.15.137:0/3882992434 >> 172.21.15.137:0/3882992434 conn(0x7fa2740fc7a0 msgr2=0x7fa2740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:34:01.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- 172.21.15.137:0/3882992434 shutdown_connections 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- 172.21.15.137:0/3882992434 wait complete. 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 Processor -- start 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- start start 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa274102d40 0x7fa274198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa274103660 0x7fa274198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27410c170 0x7fa27419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa274111f70 con 0x7fa27410c170 2024-09-12T12:34:01.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa274111df0 con 0x7fa274102d40 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.102+0000 7fa27c07e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741120f0 con 0x7fa274103660 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27410c170 0x7fa27419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27410c170 0x7fa27419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45310/0 (socket says 172.21.15.137:45310) 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 -- 172.21.15.137:0/1898824452 learned_addr learned my addr 172.21.15.137:0/1898824452 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa279e1a640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa274102d40 0x7fa274198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa279619640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa274103660 0x7fa274198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 -- 172.21.15.137:0/1898824452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa274103660 msgr2=0x7fa274198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:34:01.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa274103660 0x7fa274198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 -- 172.21.15.137:0/1898824452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa274102d40 msgr2=0x7fa274198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:34:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa274102d40 0x7fa274198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 -- 172.21.15.137:0/1898824452 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27419d3c0 con 0x7fa27410c170 2024-09-12T12:34:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa279619640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa274103660 0x7fa274198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T12:34:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27a61b640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27410c170 0x7fa27419cce0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00eb10 tx=0x7fa26c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:34:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa26affd640 1 -- 172.21.15.137:0/1898824452 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004b20 con 0x7fa27410c170 2024-09-12T12:34:01.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27419d6b0 con 0x7fa27410c170 2024-09-12T12:34:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa26affd640 1 -- 172.21.15.137:0/1898824452 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa26c004cc0 con 0x7fa27410c170 2024-09-12T12:34:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa26affd640 1 -- 172.21.15.137:0/1898824452 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c010760 con 0x7fa27410c170 2024-09-12T12:34:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.103+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2741a4f90 con 0x7fa27410c170 2024-09-12T12:34:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.104+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa274103120 con 0x7fa27410c170 2024-09-12T12:34:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.105+0000 7fa26affd640 1 -- 172.21.15.137:0/1898824452 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa26c005020 con 0x7fa27410c170 2024-09-12T12:34:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.106+0000 7fa26affd640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa2600a2cf0 0x7fa2600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T12:34:01.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.106+0000 7fa26affd640 1 -- 172.21.15.137:0/1898824452 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8202+0+0 (secure 0 0 0) 0x7fa26c0cfa00 con 0x7fa27410c170 2024-09-12T12:34:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.106+0000 7fa279e1a640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa2600a2cf0 0x7fa2600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T12:34:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.106+0000 7fa279e1a640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa2600a2cf0 0x7fa2600a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa270009230 tx=0x7fa2700058d0 comp rx=0 tx=0).ready entity=mgr.15171 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T12:34:01.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.110+0000 7fa26affd640 1 -- 172.21.15.137:0/1898824452 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa26c09c760 con 0x7fa27410c170 2024-09-12T12:34:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.277+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa274106c80 con 0x7fa27410c170 2024-09-12T12:34:01.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.517+0000 7fa26affd640 1 -- 172.21.15.137:0/1898824452 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7fa26c09c060 con 0x7fa27410c170 2024-09-12T12:34:01.517 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_data' removed 2024-09-12T12:34:01.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa2600a2cf0 msgr2=0x7fa2600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:34:01.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa2600a2cf0 0x7fa2600a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa270009230 tx=0x7fa2700058d0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27410c170 msgr2=0x7fa27419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T12:34:01.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27410c170 0x7fa27419cce0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00eb10 tx=0x7fa26c00c2d0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 shutdown_connections 2024-09-12T12:34:01.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:6800/141828178,v1:172.21.15.137:6801/141828178] conn(0x7fa2600a2cf0 0x7fa2600a51b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa27410c170 0x7fa27419cce0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa274103660 0x7fa274198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 --2- 172.21.15.137:0/1898824452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa274102d40 0x7fa274198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T12:34:01.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 >> 172.21.15.137:0/1898824452 conn(0x7fa2740fc7a0 msgr2=0x7fa2740fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T12:34:01.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.520+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 shutdown_connections 2024-09-12T12:34:01.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-09-12T12:34:01.521+0000 7fa27c07e640 1 -- 172.21.15.137:0/1898824452 wait complete. 2024-09-12T12:34:01.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:01 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1898824452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T12:34:01.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:01 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1898824452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T12:34:01.949 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:01 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1898824452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T12:34:02.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:34:02.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:34:02.262 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:34:02.305 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T12:34:02.305 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T12:34:02.321 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T12:34:02.340 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T12:34:02.361 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T12:34:02.362 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T12:34:02.362 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a 2024-09-12T12:34:02.524 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:02 smithi137 systemd[1]: Stopping Ceph mon.a for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:02.590 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:02 smithi189 ceph-mon[35475]: from='client.? 172.21.15.137:0/1898824452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T12:34:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:02 smithi189 ceph-mon[35475]: osdmap e123: 12 total, 12 up, 12 in 2024-09-12T12:34:02.591 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:02 smithi189 ceph-mon[35475]: pgmap v665: 1 pgs: 1 active+clean; 577 KiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:34:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:02 smithi137 ceph-mon[30950]: from='client.? 172.21.15.137:0/1898824452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T12:34:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:02 smithi137 ceph-mon[30950]: osdmap e123: 12 total, 12 up, 12 in 2024-09-12T12:34:02.799 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:02 smithi137 ceph-mon[30950]: pgmap v665: 1 pgs: 1 active+clean; 577 KiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:34:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-12T12:34:02.663+0000 7f7d596d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a[30946]: 2024-09-12T12:34:02.663+0000 7f7d596d0640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T12:34:02.800 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:02 smithi137 podman[102671]: 2024-09-12 12:34:02.755059122 +0000 UTC m=+0.200948452 container died f9bc5e1ab701817bb623b06976269668ae1f98e19277476c5080fd57b4f4b94a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T12:34:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:02 smithi171 ceph-mon[35098]: from='client.? 172.21.15.137:0/1898824452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T12:34:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:02 smithi171 ceph-mon[35098]: osdmap e123: 12 total, 12 up, 12 in 2024-09-12T12:34:02.950 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:02 smithi171 ceph-mon[35098]: pgmap v665: 1 pgs: 1 active+clean; 577 KiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T12:34:03.439 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:03 smithi137 podman[102671]: 2024-09-12 12:34:03.17962495 +0000 UTC m=+0.625514271 container cleanup f9bc5e1ab701817bb623b06976269668ae1f98e19277476c5080fd57b4f4b94a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:34:03.439 INFO:journalctl@ceph.mon.a.smithi137.stdout:Sep 12 12:34:03 smithi137 bash[102671]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-a 2024-09-12T12:34:03.658 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T12:34:03.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:03.702 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T12:34:03.702 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T12:34:03.702 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.b 2024-09-12T12:34:04.077 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:03 smithi171 systemd[1]: Stopping Ceph mon.b for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:04.350 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b[35094]: 2024-09-12T12:34:04.077+0000 7fdf8cb90640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:04.351 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b[35094]: 2024-09-12T12:34:04.077+0000 7fdf8cb90640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T12:34:04.351 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:04 smithi171 podman[72474]: 2024-09-12 12:34:04.199854445 +0000 UTC m=+0.252147171 container died 03ba81652799a972d2b01af54dc071512d5243d3e5c0a82130b1972b13170f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-12T12:34:04.603 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:04 smithi171 podman[72474]: 2024-09-12 12:34:04.585257793 +0000 UTC m=+0.637550498 container cleanup 03ba81652799a972d2b01af54dc071512d5243d3e5c0a82130b1972b13170f5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T12:34:04.603 INFO:journalctl@ceph.mon.b.smithi171.stdout:Sep 12 12:34:04 smithi171 bash[72474]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-b 2024-09-12T12:34:05.069 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T12:34:05.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:05.120 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T12:34:05.120 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T12:34:05.120 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.c 2024-09-12T12:34:05.533 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:05 smithi189 systemd[1]: Stopping Ceph mon.c for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:05.840 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-c[35471]: 2024-09-12T12:34:05.532+0000 7f41a5685640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:05.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-c[35471]: 2024-09-12T12:34:05.532+0000 7f41a5685640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T12:34:05.841 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:05 smithi189 podman[74953]: 2024-09-12 12:34:05.716749047 +0000 UTC m=+0.344903624 container died 8560af655c48a78dc1190e710553c6f2149fa156267ed4299d02772a73178a8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T12:34:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:06 smithi189 podman[74953]: 2024-09-12 12:34:06.125243912 +0000 UTC m=+0.753398491 container cleanup 8560af655c48a78dc1190e710553c6f2149fa156267ed4299d02772a73178a8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:34:06.341 INFO:journalctl@ceph.mon.c.smithi189.stdout:Sep 12 12:34:06 smithi189 bash[74953]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mon-c 2024-09-12T12:34:06.530 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T12:34:06.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:06.600 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T12:34:06.601 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T12:34:06.601 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x 2024-09-12T12:34:06.909 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:06 smithi137 systemd[1]: Stopping Ceph mgr.x for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:07.235 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:06 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x[31176]: 2024-09-12T12:34:06.908+0000 7f4fc2d3c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:07.235 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:06 smithi137 podman[102788]: 2024-09-12 12:34:06.940503231 +0000 UTC m=+0.155517203 container died 9413d1f0ffbbd8d89fac9924063b5504724ee44b2f9a1ee7de95713b49b1b677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:34:07.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:07 smithi137 podman[102788]: 2024-09-12 12:34:07.368494652 +0000 UTC m=+0.583508624 container cleanup 9413d1f0ffbbd8d89fac9924063b5504724ee44b2f9a1ee7de95713b49b1b677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T12:34:07.549 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:07 smithi137 bash[102788]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x 2024-09-12T12:34:07.841 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T12:34:07.864 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:07 smithi137 podman[102800]: 2024-09-12 12:34:07.603825496 +0000 UTC m=+0.657810523 container remove 9413d1f0ffbbd8d89fac9924063b5504724ee44b2f9a1ee7de95713b49b1b677 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:34:07.864 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:07 smithi137 systemd[1]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-12T12:34:07.864 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:07 smithi137 systemd[1]: Stopped Ceph mgr.x for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:34:07.865 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Sep 12 12:34:07 smithi137 systemd[1]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.x.service: Consumed 53.285s CPU time. 2024-09-12T12:34:07.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:07.878 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T12:34:07.878 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T12:34:07.878 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.y 2024-09-12T12:34:08.200 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:34:07 smithi171 systemd[1]: Stopping Ceph mgr.y for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:08.575 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:34:08 smithi171 podman[72607]: 2024-09-12 12:34:08.247601079 +0000 UTC m=+0.178708706 container died ac545c0fa2692ea33d0492c5614f047bbab56e3ebca99661f21a655f859d58c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T12:34:08.925 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:34:08 smithi171 podman[72607]: 2024-09-12 12:34:08.682301843 +0000 UTC m=+0.613409458 container cleanup ac545c0fa2692ea33d0492c5614f047bbab56e3ebca99661f21a655f859d58c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-12T12:34:08.925 INFO:journalctl@ceph.mgr.y.smithi171.stdout:Sep 12 12:34:08 smithi171 bash[72607]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-y 2024-09-12T12:34:09.136 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T12:34:09.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:09.174 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T12:34:09.174 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T12:34:09.174 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.z 2024-09-12T12:34:09.518 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:34:09 smithi189 systemd[1]: Stopping Ceph mgr.z for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:09.823 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:34:09 smithi189 podman[75085]: 2024-09-12 12:34:09.518824602 +0000 UTC m=+0.156247584 container died 39d267a0e552249fa20646fedc55473d07c6184e16797404e50afa091d276e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T12:34:10.090 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:34:09 smithi189 podman[75085]: 2024-09-12 12:34:09.940589042 +0000 UTC m=+0.578012010 container cleanup 39d267a0e552249fa20646fedc55473d07c6184e16797404e50afa091d276e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T12:34:10.091 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Sep 12 12:34:09 smithi189 bash[75085]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mgr-z 2024-09-12T12:34:10.378 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T12:34:10.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:10.414 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T12:34:10.415 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T12:34:10.415 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.0 2024-09-12T12:34:10.770 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:10 smithi137 systemd[1]: Stopping Ceph osd.0 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:11.049 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0[42068]: 2024-09-12T12:34:10.769+0000 7f75be0ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:11.049 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0[42068]: 2024-09-12T12:34:10.769+0000 7f75be0ac640 -1 osd.0 123 *** Got signal Terminated *** 2024-09-12T12:34:11.049 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0[42068]: 2024-09-12T12:34:10.769+0000 7f75be0ac640 -1 osd.0 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:34:16.650 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:16 smithi137 podman[102910]: 2024-09-12 12:34:16.339823426 +0000 UTC m=+5.726679099 container died cd82463bcfe4c4d21093d8e85f2d18ed2dfabb83d6752016a12bb313832743a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T12:34:17.017 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:16 smithi137 podman[102910]: 2024-09-12 12:34:16.757729765 +0000 UTC m=+6.144585437 container cleanup cd82463bcfe4c4d21093d8e85f2d18ed2dfabb83d6752016a12bb313832743a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T12:34:17.017 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:16 smithi137 bash[102910]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0 2024-09-12T12:34:17.299 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:17 smithi137 podman[102925]: 2024-09-12 12:34:17.017774308 +0000 UTC m=+0.678815839 container remove cd82463bcfe4c4d21093d8e85f2d18ed2dfabb83d6752016a12bb313832743a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T12:34:17.796 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:17 smithi137 podman[103027]: 2024-09-12 12:34:17.35830765 +0000 UTC m=+0.015677323 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:34:17.797 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:17 smithi137 podman[103027]: 2024-09-12 12:34:17.493811724 +0000 UTC m=+0.151181391 container create f27cf1779a16530d91b4c60b47b0c9ab5fb232a1d4845f6960a8467893a944df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T12:34:18.049 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:17 smithi137 podman[103027]: 2024-09-12 12:34:17.797044042 +0000 UTC m=+0.454413714 container init f27cf1779a16530d91b4c60b47b0c9ab5fb232a1d4845f6960a8467893a944df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T12:34:18.050 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:17 smithi137 podman[103027]: 2024-09-12 12:34:17.800644095 +0000 UTC m=+0.458013767 container start f27cf1779a16530d91b4c60b47b0c9ab5fb232a1d4845f6960a8467893a944df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T12:34:18.050 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:17 smithi137 podman[103027]: 2024-09-12 12:34:17.922213598 +0000 UTC m=+0.579583275 container attach f27cf1779a16530d91b4c60b47b0c9ab5fb232a1d4845f6960a8467893a944df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T12:34:18.050 INFO:journalctl@ceph.osd.0.smithi137.stdout:Sep 12 12:34:17 smithi137 podman[103027]: 2024-09-12 12:34:17.992001853 +0000 UTC m=+0.649371534 container died f27cf1779a16530d91b4c60b47b0c9ab5fb232a1d4845f6960a8467893a944df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T12:34:18.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T12:34:18.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:18.732 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T12:34:18.733 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T12:34:18.733 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.1 2024-09-12T12:34:19.045 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:18 smithi171 systemd[1]: Stopping Ceph osd.1 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:19.449 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1[39551]: 2024-09-12T12:34:19.044+0000 7feac21ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:19.450 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1[39551]: 2024-09-12T12:34:19.044+0000 7feac21ad640 -1 osd.1 123 *** Got signal Terminated *** 2024-09-12T12:34:19.450 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1[39551]: 2024-09-12T12:34:19.044+0000 7feac21ad640 -1 osd.1 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:34:24.700 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:24 smithi171 podman[72731]: 2024-09-12 12:34:24.437145821 +0000 UTC m=+5.507907060 container died a822202aefa15a2e5335ca85072b1aa5dc3ccb316026df4388cbaab788d9e676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:34:25.121 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:24 smithi171 podman[72731]: 2024-09-12 12:34:24.819820309 +0000 UTC m=+5.890581557 container cleanup a822202aefa15a2e5335ca85072b1aa5dc3ccb316026df4388cbaab788d9e676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T12:34:25.121 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:24 smithi171 bash[72731]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1 2024-09-12T12:34:25.427 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:25 smithi171 podman[72748]: 2024-09-12 12:34:25.121146602 +0000 UTC m=+0.678368819 container remove a822202aefa15a2e5335ca85072b1aa5dc3ccb316026df4388cbaab788d9e676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T12:34:25.700 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:25 smithi171 podman[72837]: 2024-09-12 12:34:25.330593622 +0000 UTC m=+0.015765581 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:34:25.700 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:25 smithi171 podman[72837]: 2024-09-12 12:34:25.463570044 +0000 UTC m=+0.148742004 container create 1918f0ae986b7a489868160b0af6cd296054d0b9b2cb90222b0f3a0632cc2735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-12T12:34:26.097 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:25 smithi171 podman[72837]: 2024-09-12 12:34:25.724771669 +0000 UTC m=+0.409943628 container init 1918f0ae986b7a489868160b0af6cd296054d0b9b2cb90222b0f3a0632cc2735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T12:34:26.097 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:25 smithi171 podman[72837]: 2024-09-12 12:34:25.728231677 +0000 UTC m=+0.413403637 container start 1918f0ae986b7a489868160b0af6cd296054d0b9b2cb90222b0f3a0632cc2735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T12:34:26.098 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:25 smithi171 podman[72837]: 2024-09-12 12:34:25.84142935 +0000 UTC m=+0.526601355 container attach 1918f0ae986b7a489868160b0af6cd296054d0b9b2cb90222b0f3a0632cc2735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T12:34:26.379 INFO:journalctl@ceph.osd.1.smithi171.stdout:Sep 12 12:34:26 smithi171 podman[72837]: 2024-09-12 12:34:26.098053404 +0000 UTC m=+0.783225373 container died 1918f0ae986b7a489868160b0af6cd296054d0b9b2cb90222b0f3a0632cc2735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T12:34:26.801 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T12:34:26.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:26.842 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T12:34:26.842 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T12:34:26.842 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.2 2024-09-12T12:34:27.170 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:26 smithi189 systemd[1]: Stopping Ceph osd.2 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:27.590 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2[40435]: 2024-09-12T12:34:27.168+0000 7f73b1898640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:27.591 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2[40435]: 2024-09-12T12:34:27.168+0000 7f73b1898640 -1 osd.2 123 *** Got signal Terminated *** 2024-09-12T12:34:27.591 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2[40435]: 2024-09-12T12:34:27.168+0000 7f73b1898640 -1 osd.2 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:34:33.091 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:32 smithi189 podman[75211]: 2024-09-12 12:34:32.794414064 +0000 UTC m=+5.747849935 container died aeabff18f54844631bddd04a16c6269a5eb906d049d5578874d448f2f8e244c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T12:34:33.564 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:33 smithi189 podman[75211]: 2024-09-12 12:34:33.297019469 +0000 UTC m=+6.250455339 container cleanup aeabff18f54844631bddd04a16c6269a5eb906d049d5578874d448f2f8e244c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T12:34:33.565 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:33 smithi189 bash[75211]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2 2024-09-12T12:34:33.840 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:33 smithi189 podman[75226]: 2024-09-12 12:34:33.565049994 +0000 UTC m=+0.754005135 container remove aeabff18f54844631bddd04a16c6269a5eb906d049d5578874d448f2f8e244c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-12T12:34:34.176 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:33 smithi189 podman[75318]: 2024-09-12 12:34:33.80976629 +0000 UTC m=+0.016293918 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:34:34.177 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:33 smithi189 podman[75318]: 2024-09-12 12:34:33.932553556 +0000 UTC m=+0.139081177 container create 55c52bbf48167600a91c56ea274a4ad55d4de3cab06bd8d249bea0529cab41ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T12:34:34.544 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:34 smithi189 podman[75318]: 2024-09-12 12:34:34.177009545 +0000 UTC m=+0.383537166 container init 55c52bbf48167600a91c56ea274a4ad55d4de3cab06bd8d249bea0529cab41ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T12:34:34.544 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:34 smithi189 podman[75318]: 2024-09-12 12:34:34.180487245 +0000 UTC m=+0.387014876 container start 55c52bbf48167600a91c56ea274a4ad55d4de3cab06bd8d249bea0529cab41ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T12:34:34.544 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:34 smithi189 podman[75318]: 2024-09-12 12:34:34.285548607 +0000 UTC m=+0.492076230 container attach 55c52bbf48167600a91c56ea274a4ad55d4de3cab06bd8d249bea0529cab41ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:34:34.841 INFO:journalctl@ceph.osd.2.smithi189.stdout:Sep 12 12:34:34 smithi189 podman[75318]: 2024-09-12 12:34:34.544369117 +0000 UTC m=+0.750896741 container died 55c52bbf48167600a91c56ea274a4ad55d4de3cab06bd8d249bea0529cab41ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T12:34:35.275 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T12:34:35.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:35.319 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T12:34:35.319 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T12:34:35.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.3 2024-09-12T12:34:35.629 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:35 smithi137 systemd[1]: Stopping Ceph osd.3 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:36.052 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:35 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3[48530]: 2024-09-12T12:34:35.627+0000 7f93eb12c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:36.052 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:35 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3[48530]: 2024-09-12T12:34:35.627+0000 7f93eb12c640 -1 osd.3 123 *** Got signal Terminated *** 2024-09-12T12:34:36.052 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:35 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3[48530]: 2024-09-12T12:34:35.627+0000 7f93eb12c640 -1 osd.3 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:34:38.549 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:38 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:38.096+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:38.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:38.524+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:39.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:39 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:39.092+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:39.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:38 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:38.868+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:39.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:39 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:39.212+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:39.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:39.543+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:40.102 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:39 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:39.823+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:40.549 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:40 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:40.101+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:40.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:40 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:40.235+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:40.840 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:40.519+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:41.151 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:40 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:40.826+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:41.414 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:41 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:41.151+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:41.415 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:41 smithi137 podman[103131]: 2024-09-12 12:34:41.157612396 +0000 UTC m=+5.643008551 container died 13dc2f41d572c475ab0a6d8989c8140198a58a1c361cf405b78ce3fcee394952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:34:41.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:41 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:41.267+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:41.799 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:41 smithi137 podman[103131]: 2024-09-12 12:34:41.613276465 +0000 UTC m=+6.098672621 container cleanup 13dc2f41d572c475ab0a6d8989c8140198a58a1c361cf405b78ce3fcee394952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-12T12:34:41.799 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:41 smithi137 bash[103131]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3 2024-09-12T12:34:41.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:41.494+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:41.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:41.582+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:42.129 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:41 smithi137 podman[103149]: 2024-09-12 12:34:41.906679693 +0000 UTC m=+0.751713254 container remove 13dc2f41d572c475ab0a6d8989c8140198a58a1c361cf405b78ce3fcee394952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T12:34:42.130 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:41 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:41.870+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:42.382 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:42 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:42.129+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:42.383 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:42 smithi137 podman[103239]: 2024-09-12 12:34:42.131463412 +0000 UTC m=+0.016043103 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:34:42.383 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:42 smithi137 podman[103239]: 2024-09-12 12:34:42.257530455 +0000 UTC m=+0.142110145 container create ceb093c17ae99ef208544f74a7d8af7025d07cc292dc91c03726a1f4e1153a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:34:42.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:42 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:42.228+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:42.799 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:42 smithi137 podman[103239]: 2024-09-12 12:34:42.527882997 +0000 UTC m=+0.412462686 container init ceb093c17ae99ef208544f74a7d8af7025d07cc292dc91c03726a1f4e1153a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T12:34:42.800 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:42 smithi137 podman[103239]: 2024-09-12 12:34:42.531204958 +0000 UTC m=+0.415784654 container start ceb093c17ae99ef208544f74a7d8af7025d07cc292dc91c03726a1f4e1153a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T12:34:42.800 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:42 smithi137 podman[103239]: 2024-09-12 12:34:42.627250225 +0000 UTC m=+0.511829917 container attach ceb093c17ae99ef208544f74a7d8af7025d07cc292dc91c03726a1f4e1153a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:34:42.800 INFO:journalctl@ceph.osd.3.smithi137.stdout:Sep 12 12:34:42 smithi137 podman[103239]: 2024-09-12 12:34:42.719301233 +0000 UTC m=+0.603880923 container died ceb093c17ae99ef208544f74a7d8af7025d07cc292dc91c03726a1f4e1153a2d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-12T12:34:42.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:42.509+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:42.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:42.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:43.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:42 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:42.918+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:43.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:43 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:43.096+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:43.361 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T12:34:43.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:43.403 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T12:34:43.403 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T12:34:43.404 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.4 2024-09-12T12:34:43.700 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:43 smithi171 systemd[1]: Stopping Ceph osd.4 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:43.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:43 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:43.232+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:43.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:43.535+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:43.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:43.544+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:44.200 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:43 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:34:43.721+0000 7f36bda7e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:44.200 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:43 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:34:43.721+0000 7f36bda7e640 -1 osd.4 123 *** Got signal Terminated *** 2024-09-12T12:34:44.200 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:43 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:34:43.721+0000 7f36bda7e640 -1 osd.4 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:34:44.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:43 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:43.910+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:44.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:44 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:44.049+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:44.699 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:44 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:44.273+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:44.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:44.563+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:44.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:44.537+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:44.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:44.605+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:45.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:44 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:44.932+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:45.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:45 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:45.005+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:45.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:45 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:45.005+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:45.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:45 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:45.248+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:45.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:45 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:45.248+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:45.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:45 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:45.547+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:45.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:45 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:45.640+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:45.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:45 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:45.527+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:46.199 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:45 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:45.753+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:46.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:45 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:45.965+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:46.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:46 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:46.019+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:46.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:46 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:46.019+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:46.700 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:46 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:34:46.247+0000 7f36ba094640 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.067546+0000 front 2024-09-12T12:34:22.067528+0000 (oldest deadline 2024-09-12T12:34:46.167714+0000) 2024-09-12T12:34:46.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:46 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:46.214+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:46.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:46 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:46.214+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:46.793 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:46 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:46.671+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:46.793 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:46 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:46.490+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:46.794 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:46 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:46.502+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:47.199 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:46 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:46.724+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:47.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:46 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:46.978+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:47.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:47.060+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:47.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:47.060+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:47.493 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:47 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:34:47.223+0000 7f36ba094640 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.067546+0000 front 2024-09-12T12:34:22.067528+0000 (oldest deadline 2024-09-12T12:34:46.167714+0000) 2024-09-12T12:34:47.493 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:47 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:47.205+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:47.494 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:47 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:47.205+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:47.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:47 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:47.640+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:47.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:47 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:47.457+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:47.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:47 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:47.508+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:47.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:47 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:47.508+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:47.950 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:47 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:47.717+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:48.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:48 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:48.021+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:48.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:48 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:48.096+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:48.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:48 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:48.096+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:48.450 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:48 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4[46314]: 2024-09-12T12:34:48.240+0000 7f36ba094640 -1 osd.4 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.067546+0000 front 2024-09-12T12:34:22.067528+0000 (oldest deadline 2024-09-12T12:34:46.167714+0000) 2024-09-12T12:34:48.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:48 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:48.162+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:48.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:48 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:48.162+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:48.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:48 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:48.410+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:48.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:48 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:48.550+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:48.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:48 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:48.550+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:48.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:48 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:48.618+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:49.107 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:48 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:48.766+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:49.418 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:49 smithi171 podman[72956]: 2024-09-12 12:34:49.107625679 +0000 UTC m=+5.505126213 container died 406d3841e6f108469d20794117664e4c2da0bd699c2c723398dbbef81b39b483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T12:34:49.419 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:49 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:49.118+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:49.419 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:49 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:49.118+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:49.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:49 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:49.070+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:49.549 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:49 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:49.066+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:49.549 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:49 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:49.066+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:49.700 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:49 smithi171 podman[72956]: 2024-09-12 12:34:49.522630914 +0000 UTC m=+5.920131484 container cleanup 406d3841e6f108469d20794117664e4c2da0bd699c2c723398dbbef81b39b483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T12:34:49.700 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:49 smithi171 bash[72956]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4 2024-09-12T12:34:49.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:49 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:49.579+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:49.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:49 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:49.552+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:49.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:49 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:49.552+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:49.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:49 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:49.420+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:49.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:49 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:49.420+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:50.100 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:49 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:49.770+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:50.100 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:49 smithi171 podman[72973]: 2024-09-12 12:34:49.807282592 +0000 UTC m=+0.663608418 container remove 406d3841e6f108469d20794117664e4c2da0bd699c2c723398dbbef81b39b483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T12:34:50.410 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:50 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:50.099+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:50.410 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:50 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:50.099+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:50.410 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:50 smithi171 podman[73062]: 2024-09-12 12:34:50.034874536 +0000 UTC m=+0.016051699 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:34:50.410 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:50 smithi171 podman[73062]: 2024-09-12 12:34:50.158085389 +0000 UTC m=+0.139262550 container create 002eed59a999084ade58d70dcbbdd8c414abf4866ea0038464c8529f32f57d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:34:50.410 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:50 smithi171 podman[73062]: 2024-09-12 12:34:50.410490176 +0000 UTC m=+0.391667337 container init 002eed59a999084ade58d70dcbbdd8c414abf4866ea0038464c8529f32f57d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T12:34:50.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:50.116+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:50.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:50.116+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:50.549 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:50.106+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:50.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:50 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:50.106+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:50.700 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:50 smithi171 podman[73062]: 2024-09-12 12:34:50.413729533 +0000 UTC m=+0.394906695 container start 002eed59a999084ade58d70dcbbdd8c414abf4866ea0038464c8529f32f57d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T12:34:50.700 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:50 smithi171 podman[73062]: 2024-09-12 12:34:50.510699593 +0000 UTC m=+0.491876767 container attach 002eed59a999084ade58d70dcbbdd8c414abf4866ea0038464c8529f32f57d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:34:50.701 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:50 smithi171 podman[73062]: 2024-09-12 12:34:50.609443927 +0000 UTC m=+0.590621089 container died 002eed59a999084ade58d70dcbbdd8c414abf4866ea0038464c8529f32f57d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T12:34:50.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:50 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:50.617+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:50.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:50 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:50.561+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:50.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:50 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:50.561+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:50.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:50 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:50.373+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:50.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:50 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:50.373+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:51.064 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:50 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:50.748+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:51.315 INFO:journalctl@ceph.osd.4.smithi171.stdout:Sep 12 12:34:51 smithi171 podman[73085]: 2024-09-12 12:34:51.297954915 +0000 UTC m=+0.679627779 container remove 002eed59a999084ade58d70dcbbdd8c414abf4866ea0038464c8529f32f57d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T12:34:51.316 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:51 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:51.064+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:51.316 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:51 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:51.064+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:51.319 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T12:34:51.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:51.359 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T12:34:51.360 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T12:34:51.360 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.5 2024-09-12T12:34:51.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:51.150+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:51.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:51.150+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:51.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:51.115+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:51.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:51 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:51.115+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:51.726 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:51.599+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:51.726 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:51.371+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:51.727 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:51.371+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:51.727 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:51 smithi189 systemd[1]: Stopping Ceph osd.5 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:51.727 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:51.581+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:51.727 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:51.581+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:51.727 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:51.725+0000 7f2c7306c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:34:51.727 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:51.725+0000 7f2c7306c640 -1 osd.5 123 *** Got signal Terminated *** 2024-09-12T12:34:51.727 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:51 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:51.725+0000 7f2c7306c640 -1 osd.5 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:34:52.103 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:51 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:51.770+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:52.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:52 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:52.103+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:52.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:52 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:52.103+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:52.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:52 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:52.147+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:52.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:52 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:52.147+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:52.549 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:52 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:52.139+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:52.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:52 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:52.139+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:52.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:52 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:52.635+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:52.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:52 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:52.587+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:52.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:52 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:52.587+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:52.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:52 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:52.419+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:52.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:52 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:52.419+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:53.118 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:52 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:52.736+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:53.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:53 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:53.118+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:53.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:53 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:53.118+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:53.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:53 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:53.172+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:53.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:53 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:53.172+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:53.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:53 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:53.143+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:53.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:53 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:53.143+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:53.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:53.586+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:53.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:53.586+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:34:53.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:53.391+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:53.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:53.391+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:53.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:53.391+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:34:53.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:53.556+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:53.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:53.556+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:53.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:53 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:53.556+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.115372+0000 front 2024-09-12T12:34:30.115397+0000 (oldest deadline 2024-09-12T12:34:53.015526+0000) 2024-09-12T12:34:53.949 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:53 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:53.686+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:54.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:54 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:54.138+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:54.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:54 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:54.138+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:54.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:54.219+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:54.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:54.219+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:54.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:54.111+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:54.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:54.111+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:54.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:54 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:54.111+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:34:54.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:54.595+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:54.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:54.595+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:34:54.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:54.415+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:54.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:54.415+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:54.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:54.415+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:34:54.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:54.544+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:54.843 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:54.544+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:54.843 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:54 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:54.544+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.115372+0000 front 2024-09-12T12:34:30.115397+0000 (oldest deadline 2024-09-12T12:34:53.015526+0000) 2024-09-12T12:34:55.160 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:54 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:54.726+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:55.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:55.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:55.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:55.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:55.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:55.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:34:55.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:55 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:55.253+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:55.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:55 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:55.253+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:55.549 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:55 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:55.065+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:55.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:55 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:55.065+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:55.550 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:55 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:55.065+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:34:55.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:55.601+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:55.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:55.601+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:34:55.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:55.432+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:55.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:55.432+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:55.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:55.432+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:34:55.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:55.522+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:55.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:55.522+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:55.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:55 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:55.522+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.115372+0000 front 2024-09-12T12:34:30.115397+0000 (oldest deadline 2024-09-12T12:34:53.015526+0000) 2024-09-12T12:34:56.148 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:55 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:55.746+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:56.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:56.265+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:56.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:56.265+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:56.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:56.265+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:34:56.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:56.033+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:56.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:56.033+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:56.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:56.033+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:34:56.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:56 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:56.147+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:56.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:56 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:56.147+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:56.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:56 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:56.147+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:34:56.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:56.607+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:56.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:56.607+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:34:56.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:56.510+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:12.112329+0000 front 2024-09-12T12:34:12.112175+0000 (oldest deadline 2024-09-12T12:34:38.012281+0000) 2024-09-12T12:34:56.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:56.510+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:21.413946+0000 front 2024-09-12T12:34:21.413868+0000 (oldest deadline 2024-09-12T12:34:46.713798+0000) 2024-09-12T12:34:56.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5[47351]: 2024-09-12T12:34:56.510+0000 7f2c6f682640 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.115372+0000 front 2024-09-12T12:34:30.115397+0000 (oldest deadline 2024-09-12T12:34:53.015526+0000) 2024-09-12T12:34:56.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:56.392+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:56.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:56.392+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:56.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:56 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:56.392+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:34:57.167 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:56 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:56.728+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:57.167 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:56 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:56.728+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:34:57.223 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:57.223+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:57.223 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:57.223+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:57.224 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:57.223+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:34:57.224 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:56.998+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:57.224 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:56.998+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:57.224 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:56 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:56.998+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:34:57.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:57 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:57.167+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:57.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:57 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:57.167+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:57.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:57 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:57.167+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:34:57.477 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:57 smithi189 podman[75433]: 2024-09-12 12:34:57.1713504 +0000 UTC m=+5.613272574 container died 2dffe6a129d2f8dedd22ae0b41b81e4c90eca15a6b0ab6294b5911e836f65b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T12:34:57.478 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:57 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:57.372+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:57.478 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:57 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:57.372+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:57.478 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:57 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:57.372+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:34:57.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:57 smithi189 podman[75433]: 2024-09-12 12:34:57.583896076 +0000 UTC m=+6.025818253 container cleanup 2dffe6a129d2f8dedd22ae0b41b81e4c90eca15a6b0ab6294b5911e836f65b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T12:34:57.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:57 smithi189 bash[75433]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5 2024-09-12T12:34:57.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:57 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:57.612+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:57.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:57 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:57.612+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:34:58.161 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:57 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:57.757+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:58.161 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:57 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:57.757+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:34:58.186 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:57 smithi189 podman[75448]: 2024-09-12 12:34:57.843885243 +0000 UTC m=+0.672631800 container remove 2dffe6a129d2f8dedd22ae0b41b81e4c90eca15a6b0ab6294b5911e836f65b5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-12T12:34:58.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:58 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:58.184+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:58.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:58 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:58.184+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:58.300 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:58 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:58.184+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:34:58.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:57.989+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:58.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:57.989+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:58.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:57.989+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:34:58.447 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:58 smithi189 podman[75537]: 2024-09-12 12:34:58.090097551 +0000 UTC m=+0.015878786 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:34:58.447 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:58 smithi189 podman[75537]: 2024-09-12 12:34:58.202987436 +0000 UTC m=+0.128768712 container create 58a07e1a82f51b76c043b7d76c08fc60f9b2ed155c0c12cf911d986d91fbd9a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:34:58.447 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:58 smithi189 podman[75537]: 2024-09-12 12:34:58.447345656 +0000 UTC m=+0.373126881 container init 58a07e1a82f51b76c043b7d76c08fc60f9b2ed155c0c12cf911d986d91fbd9a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T12:34:58.448 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:58 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:58.370+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:58.448 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:58 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:58.370+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:58.448 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:58 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:58.370+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:34:58.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:58 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:58.161+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:58.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:58 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:58.161+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:58.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:58 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:58.161+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:34:58.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:58 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:58.585+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:58.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:58 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:58.585+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:34:58.841 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:58 smithi189 podman[75537]: 2024-09-12 12:34:58.450800462 +0000 UTC m=+0.376581687 container start 58a07e1a82f51b76c043b7d76c08fc60f9b2ed155c0c12cf911d986d91fbd9a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-12T12:34:58.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:58 smithi189 podman[75537]: 2024-09-12 12:34:58.563908466 +0000 UTC m=+0.489689710 container attach 58a07e1a82f51b76c043b7d76c08fc60f9b2ed155c0c12cf911d986d91fbd9a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:34:58.842 INFO:journalctl@ceph.osd.5.smithi189.stdout:Sep 12 12:34:58 smithi189 podman[75537]: 2024-09-12 12:34:58.642423356 +0000 UTC m=+0.568204590 container died 58a07e1a82f51b76c043b7d76c08fc60f9b2ed155c0c12cf911d986d91fbd9a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T12:34:59.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:58 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:58.794+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:34:59.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:58 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:58.794+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:34:59.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:59.178+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:34:59.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:59.178+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:34:59.300 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:59.178+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:34:59.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:58 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:58.955+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:34:59.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:58 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:58.955+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:34:59.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:58 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:58.955+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:34:59.325 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T12:34:59.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:34:59.364 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T12:34:59.364 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T12:34:59.364 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.6 2024-09-12T12:34:59.591 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:59 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:59.539+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:34:59.591 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:34:59 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:34:59.539+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:34:59.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:59 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:59.409+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:34:59.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:59 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:59.409+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:34:59.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:34:59 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:34:59.409+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:34:59.675 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:59 smithi137 systemd[1]: Stopping Ceph osd.6 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:34:59.699 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:59 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:59.210+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:34:59.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:59 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:59.210+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:34:59.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:34:59 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:34:59.210+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:00.049 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:59.674+0000 7f6e0a0eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:00.049 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:59.674+0000 7f6e0a0eb640 -1 osd.6 123 *** Got signal Terminated *** 2024-09-12T12:35:00.049 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:34:59.674+0000 7f6e0a0eb640 -1 osd.6 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:35:00.050 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:59.911+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:00.050 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:59.911+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:00.050 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:34:59 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:34:59.911+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:00.161 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:59 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:59.786+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:00.162 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:34:59 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:34:59.786+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:00.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:00 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:00.161+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:00.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:00 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:00.161+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:00.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:00 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:00.161+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:00.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:00.182+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:35:00.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:00.182+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:35:00.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:00.182+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:35:00.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:00 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:00.517+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:00.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:00 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:00.517+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:00.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:00 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:00.418+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:00.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:00 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:00.418+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:00.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:00 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:00.418+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:01.166 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:00.892+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:01.166 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:00.892+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:01.166 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:00.892+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:01.169 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:00 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:00.756+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:01.170 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:00 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:00.756+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:01.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:01.169+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:01.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:01.169+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:01.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:01.169+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:01.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:01 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:01.165+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:35:01.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:01 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:01.165+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:35:01.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:01 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:01.165+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:35:01.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:01 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:01.473+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:01.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:01 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:01.473+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:01.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:01 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:01.455+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:01.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:01 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:01.455+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:01.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:01 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:01.455+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:02.138 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:01.761+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:02.138 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:01 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:01.761+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:02.168 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:01 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:01.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:02.168 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:01 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:01.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:02.168 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:01 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:01.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:02.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:02 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:02.138+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:02.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:02 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:02.138+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:02.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:02 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:02.138+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:02.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:02 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:02.138+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:02.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:02.167+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:35:02.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:02.167+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:35:02.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:02.167+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:35:02.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:02 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:02.509+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:02.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:02 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:02.509+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:02.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:02 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:02.428+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:02.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:02 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:02.428+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:02.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:02 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:02.428+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:02.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:02 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:02.428+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:03.106 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:02 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:02.739+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:03.106 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:02 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:02.739+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:03.106 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:02 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:02.739+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:03.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:02.949+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:03.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:02.949+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:03.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:02 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:02.949+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:03.300 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:03.187+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:35:03.300 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:03.187+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:35:03.300 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:03.187+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:35:03.300 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:03.187+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.872451+0000 front 2024-09-12T12:34:39.872478+0000 (oldest deadline 2024-09-12T12:35:02.172630+0000) 2024-09-12T12:35:03.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:03 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:03.105+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:03.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:03 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:03.105+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:03.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:03 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:03.105+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:03.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:03 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:03.105+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:03.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:03 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:03.425+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:03.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:03 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:03.425+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:03.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:03 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:03.425+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:03.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:03 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:03.425+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:03.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:03 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:03.525+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:03.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:03 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:03.525+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:03.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:03 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:03.525+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:04.125 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:03 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:03.780+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:04.126 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:03 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:03.780+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:04.126 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:03 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:03.780+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:04.301 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:04.153+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.667789+0000 front 2024-09-12T12:34:13.667822+0000 (oldest deadline 2024-09-12T12:34:38.367989+0000) 2024-09-12T12:35:04.301 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:04.153+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.369889+0000 front 2024-09-12T12:34:22.369831+0000 (oldest deadline 2024-09-12T12:34:49.970468+0000) 2024-09-12T12:35:04.301 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:04.153+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.970896+0000 front 2024-09-12T12:34:29.970914+0000 (oldest deadline 2024-09-12T12:34:55.871022+0000) 2024-09-12T12:35:04.301 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6[55390]: 2024-09-12T12:35:04.153+0000 7f6e06701640 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.872451+0000 front 2024-09-12T12:34:39.872478+0000 (oldest deadline 2024-09-12T12:35:02.172630+0000) 2024-09-12T12:35:04.301 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:03.950+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:04.302 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:03.950+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:04.302 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:03.950+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:04.302 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:03 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:03.950+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:04.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:04.125+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:04.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:04.125+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:04.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:04.125+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:04.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:04.125+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:04.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:04.511+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:04.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:04.511+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:04.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:04.511+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:04.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:04.442+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:04.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:04.442+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:04.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:04.442+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:04.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:04 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:04.442+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:05.085 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:04.810+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:05.086 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:04.810+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:05.086 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:04 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:04.810+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:05.251 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:04.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:05.251 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:04.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:05.251 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:04.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:05.251 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:04 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:04.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:05.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:05.085+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:05.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:05.085+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:05.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:05.085+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:05.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:05.085+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:05.549 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:05 smithi137 podman[103353]: 2024-09-12 12:35:05.251542274 +0000 UTC m=+5.696596092 container died ad849c208ae75b874744187cb69515a379487c4f4cc8bf779b92763884b9b483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T12:35:05.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:05.478+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:05.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:05.478+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:05.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:05.478+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:05.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:05.486+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:05.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:05.486+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:05.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:05.486+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:05.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:05 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:05.486+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:05.905 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:05 smithi137 podman[103353]: 2024-09-12 12:35:05.742020853 +0000 UTC m=+6.187074670 container cleanup ad849c208ae75b874744187cb69515a379487c4f4cc8bf779b92763884b9b483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T12:35:05.905 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:05 smithi137 bash[103353]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6 2024-09-12T12:35:06.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:05.856+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:06.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:05.856+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:06.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:05.856+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:06.200 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:06 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:06.036+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:06.201 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:06 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:06.036+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:06.201 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:06 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:06.036+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:06.201 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:06 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:06.036+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:06.299 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:05 smithi137 podman[103369]: 2024-09-12 12:35:05.963346882 +0000 UTC m=+0.704538426 container remove ad849c208ae75b874744187cb69515a379487c4f4cc8bf779b92763884b9b483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T12:35:06.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:05 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:05.904+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:06.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:05 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:05.904+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:06.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:05 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:05.904+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:06.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:05 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:05.904+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:06.746 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:06 smithi137 podman[103459]: 2024-09-12 12:35:06.402674387 +0000 UTC m=+0.110103352 container create 2425610bf41e4e65a5b75be2c1b394c9962874cf3fbe7e623bd862e6235873e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T12:35:06.746 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:06 smithi137 podman[103459]: 2024-09-12 12:35:06.308614916 +0000 UTC m=+0.016043841 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:35:06.747 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:06 smithi137 podman[103459]: 2024-09-12 12:35:06.646858197 +0000 UTC m=+0.354287116 container init 2425610bf41e4e65a5b75be2c1b394c9962874cf3fbe7e623bd862e6235873e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T12:35:06.747 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:06 smithi137 podman[103459]: 2024-09-12 12:35:06.650504593 +0000 UTC m=+0.357933515 container start 2425610bf41e4e65a5b75be2c1b394c9962874cf3fbe7e623bd862e6235873e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T12:35:06.747 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:06 smithi137 podman[103459]: 2024-09-12 12:35:06.74674326 +0000 UTC m=+0.454172212 container attach 2425610bf41e4e65a5b75be2c1b394c9962874cf3fbe7e623bd862e6235873e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T12:35:06.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:06.515+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:06.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:06.515+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:06.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:06.515+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:06.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:06.514+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:06.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:06.514+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:06.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:06.514+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:06.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:06 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:06.514+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:07.049 INFO:journalctl@ceph.osd.6.smithi137.stdout:Sep 12 12:35:06 smithi137 podman[103459]: 2024-09-12 12:35:06.840880412 +0000 UTC m=+0.548309348 container died 2425610bf41e4e65a5b75be2c1b394c9962874cf3fbe7e623bd862e6235873e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T12:35:07.049 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:06 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:06.895+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:07.050 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:06 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:06.895+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:07.050 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:06 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:06.895+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:07.050 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:06 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:06.895+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:07.102 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:07.083+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:07.102 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:07.083+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:07.102 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:07.083+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:07.102 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:07.083+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:07.102 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:06 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:06.848+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:07.103 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:06 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:06.848+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:07.103 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:06 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:06.848+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:07.480 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T12:35:07.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:07.550 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T12:35:07.551 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T12:35:07.551 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.7 2024-09-12T12:35:07.761 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:07 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:07.527+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:07.762 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:07 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:07.527+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:07.762 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:07 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:07.527+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:07.762 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:07 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:07.493+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:07.762 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:07 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:07.493+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:07.762 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:07 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:07.493+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:07.762 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:07 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:07.493+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:07.827 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:07 smithi171 systemd[1]: Stopping Ceph osd.7 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:08.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:07.827+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:08.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:07.827+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:08.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:07.827+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:08.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:07.874+0000 7fe564756640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:08.200 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:07.874+0000 7fe564756640 -1 osd.7 123 *** Got signal Terminated *** 2024-09-12T12:35:08.201 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:07 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:07.874+0000 7fe564756640 -1 osd.7 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:35:08.201 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:08.048+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:08.201 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:08.048+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:08.201 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:08.048+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:08.201 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:08.048+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:08.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:07 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:07.937+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:08.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:07 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:07.937+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:08.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:07 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:07.937+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:08.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:07 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:07.937+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:08.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:08.530+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:08.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:08.530+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:08.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:08.530+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:08.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:08.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:08.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:08.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:08.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:08.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:08.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:08 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:08.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:09.096 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:08.818+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:09.096 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:08.818+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:09.096 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:08.818+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:09.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:08.921+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:09.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:08.921+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:09.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:08.921+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:09.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:08 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:08.921+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:09.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:09.095+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:09.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:09.095+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:09.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:09.095+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:09.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:09.095+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:09.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:09.581+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:09.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:09.581+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:09.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:09.581+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:09.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:09.443+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:09.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:09.443+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:09.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:09.443+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:09.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:09 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:09.443+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:10.103 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:09.774+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:10.103 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:09.774+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:10.103 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:09 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:09.774+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:10.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:09.939+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:10.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:09.939+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:10.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:09.939+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:10.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:09 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:09.939+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:10.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:10.103+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:10.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:10.103+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:10.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:10.103+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:10.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:10.103+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:10.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:10.103+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:10.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:10.421+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:10.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:10.421+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:10.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:10.421+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:10.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:10.421+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:10.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:10.577+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:10.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:10.577+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:10.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:10.577+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:10.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:10 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:10.578+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:11.133 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:10.781+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:11.133 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:10.781+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:11.134 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:10 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:10.781+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:11.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:10.948+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:11.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:10.948+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:11.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:10.948+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:11.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:10 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:10.948+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:11.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:11.133+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:11.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:11.133+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:11.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:11.133+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:11.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:11.133+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:11.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:11.133+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:11.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:11.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:11.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:11.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:11.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:11.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:11.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:11.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:11.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:11.610+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:11.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:11.610+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:11.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:11.610+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:11.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:11 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:11.610+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:12.114 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:11.731+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:12.115 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:11.731+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:12.115 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:11.731+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:12.115 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:11.731+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.199409+0000 front 2024-09-12T12:34:48.199425+0000 (oldest deadline 2024-09-12T12:35:11.699671+0000) 2024-09-12T12:35:12.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:11.909+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:12.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:11.909+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:12.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:11.909+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:12.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:11 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:11.909+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:12.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:12.114+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:12.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:12.114+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:12.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:12.114+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:12.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:12.114+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:12.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:12.114+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:12.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:12.494+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:12.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:12.494+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:12.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:12.494+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:12.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:12.494+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:12.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:12.494+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:12.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:12.657+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:12.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:12.657+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:12.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:12.657+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:12.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:12 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:12.657+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:13.130 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:12.743+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:20.794667+0000 front 2024-09-12T12:34:20.794643+0000 (oldest deadline 2024-09-12T12:34:45.494816+0000) 2024-09-12T12:35:13.130 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:12.743+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.896766+0000 front 2024-09-12T12:34:31.896665+0000 (oldest deadline 2024-09-12T12:34:55.996858+0000) 2024-09-12T12:35:13.130 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:12.743+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:35.997570+0000 front 2024-09-12T12:34:35.997460+0000 (oldest deadline 2024-09-12T12:35:01.897475+0000) 2024-09-12T12:35:13.131 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:12 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7[52802]: 2024-09-12T12:35:12.743+0000 7fe560d6c640 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.199409+0000 front 2024-09-12T12:34:48.199425+0000 (oldest deadline 2024-09-12T12:35:11.699671+0000) 2024-09-12T12:35:13.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:12.936+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:13.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:12.936+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:13.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:12.936+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:13.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:12 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:12.936+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:13.450 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:13 smithi171 podman[73165]: 2024-09-12 12:35:13.331572436 +0000 UTC m=+5.578566777 container died b9c594d0ab7387d81520f687c03539b6f063687ace290904714e0f80ed5727c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:13.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:13.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:13.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:13.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:13.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:13.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:13.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:13.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:13.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:13 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:13.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:13.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:13.670+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:13.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:13.670+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:13.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:13.670+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:13.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:13.670+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:13.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:13.539+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:13.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:13.539+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:13.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:13.539+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:13.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:13.539+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:13.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:13 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:13.539+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:13.950 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:13 smithi171 podman[73165]: 2024-09-12 12:35:13.699849719 +0000 UTC m=+5.946844066 container cleanup b9c594d0ab7387d81520f687c03539b6f063687ace290904714e0f80ed5727c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T12:35:13.950 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:13 smithi171 bash[73165]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7 2024-09-12T12:35:14.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:13.952+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:14.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:13.952+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:14.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:13.952+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:14.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:13 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:13.952+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:14.358 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:14 smithi171 podman[73192]: 2024-09-12 12:35:14.001215874 +0000 UTC m=+0.662165736 container remove b9c594d0ab7387d81520f687c03539b6f063687ace290904714e0f80ed5727c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T12:35:14.358 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:14.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:14.358 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:14.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:14.358 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:14.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:14.358 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:14.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:14.359 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:14 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:14.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:14.608 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:14 smithi171 podman[73280]: 2024-09-12 12:35:14.260779787 +0000 UTC m=+0.015930237 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:35:14.608 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:14 smithi171 podman[73280]: 2024-09-12 12:35:14.360404381 +0000 UTC m=+0.115554811 container create 605bcdc37a6dc824c8fad00286206ce0e12849dea9b8b406e920f7c651402501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:35:14.608 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:14 smithi171 podman[73280]: 2024-09-12 12:35:14.604743788 +0000 UTC m=+0.359894229 container init 605bcdc37a6dc824c8fad00286206ce0e12849dea9b8b406e920f7c651402501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:14.609 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:14 smithi171 podman[73280]: 2024-09-12 12:35:14.608717749 +0000 UTC m=+0.363868183 container start 605bcdc37a6dc824c8fad00286206ce0e12849dea9b8b406e920f7c651402501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T12:35:14.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:14.718+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:14.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:14.718+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:14.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:14.718+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:14.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:14.718+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:14.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:14.584+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:14.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:14.584+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:14.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:14.584+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:14.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:14.584+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:14.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:14 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:14.584+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:14.950 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:14 smithi171 podman[73280]: 2024-09-12 12:35:14.721285516 +0000 UTC m=+0.476435957 container attach 605bcdc37a6dc824c8fad00286206ce0e12849dea9b8b406e920f7c651402501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T12:35:14.950 INFO:journalctl@ceph.osd.7.smithi171.stdout:Sep 12 12:35:14 smithi171 podman[73280]: 2024-09-12 12:35:14.791147471 +0000 UTC m=+0.546297903 container died 605bcdc37a6dc824c8fad00286206ce0e12849dea9b8b406e920f7c651402501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-12T12:35:15.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:14.906+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:15.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:14.906+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:15.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:14.906+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:15.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:14 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:14.906+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:15.449 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:15 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:15.154+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:15.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:15 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:15.154+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:15.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:15 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:15.154+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:15.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:15 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:15.154+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:15.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:15 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:15.154+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:15.480 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T12:35:15.525 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:15.525 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T12:35:15.526 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T12:35:15.526 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.8 2024-09-12T12:35:15.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:15.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:15.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:15.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:15.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:15.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:15.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:15.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:15.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:15.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:15.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 systemd[1]: Stopping Ceph osd.8 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:15.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:15.731+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:15.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:15.731+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:15.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:15.731+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:15.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:15.731+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:16.200 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:15.924+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:16.200 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:15.924+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:16.201 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:15.924+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:16.201 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:15 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:15.924+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:16.341 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:15.852+0000 7efe91e74640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:16.341 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:15.852+0000 7efe91e74640 -1 osd.8 123 *** Got signal Terminated *** 2024-09-12T12:35:16.341 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:15 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:15.852+0000 7efe91e74640 -1 osd.8 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:35:16.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:16.203+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:16.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:16.203+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:16.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:16.203+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:16.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:16.203+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:16.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:16.203+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:16.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:16.733+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:16.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:16.733+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:16.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:16.733+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:16.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:16.733+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:16.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:16.576+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:16.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:16.576+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:16.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:16.576+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:16.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:16.576+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:16.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:16 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:16.576+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:17.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:16.918+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:17.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:16.918+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:17.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:16.918+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:17.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:16 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:16.918+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:17.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:17.223+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:17.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:17.223+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:17.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:17.223+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:17.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:17.223+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:17.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:17 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:17.223+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:17.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:17.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:17.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:17.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:17.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:17.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:17.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:17.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:17.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:17.545+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:17.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:17.545+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:17.843 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:17.545+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:17.843 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:17.545+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:17.843 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:17 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:17.545+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:18.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:17 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:17.934+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:18.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:17 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:17.934+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:18.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:17 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:17.934+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:18.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:17 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:17.934+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:18.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:18.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:18.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:18.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:18.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:18.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:18.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:18.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:18.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:18 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:18.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:18.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:18.703+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:18.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:18.703+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:18.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:18.703+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:18.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:18.703+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:18.841 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:18.703+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:53.053767+0000 front 2024-09-12T12:34:53.053784+0000 (oldest deadline 2024-09-12T12:35:18.254416+0000) 2024-09-12T12:35:18.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:18.531+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:18.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:18.531+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:18.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:18.531+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:18.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:18.531+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:18.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:18 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:18.531+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:19.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:18 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:18.888+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:19.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:18 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:18.888+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:19.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:18 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:18.888+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:19.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:18 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:18.888+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:19.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:19.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:19.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:19.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:19.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:19.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:19.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:19.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:19.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:19 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:19.164+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:19.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:19.483+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:19.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:19.483+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:19.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:19.483+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:19.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:19.483+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:19.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:19.483+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:19.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:19.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:19.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:19.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:19.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:19.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:19.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:19.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:19.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:19 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:19.687+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:53.053767+0000 front 2024-09-12T12:34:53.053784+0000 (oldest deadline 2024-09-12T12:35:18.254416+0000) 2024-09-12T12:35:20.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:19 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:19.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:20.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:19 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:19.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:20.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:19 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:19.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:20.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:19 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:19.900+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:20.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:20 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:20.207+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:20.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:20 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:20.207+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:20.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:20 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:20.207+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:20.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:20 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:20.207+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:20.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:20 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:20.207+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:20.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:20.436+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:20.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:20.436+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:20.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:20.436+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:20.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:20.436+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:20.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:20.436+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:20.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:20.662+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:19.747870+0000 front 2024-09-12T12:34:19.747960+0000 (oldest deadline 2024-09-12T12:34:44.447970+0000) 2024-09-12T12:35:20.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:20.662+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:29.749222+0000 front 2024-09-12T12:34:29.749200+0000 (oldest deadline 2024-09-12T12:34:53.249360+0000) 2024-09-12T12:35:20.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:20.662+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.150504+0000 front 2024-09-12T12:34:39.150485+0000 (oldest deadline 2024-09-12T12:35:02.650491+0000) 2024-09-12T12:35:20.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:20.662+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.652488+0000 front 2024-09-12T12:34:46.652620+0000 (oldest deadline 2024-09-12T12:35:10.152613+0000) 2024-09-12T12:35:20.842 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:20 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8[54008]: 2024-09-12T12:35:20.662+0000 7efe8e48a640 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:53.053767+0000 front 2024-09-12T12:34:53.053784+0000 (oldest deadline 2024-09-12T12:35:18.254416+0000) 2024-09-12T12:35:21.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:20 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:20.864+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:21.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:20 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:20.864+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:21.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:20 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:20.864+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:21.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:20 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:20.864+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:21.301 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:20 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:20.864+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:21.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:21 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:21.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:21.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:21 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:21.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:21.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:21 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:21.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:21.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:21 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:21.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:21.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:21 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:21.160+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:21.746 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:21 smithi189 podman[75647]: 2024-09-12 12:35:21.409377098 +0000 UTC m=+5.685436757 container died e95f6021b113b4827e05feaad5adc3cb87ae1fe5be34fb6a18aebe0274cd4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-12T12:35:21.746 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:21 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:21.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:21.746 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:21 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:21.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:21.747 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:21 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:21.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:21.747 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:21 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:21.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:21.747 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:21 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:21.445+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:22.079 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:21 smithi189 podman[75647]: 2024-09-12 12:35:21.828666643 +0000 UTC m=+6.104726294 container cleanup e95f6021b113b4827e05feaad5adc3cb87ae1fe5be34fb6a18aebe0274cd4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T12:35:22.079 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:21 smithi189 bash[75647]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8 2024-09-12T12:35:22.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:21 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:21.835+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:22.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:21 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:21.835+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:22.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:21 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:21.835+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:22.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:21 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:21.835+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:22.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:21 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:21.835+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:22.340 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:22 smithi189 podman[75664]: 2024-09-12 12:35:22.079996606 +0000 UTC m=+0.664155877 container remove e95f6021b113b4827e05feaad5adc3cb87ae1fe5be34fb6a18aebe0274cd4efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T12:35:22.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:22 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:22.134+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:22.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:22 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:22.134+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:22.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:22 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:22.134+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:22.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:22 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:22.134+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:22.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:22 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:22.134+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:22.800 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:22 smithi189 podman[75753]: 2024-09-12 12:35:22.39796176 +0000 UTC m=+0.015832438 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:35:22.801 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:22 smithi189 podman[75753]: 2024-09-12 12:35:22.522330169 +0000 UTC m=+0.140200836 container create 8c213e3935bb98cf4a647a009c0c2a8a067b1b738d5b76d8dc14f8ba40d2feb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T12:35:22.801 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:22 smithi189 podman[75753]: 2024-09-12 12:35:22.800519622 +0000 UTC m=+0.418390304 container init 8c213e3935bb98cf4a647a009c0c2a8a067b1b738d5b76d8dc14f8ba40d2feb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T12:35:22.801 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:22.462+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:22.801 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:22.462+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:22.802 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:22.462+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:22.802 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:22.462+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:22.802 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:22.462+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:22.802 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:22.462+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:23.091 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:22 smithi189 podman[75753]: 2024-09-12 12:35:22.80423144 +0000 UTC m=+0.422102108 container start 8c213e3935bb98cf4a647a009c0c2a8a067b1b738d5b76d8dc14f8ba40d2feb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:23.091 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:22 smithi189 podman[75753]: 2024-09-12 12:35:22.900099396 +0000 UTC m=+0.517970073 container attach 8c213e3935bb98cf4a647a009c0c2a8a067b1b738d5b76d8dc14f8ba40d2feb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T12:35:23.091 INFO:journalctl@ceph.osd.8.smithi189.stdout:Sep 12 12:35:22 smithi189 podman[75753]: 2024-09-12 12:35:22.994532152 +0000 UTC m=+0.612402823 container died 8c213e3935bb98cf4a647a009c0c2a8a067b1b738d5b76d8dc14f8ba40d2feb4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T12:35:23.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:22 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:22.828+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:23.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:22 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:22.828+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:23.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:22 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:22.828+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:23.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:22 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:22.828+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:23.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:22 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:22.828+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:23.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:23 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:23.112+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:23.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:23 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:23.112+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:23.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:23 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:23.112+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:23.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:23 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:23.112+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:23.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:23 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:23.112+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:23.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:23 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:23.112+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:23.636 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T12:35:23.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:23.675 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T12:35:23.675 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T12:35:23.675 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.9 2024-09-12T12:35:23.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:23 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:23.510+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:23.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:23 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:23.510+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:23.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:23 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:23.510+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:23.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:23 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:23.510+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:23.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:23 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:23.510+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:23.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:23 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:23.510+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:23.995 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 systemd[1]: Stopping Ceph osd.9 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:23.995 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.845+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:23.995 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.845+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:23.995 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.845+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:23.996 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.845+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:23.996 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.845+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:23.996 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.994+0000 7f6183d7e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:23.996 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.994+0000 7f6183d7e640 -1 osd.9 123 *** Got signal Terminated *** 2024-09-12T12:35:23.996 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:23 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:23.994+0000 7f6183d7e640 -1 osd.9 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:35:24.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:24 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:24.121+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:24.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:24 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:24.121+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:24.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:24 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:24.121+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:24.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:24 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:24.121+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:24.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:24 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:24.121+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:24.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:24 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:24.121+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:24.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:24.551+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:24.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:24.551+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:24.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:24.551+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:24.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:24.551+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:24.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:24.551+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:24.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:24.551+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:25.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:24 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:24.868+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:25.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:24 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:24.868+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:25.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:24 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:24.868+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:25.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:24 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:24.868+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:25.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:24 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:24.868+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:25.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:25 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:25.097+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:25.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:25 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:25.097+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:25.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:25 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:25.097+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:25.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:25 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:25.097+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:25.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:25 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:25.097+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:25.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:25 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:25.097+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:25.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:25 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:25.550+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:25.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:25 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:25.550+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:25.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:25 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:25.550+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:25.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:25 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:25.550+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:25.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:25 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:25.550+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:25.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:25 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:25.550+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:26.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:25 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:25.914+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:26.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:25 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:25.914+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:26.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:25 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:25.914+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:26.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:25 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:25.914+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:26.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:25 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:25.914+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:26.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:25 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:25.914+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:00.637832+0000 front 2024-09-12T12:35:00.637888+0000 (oldest deadline 2024-09-12T12:35:25.337974+0000) 2024-09-12T12:35:26.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:26 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:26.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:26.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:26 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:26.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:26.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:26 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:26.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:26.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:26 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:26.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:26.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:26 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:26.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:26.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:26 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:26.129+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:26.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:26.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:26.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:26.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:26.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:26.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:26.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:26.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:26.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:26.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:26.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:26 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:26.572+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:27.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:26.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:27.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:26.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:27.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:26.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:27.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:26.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:27.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:26.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:27.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:26 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:26.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:00.637832+0000 front 2024-09-12T12:35:00.637888+0000 (oldest deadline 2024-09-12T12:35:25.337974+0000) 2024-09-12T12:35:27.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:27 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:27.143+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:27.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:27 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:27.143+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:27.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:27 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:27.143+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:27.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:27 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:27.143+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:27.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:27 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:27.143+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:27.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:27 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:27.143+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:28.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:27.620+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:28.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:27.620+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:28.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:27.620+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:28.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:27.620+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:28.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:27.620+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:28.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:27 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:27.620+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:28.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:27.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:28.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:27.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:28.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:27.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:28.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:27.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:28.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:27.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:28.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:27 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:27.905+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:00.637832+0000 front 2024-09-12T12:35:00.637888+0000 (oldest deadline 2024-09-12T12:35:25.337974+0000) 2024-09-12T12:35:28.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:28 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:28.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:28.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:28 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:28.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:28.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:28 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:28.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:28.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:28 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:28.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:28.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:28 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:28.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:28.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:28 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:28.181+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:29.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:28.616+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:29.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:28.616+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:29.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:28.616+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:29.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:28.616+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:29.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:28.616+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:29.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:28.616+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:29.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:28 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:28.923+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:13.126930+0000 front 2024-09-12T12:34:13.126963+0000 (oldest deadline 2024-09-12T12:34:37.827127+0000) 2024-09-12T12:35:29.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:28 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:28.923+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.528437+0000 front 2024-09-12T12:34:22.528377+0000 (oldest deadline 2024-09-12T12:34:44.729187+0000) 2024-09-12T12:35:29.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:28 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:28.923+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:31.631901+0000 front 2024-09-12T12:34:31.631909+0000 (oldest deadline 2024-09-12T12:34:53.931914+0000) 2024-09-12T12:35:29.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:28 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:28.923+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:38.533623+0000 front 2024-09-12T12:34:38.533726+0000 (oldest deadline 2024-09-12T12:35:03.233854+0000) 2024-09-12T12:35:29.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:28 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:28.923+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.937605+0000 front 2024-09-12T12:34:55.937555+0000 (oldest deadline 2024-09-12T12:35:20.637504+0000) 2024-09-12T12:35:29.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:28 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9[61919]: 2024-09-12T12:35:28.923+0000 7f617c38c640 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:00.637832+0000 front 2024-09-12T12:35:00.637888+0000 (oldest deadline 2024-09-12T12:35:25.337974+0000) 2024-09-12T12:35:29.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:29 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:29.219+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:29.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:29 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:29.219+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:29.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:29 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:29.219+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:29.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:29 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:29.219+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:29.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:29 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:29.219+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:29.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:29 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:29.219+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:29.836 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:29 smithi137 podman[103573]: 2024-09-12 12:35:29.560941003 +0000 UTC m=+5.689570319 container died 4bec165e789a387b32454637a04127f4f7fa0379efdba6464ede90e78becf10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T12:35:30.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:29 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:29.600+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:30.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:29 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:29.600+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:30.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:29 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:29.600+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:30.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:29 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:29.600+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:30.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:29 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:29.600+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:30.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:29 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:29.600+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:30.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:30 smithi137 podman[103573]: 2024-09-12 12:35:30.018959434 +0000 UTC m=+6.147588738 container cleanup 4bec165e789a387b32454637a04127f4f7fa0379efdba6464ede90e78becf10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:30.299 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:30 smithi137 bash[103573]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9 2024-09-12T12:35:30.678 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:30 smithi137 podman[103588]: 2024-09-12 12:35:30.345349902 +0000 UTC m=+0.777716882 container remove 4bec165e789a387b32454637a04127f4f7fa0379efdba6464ede90e78becf10a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:30.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:30 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:30.215+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:30.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:30 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:30.215+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:30.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:30 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:30.215+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:30.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:30 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:30.215+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:30.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:30 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:30.215+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:30.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:30 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:30.215+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:30.939 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:30 smithi137 podman[103677]: 2024-09-12 12:35:30.581363836 +0000 UTC m=+0.015669574 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:35:30.940 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:30 smithi137 podman[103677]: 2024-09-12 12:35:30.704726116 +0000 UTC m=+0.139031888 container create fe07349bcad9d7ff18e18ee0268a835a6c960790814be150b2944ef24256ee6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T12:35:31.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:30.629+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:31.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:30.629+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:31.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:30.629+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:31.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:30.629+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:31.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:30.629+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:31.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:30 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:30.629+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:31.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:30 smithi137 podman[103677]: 2024-09-12 12:35:30.940316619 +0000 UTC m=+0.374622349 container init fe07349bcad9d7ff18e18ee0268a835a6c960790814be150b2944ef24256ee6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T12:35:31.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:30 smithi137 podman[103677]: 2024-09-12 12:35:30.943645685 +0000 UTC m=+0.377951416 container start fe07349bcad9d7ff18e18ee0268a835a6c960790814be150b2944ef24256ee6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:35:31.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:31 smithi137 podman[103677]: 2024-09-12 12:35:31.023532386 +0000 UTC m=+0.457838129 container attach fe07349bcad9d7ff18e18ee0268a835a6c960790814be150b2944ef24256ee6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T12:35:31.300 INFO:journalctl@ceph.osd.9.smithi137.stdout:Sep 12 12:35:31 smithi137 podman[103677]: 2024-09-12 12:35:31.133419285 +0000 UTC m=+0.567725023 container died fe07349bcad9d7ff18e18ee0268a835a6c960790814be150b2944ef24256ee6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T12:35:31.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:31.255+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:31.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:31.255+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:31.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:31.255+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:31.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:31.255+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:31.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:31.255+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:31.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:31.255+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:31.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:31.255+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:02.889586+0000 front 2024-09-12T12:35:02.889546+0000 (oldest deadline 2024-09-12T12:35:30.490066+0000) 2024-09-12T12:35:31.758 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T12:35:31.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:31.798 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T12:35:31.798 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T12:35:31.798 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.10 2024-09-12T12:35:32.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:31.632+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:32.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:31.632+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:32.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:31.632+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:32.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:31.632+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:32.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:31.632+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:32.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:31 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:31.632+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:32.101 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:31 smithi171 systemd[1]: Stopping Ceph osd.10 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:32.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.100+0000 7fd18a6c4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:32.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.100+0000 7fd18a6c4640 -1 osd.10 123 *** Got signal Terminated *** 2024-09-12T12:35:32.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.100+0000 7fd18a6c4640 -1 osd.10 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:35:32.450 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.294+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:32.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.294+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:32.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.294+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:32.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.294+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:32.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.294+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:32.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.294+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:32.451 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:32 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:32.294+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:02.889586+0000 front 2024-09-12T12:35:02.889546+0000 (oldest deadline 2024-09-12T12:35:30.490066+0000) 2024-09-12T12:35:33.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:32.657+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:33.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:32.657+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:33.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:32.657+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:33.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:32.657+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:33.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:32.657+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:33.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:32.657+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:33.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:33.283+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:33.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:33.283+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:33.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:33.283+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:33.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:33.283+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:33.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:33.283+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:33.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:33.283+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:33.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:33 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:33.283+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:02.889586+0000 front 2024-09-12T12:35:02.889546+0000 (oldest deadline 2024-09-12T12:35:30.490066+0000) 2024-09-12T12:35:34.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:33 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:33.680+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:34.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:33 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:33.680+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:34.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:33 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:33.680+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:34.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:33 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:33.680+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:34.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:33 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:33.680+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:34.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:33 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:33.680+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:34.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:33 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:33.680+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:34.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:34 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:34.310+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:34.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:34 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:34.310+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:34.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:34 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:34.310+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:34.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:34 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:34.310+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:34.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:34 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:34.310+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:34.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:34 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:34.310+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:34.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:34 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:34.310+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:02.889586+0000 front 2024-09-12T12:35:02.889546+0000 (oldest deadline 2024-09-12T12:35:30.490066+0000) 2024-09-12T12:35:35.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:34 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:34.711+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:35.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:34 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:34.711+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:35.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:34 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:34.711+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:35.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:34 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:34.711+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:35.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:34 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:34.711+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:35.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:34 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:34.711+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:35.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:34 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:34.711+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:35.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:35.342+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:35.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:35.342+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:35.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:35.342+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:35.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:35.342+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:35.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:35.342+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:35.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:35.342+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:35.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:35 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:35.342+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:02.889586+0000 front 2024-09-12T12:35:02.889546+0000 (oldest deadline 2024-09-12T12:35:30.490066+0000) 2024-09-12T12:35:36.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:35.715+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:36.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:35.715+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:36.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:35.715+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:36.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:35.715+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:36.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:35.715+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:36.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:35.715+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:36.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:35 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:35.715+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:36.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:36 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:36.317+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:15.080896+0000 front 2024-09-12T12:34:15.080912+0000 (oldest deadline 2024-09-12T12:34:38.581020+0000) 2024-09-12T12:35:36.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:36 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:36.317+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:18.581468+0000 front 2024-09-12T12:34:18.581495+0000 (oldest deadline 2024-09-12T12:34:44.481635+0000) 2024-09-12T12:35:36.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:36 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:36.317+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:30.382912+0000 front 2024-09-12T12:34:30.382938+0000 (oldest deadline 2024-09-12T12:34:55.083115+0000) 2024-09-12T12:35:36.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:36 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:36.317+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:39.784279+0000 front 2024-09-12T12:34:39.784335+0000 (oldest deadline 2024-09-12T12:35:02.084300+0000) 2024-09-12T12:35:36.700 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:36 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:36.317+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:46.586443+0000 front 2024-09-12T12:34:46.586397+0000 (oldest deadline 2024-09-12T12:35:09.486613+0000) 2024-09-12T12:35:36.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:36 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:36.317+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.387947+0000 front 2024-09-12T12:34:55.387875+0000 (oldest deadline 2024-09-12T12:35:22.388258+0000) 2024-09-12T12:35:36.701 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:36 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10[59348]: 2024-09-12T12:35:36.317+0000 7fd186cda640 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-09-12T12:35:02.889586+0000 front 2024-09-12T12:35:02.889546+0000 (oldest deadline 2024-09-12T12:35:30.490066+0000) 2024-09-12T12:35:36.992 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:36 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:36.733+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:36.993 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:36 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:36.733+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:36.993 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:36 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:36.733+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:36.993 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:36 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:36.733+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:36.993 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:36 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:36.733+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:36.993 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:36 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:36.733+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:36.993 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:36 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:36.733+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:37.869 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:37 smithi171 podman[73384]: 2024-09-12 12:35:37.606951938 +0000 UTC m=+5.613714504 container died d48240775ec884da71582cc010f0a934894d4789919c8e4c4033c9f1e798ce7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:38.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:37 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:37.696+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:38.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:37 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:37.696+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:38.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:37 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:37.696+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:38.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:37 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:37.696+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:38.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:37 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:37.696+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:38.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:37 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:37.696+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:38.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:37 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:37.696+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:38.200 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:37 smithi171 podman[73384]: 2024-09-12 12:35:37.967932488 +0000 UTC m=+5.974695059 container cleanup d48240775ec884da71582cc010f0a934894d4789919c8e4c4033c9f1e798ce7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T12:35:38.200 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:37 smithi171 bash[73384]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10 2024-09-12T12:35:38.569 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:38 smithi171 podman[73400]: 2024-09-12 12:35:38.244067589 +0000 UTC m=+0.633844321 container remove d48240775ec884da71582cc010f0a934894d4789919c8e4c4033c9f1e798ce7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:38.896 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:38 smithi171 podman[73489]: 2024-09-12 12:35:38.56993842 +0000 UTC m=+0.104193990 container create c0b5b57a55595bc70883e474806089cc6671df130b85f44cc9a19a310e41df16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-12T12:35:38.897 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:38 smithi171 podman[73489]: 2024-09-12 12:35:38.481507344 +0000 UTC m=+0.015762922 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:35:38.897 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:38 smithi171 podman[73489]: 2024-09-12 12:35:38.81376145 +0000 UTC m=+0.348017021 container init c0b5b57a55595bc70883e474806089cc6671df130b85f44cc9a19a310e41df16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T12:35:38.897 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:38 smithi171 podman[73489]: 2024-09-12 12:35:38.81717086 +0000 UTC m=+0.351426434 container start c0b5b57a55595bc70883e474806089cc6671df130b85f44cc9a19a310e41df16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T12:35:38.897 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:38 smithi171 podman[73489]: 2024-09-12 12:35:38.897035562 +0000 UTC m=+0.431291144 container attach c0b5b57a55595bc70883e474806089cc6671df130b85f44cc9a19a310e41df16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T12:35:39.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:38.725+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:39.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:38.725+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:39.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:38.725+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:39.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:38.725+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:39.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:38.725+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:39.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:38.725+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:39.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:38 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:38.725+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:39.200 INFO:journalctl@ceph.osd.10.smithi171.stdout:Sep 12 12:35:39 smithi171 podman[73489]: 2024-09-12 12:35:39.007612808 +0000 UTC m=+0.541868379 container died c0b5b57a55595bc70883e474806089cc6671df130b85f44cc9a19a310e41df16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:35:39.588 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T12:35:39.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:39.628 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T12:35:39.628 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T12:35:39.628 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.11 2024-09-12T12:35:40.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 systemd[1]: Stopping Ceph osd.11 for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:40.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:39.710+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:40.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:39.710+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:40.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:39.710+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:40.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:39.710+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:40.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:39.710+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:40.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:39.710+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:40.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:39 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:39.710+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:40.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.172+0000 7f918cfd5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:40.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.172+0000 7f918cfd5640 -1 osd.11 123 *** Got signal Terminated *** 2024-09-12T12:35:40.591 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.172+0000 7f918cfd5640 -1 osd.11 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T12:35:41.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.692+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:41.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.692+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:41.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.692+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:41.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.692+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:41.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.692+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:41.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.692+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:41.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:40 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:40.692+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:42.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:41.660+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:42.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:41.660+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:42.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:41.660+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:42.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:41.660+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:42.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:41.660+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:42.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:41.660+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:42.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:41 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:41.660+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:43.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:42.643+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:43.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:42.643+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:43.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:42.643+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:43.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:42.643+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:43.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:42.643+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:43.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:42.643+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:43.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:42 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:42.643+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:44.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:43.601+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:44.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:43.601+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:44.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:43.601+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:44.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:43.601+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:44.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:43.601+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:44.091 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:43.601+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:44.092 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:43 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:43.601+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:44.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-09-12T12:34:09.960992+0000 front 2024-09-12T12:34:09.961048+0000 (oldest deadline 2024-09-12T12:34:41.161600+0000) 2024-09-12T12:35:44.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6804 osd.1 since back 2024-09-12T12:34:22.863094+0000 front 2024-09-12T12:34:22.863108+0000 (oldest deadline 2024-09-12T12:34:48.763109+0000) 2024-09-12T12:35:44.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-09-12T12:34:28.763526+0000 front 2024-09-12T12:34:28.763628+0000 (oldest deadline 2024-09-12T12:34:52.863638+0000) 2024-09-12T12:35:44.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-09-12T12:34:40.266529+0000 front 2024-09-12T12:34:40.266647+0000 (oldest deadline 2024-09-12T12:35:01.966633+0000) 2024-09-12T12:35:44.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6812 osd.4 since back 2024-09-12T12:34:48.269050+0000 front 2024-09-12T12:34:48.268981+0000 (oldest deadline 2024-09-12T12:35:11.769030+0000) 2024-09-12T12:35:44.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-09-12T12:34:55.770652+0000 front 2024-09-12T12:34:55.770524+0000 (oldest deadline 2024-09-12T12:35:21.670713+0000) 2024-09-12T12:35:44.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.171:6820 osd.7 since back 2024-09-12T12:35:07.472269+0000 front 2024-09-12T12:35:07.472164+0000 (oldest deadline 2024-09-12T12:35:33.372125+0000) 2024-09-12T12:35:44.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:44 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11[61008]: 2024-09-12T12:35:44.566+0000 7f91895eb640 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.189:6820 osd.8 since back 2024-09-12T12:35:19.272788+0000 front 2024-09-12T12:35:19.272750+0000 (oldest deadline 2024-09-12T12:35:44.473323+0000) 2024-09-12T12:35:45.901 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:45 smithi189 podman[75858]: 2024-09-12 12:35:45.611940089 +0000 UTC m=+5.786634166 container died da7e9f24536c7eed214cec3724bab590cde112ee12f69b4ae520cf326aa307b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T12:35:46.230 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:45 smithi189 podman[75858]: 2024-09-12 12:35:45.971834498 +0000 UTC m=+6.146528575 container cleanup da7e9f24536c7eed214cec3724bab590cde112ee12f69b4ae520cf326aa307b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T12:35:46.231 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:45 smithi189 bash[75858]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11 2024-09-12T12:35:46.588 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:46 smithi189 podman[75873]: 2024-09-12 12:35:46.231158599 +0000 UTC m=+0.619246694 container remove da7e9f24536c7eed214cec3724bab590cde112ee12f69b4ae520cf326aa307b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:46.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:46 smithi189 podman[75961]: 2024-09-12 12:35:46.49193882 +0000 UTC m=+0.015905398 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T12:35:46.841 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:46 smithi189 podman[75961]: 2024-09-12 12:35:46.598645363 +0000 UTC m=+0.122611967 container create 35833d4892bda5b760e0000149a6c084b27058632a6940a8849bf742c00478c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-12T12:35:46.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:46 smithi189 podman[75961]: 2024-09-12 12:35:46.809294883 +0000 UTC m=+0.333261462 container init 35833d4892bda5b760e0000149a6c084b27058632a6940a8849bf742c00478c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:35:46.842 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:46 smithi189 podman[75961]: 2024-09-12 12:35:46.812845456 +0000 UTC m=+0.336812015 container start 35833d4892bda5b760e0000149a6c084b27058632a6940a8849bf742c00478c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T12:35:47.294 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:46 smithi189 podman[75961]: 2024-09-12 12:35:46.892247585 +0000 UTC m=+0.416214144 container attach 35833d4892bda5b760e0000149a6c084b27058632a6940a8849bf742c00478c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-12T12:35:47.294 INFO:journalctl@ceph.osd.11.smithi189.stdout:Sep 12 12:35:47 smithi189 podman[75961]: 2024-09-12 12:35:47.002347576 +0000 UTC m=+0.526314147 container died 35833d4892bda5b760e0000149a6c084b27058632a6940a8849bf742c00478c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T12:35:47.617 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T12:35:47.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:47.656 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T12:35:47.657 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T12:35:47.657 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.a 2024-09-12T12:35:47.953 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:35:47 smithi137 systemd[1]: Stopping Ceph mds.a for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:48.217 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:35:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a[69166]: 2024-09-12T12:35:47.953+0000 7f22306b5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:48.218 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:35:47 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a[69166]: 2024-09-12T12:35:47.953+0000 7f22306b5640 -1 mds.a *** got signal Terminated *** 2024-09-12T12:35:51.619 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:35:51 smithi137 podman[103782]: 2024-09-12 12:35:51.313198477 +0000 UTC m=+3.465866211 container died 4db1b9d79bd2ea269b1b58bd1efd1c46f86c5821cc95b80c267a79a85643cbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T12:35:51.620 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:35:51 smithi137 podman[103782]: 2024-09-12 12:35:51.619704238 +0000 UTC m=+3.772371979 container cleanup 4db1b9d79bd2ea269b1b58bd1efd1c46f86c5821cc95b80c267a79a85643cbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T12:35:52.033 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:35:51 smithi137 bash[103782]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a 2024-09-12T12:35:52.034 INFO:journalctl@ceph.mds.a.smithi137.stdout:Sep 12 12:35:51 smithi137 podman[103806]: 2024-09-12 12:35:51.829307616 +0000 UTC m=+0.507994355 container remove 4db1b9d79bd2ea269b1b58bd1efd1c46f86c5821cc95b80c267a79a85643cbbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-12T12:35:52.052 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T12:35:52.087 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:52.087 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T12:35:52.088 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T12:35:52.088 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.d 2024-09-12T12:35:52.516 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:52 smithi137 systemd[1]: Stopping Ceph mds.d for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:52.799 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:52 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d[67975]: 2024-09-12T12:35:52.516+0000 7fcb81ce7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:52.799 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:52 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d[67975]: 2024-09-12T12:35:52.516+0000 7fcb81ce7640 -1 mds.d *** got signal Terminated *** 2024-09-12T12:35:56.299 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:56 smithi137 podman[103915]: 2024-09-12 12:35:56.011964178 +0000 UTC m=+3.698805946 container died 4d9bdea95fb4794a1b531e3f4bd7b15719902d222063a5c46488cc286f6f10b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T12:35:56.574 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:56 smithi137 podman[103915]: 2024-09-12 12:35:56.389800183 +0000 UTC m=+4.076641921 container cleanup 4d9bdea95fb4794a1b531e3f4bd7b15719902d222063a5c46488cc286f6f10b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T12:35:56.574 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:56 smithi137 bash[103915]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d 2024-09-12T12:35:56.816 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T12:35:56.838 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:56 smithi137 podman[103927]: 2024-09-12 12:35:56.574323075 +0000 UTC m=+0.560502272 container remove 4d9bdea95fb4794a1b531e3f4bd7b15719902d222063a5c46488cc286f6f10b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T12:35:56.838 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:56 smithi137 systemd[1]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-12T12:35:56.838 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:56 smithi137 systemd[1]: Stopped Ceph mds.d for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:35:56.838 INFO:journalctl@ceph.mds.d.smithi137.stdout:Sep 12 12:35:56 smithi137 systemd[1]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.d.service: Consumed 1.141s CPU time. 2024-09-12T12:35:56.850 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:35:56.850 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T12:35:56.851 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T12:35:56.851 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.g 2024-09-12T12:35:57.177 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:35:56 smithi137 systemd[1]: Stopping Ceph mds.g for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:35:57.549 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:35:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g[68782]: 2024-09-12T12:35:57.176+0000 7fa71be32640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:35:57.549 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:35:57 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g[68782]: 2024-09-12T12:35:57.176+0000 7fa71be32640 -1 mds.g *** got signal Terminated *** 2024-09-12T12:36:00.085 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:35:59 smithi137 podman[104035]: 2024-09-12 12:35:59.808682565 +0000 UTC m=+2.747704114 container died 04bed9820499b68a47d5e6fa6d77ad74e63e3bc5f7d134bcf162495a564f4409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T12:36:00.461 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:36:00 smithi137 podman[104035]: 2024-09-12 12:36:00.185419501 +0000 UTC m=+3.124441045 container cleanup 04bed9820499b68a47d5e6fa6d77ad74e63e3bc5f7d134bcf162495a564f4409 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T12:36:00.461 INFO:journalctl@ceph.mds.g.smithi137.stdout:Sep 12 12:36:00 smithi137 bash[104035]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-g 2024-09-12T12:36:00.621 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T12:36:00.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:00.655 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T12:36:00.655 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T12:36:00.656 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.j 2024-09-12T12:36:00.980 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:36:00 smithi137 systemd[1]: Stopping Ceph mds.j for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:01.299 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:36:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j[67598]: 2024-09-12T12:36:00.980+0000 7f9c119da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:01.299 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:36:00 smithi137 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j[67598]: 2024-09-12T12:36:00.980+0000 7f9c119da640 -1 mds.j *** got signal Terminated *** 2024-09-12T12:36:04.698 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:36:04 smithi137 podman[104156]: 2024-09-12 12:36:04.371783828 +0000 UTC m=+3.498738754 container died 19c51bf267dd5acb66926786cf7beceffe311aa04fdc75c32780cccbc9e0764a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T12:36:04.973 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:36:04 smithi137 podman[104156]: 2024-09-12 12:36:04.806015105 +0000 UTC m=+3.932970028 container cleanup 19c51bf267dd5acb66926786cf7beceffe311aa04fdc75c32780cccbc9e0764a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T12:36:04.973 INFO:journalctl@ceph.mds.j.smithi137.stdout:Sep 12 12:36:04 smithi137 bash[104156]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-j 2024-09-12T12:36:05.168 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T12:36:05.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:05.201 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T12:36:05.202 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T12:36:05.202 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.b 2024-09-12T12:36:05.575 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:36:05 smithi171 systemd[1]: Stopping Ceph mds.b for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:05.949 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:36:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: 2024-09-12T12:36:05.574+0000 7f6159c8c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:05.950 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:36:05 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b[64958]: 2024-09-12T12:36:05.574+0000 7f6159c8c640 -1 mds.b *** got signal Terminated *** 2024-09-12T12:36:07.851 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:36:07 smithi171 podman[73594]: 2024-09-12 12:36:07.588654255 +0000 UTC m=+2.189664260 container died be8aa567e0a4f938c8c4cecb1279f2760f5ba5fe1cd0f537c19c59cab456f6f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T12:36:08.200 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:36:07 smithi171 podman[73594]: 2024-09-12 12:36:07.95801672 +0000 UTC m=+2.559026724 container cleanup be8aa567e0a4f938c8c4cecb1279f2760f5ba5fe1cd0f537c19c59cab456f6f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T12:36:08.200 INFO:journalctl@ceph.mds.b.smithi171.stdout:Sep 12 12:36:07 smithi171 bash[73594]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-b 2024-09-12T12:36:08.404 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T12:36:08.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:08.441 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T12:36:08.441 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T12:36:08.442 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.e 2024-09-12T12:36:08.772 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:36:08 smithi171 systemd[1]: Stopping Ceph mds.e for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:09.034 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:36:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e[64345]: 2024-09-12T12:36:08.772+0000 7f7f53fe0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:09.035 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:36:08 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e[64345]: 2024-09-12T12:36:08.772+0000 7f7f53fe0640 -1 mds.e *** got signal Terminated *** 2024-09-12T12:36:11.123 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:36:10 smithi171 podman[73725]: 2024-09-12 12:36:10.72353859 +0000 UTC m=+2.061907189 container died 204a2a4d658528ef1229ed0c239515d9b5bcf20f5bb7b4aee8d3bff136263e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T12:36:11.441 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:36:11 smithi171 podman[73725]: 2024-09-12 12:36:11.223151304 +0000 UTC m=+2.561519904 container cleanup 204a2a4d658528ef1229ed0c239515d9b5bcf20f5bb7b4aee8d3bff136263e63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T12:36:11.441 INFO:journalctl@ceph.mds.e.smithi171.stdout:Sep 12 12:36:11 smithi171 bash[73725]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-e 2024-09-12T12:36:11.568 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T12:36:11.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:11.601 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T12:36:11.602 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T12:36:11.602 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.h 2024-09-12T12:36:11.700 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:36:11 smithi171 systemd[1]: Stopping Ceph mds.h for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:12.200 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:36:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h[64643]: 2024-09-12T12:36:11.935+0000 7f0e2502e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:12.200 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:36:11 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h[64643]: 2024-09-12T12:36:11.935+0000 7f0e2502e640 -1 mds.h *** got signal Terminated *** 2024-09-12T12:36:15.937 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:36:15 smithi171 podman[73847]: 2024-09-12 12:36:15.584119137 +0000 UTC m=+3.763666207 container died 143f0538919948cb8d5c2efc487745c7955a7c8560ef455b395ac8aefd016025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-12T12:36:16.201 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:36:16 smithi171 podman[73847]: 2024-09-12 12:36:16.020040025 +0000 UTC m=+4.199570662 container cleanup 143f0538919948cb8d5c2efc487745c7955a7c8560ef455b395ac8aefd016025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-12T12:36:16.201 INFO:journalctl@ceph.mds.h.smithi171.stdout:Sep 12 12:36:16 smithi171 bash[73847]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-h 2024-09-12T12:36:16.610 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T12:36:16.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:16.644 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T12:36:16.644 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T12:36:16.644 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.k 2024-09-12T12:36:17.200 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:36:16 smithi171 systemd[1]: Stopping Ceph mds.k for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:17.200 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:36:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k[64035]: 2024-09-12T12:36:16.948+0000 7f8d79eb6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:17.200 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:36:16 smithi171 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k[64035]: 2024-09-12T12:36:16.948+0000 7f8d79eb6640 -1 mds.k *** got signal Terminated *** 2024-09-12T12:36:21.201 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:36:20 smithi171 podman[73969]: 2024-09-12 12:36:20.91432236 +0000 UTC m=+4.056851032 container died dd5bf0673ac56a852eaedb63d13a40e6f1d4352b687e0ff431af370675d48d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:36:21.659 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:36:21 smithi171 podman[73969]: 2024-09-12 12:36:21.36685644 +0000 UTC m=+4.509385101 container cleanup dd5bf0673ac56a852eaedb63d13a40e6f1d4352b687e0ff431af370675d48d9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T12:36:21.660 INFO:journalctl@ceph.mds.k.smithi171.stdout:Sep 12 12:36:21 smithi171 bash[73969]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-k 2024-09-12T12:36:21.791 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T12:36:21.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:21.823 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T12:36:21.824 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T12:36:21.824 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.c 2024-09-12T12:36:22.203 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:36:21 smithi189 systemd[1]: Stopping Ceph mds.c for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:22.458 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:36:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c[65837]: 2024-09-12T12:36:22.203+0000 7f6a4a1ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:22.458 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:36:22 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c[65837]: 2024-09-12T12:36:22.203+0000 7f6a4a1ef640 -1 mds.c *** got signal Terminated *** 2024-09-12T12:36:23.841 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:36:23 smithi189 podman[76066]: 2024-09-12 12:36:23.566444772 +0000 UTC m=+1.555024813 container died 794fd1523170b41d17651c0fb6d273c6626d54ea027bd025db0e4486df28438b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T12:36:24.186 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:36:23 smithi189 podman[76066]: 2024-09-12 12:36:23.936198623 +0000 UTC m=+1.924778664 container cleanup 794fd1523170b41d17651c0fb6d273c6626d54ea027bd025db0e4486df28438b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T12:36:24.187 INFO:journalctl@ceph.mds.c.smithi189.stdout:Sep 12 12:36:23 smithi189 bash[76066]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-c 2024-09-12T12:36:24.317 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T12:36:24.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:24.491 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T12:36:24.491 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T12:36:24.491 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.f 2024-09-12T12:36:24.815 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:36:24 smithi189 systemd[1]: Stopping Ceph mds.f for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:25.091 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:36:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: 2024-09-12T12:36:24.814+0000 7f282e4d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:25.091 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:36:24 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f[65536]: 2024-09-12T12:36:24.814+0000 7f282e4d2640 -1 mds.f *** got signal Terminated *** 2024-09-12T12:36:27.841 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:36:27 smithi189 podman[76200]: 2024-09-12 12:36:27.563240566 +0000 UTC m=+2.885550343 container died 621c43d9b50d547e2fe40dfeffe7df7efea64ff3df542c64ddb0e277761aed3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:36:28.190 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:36:27 smithi189 podman[76200]: 2024-09-12 12:36:27.914834143 +0000 UTC m=+3.237143922 container cleanup 621c43d9b50d547e2fe40dfeffe7df7efea64ff3df542c64ddb0e277761aed3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T12:36:28.190 INFO:journalctl@ceph.mds.f.smithi189.stdout:Sep 12 12:36:27 smithi189 bash[76200]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-f 2024-09-12T12:36:28.346 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T12:36:28.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:28.381 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T12:36:28.382 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T12:36:28.382 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.i 2024-09-12T12:36:28.702 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:36:28 smithi189 systemd[1]: Stopping Ceph mds.i for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:29.090 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:36:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i[66137]: 2024-09-12T12:36:28.701+0000 7f6d70766640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:29.091 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:36:28 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i[66137]: 2024-09-12T12:36:28.701+0000 7f6d70766640 -1 mds.i *** got signal Terminated *** 2024-09-12T12:36:31.841 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:36:31 smithi189 podman[76321]: 2024-09-12 12:36:31.559861779 +0000 UTC m=+2.958245289 container died 29cdd59e7968169ddfc0a0690667049e8912ecb7cb312ea693bab6df472d9a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T12:36:32.202 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:36:31 smithi189 podman[76321]: 2024-09-12 12:36:31.94289818 +0000 UTC m=+3.341281679 container cleanup 29cdd59e7968169ddfc0a0690667049e8912ecb7cb312ea693bab6df472d9a08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T12:36:32.202 INFO:journalctl@ceph.mds.i.smithi189.stdout:Sep 12 12:36:31 smithi189 bash[76321]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-i 2024-09-12T12:36:32.377 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T12:36:32.410 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:32.411 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T12:36:32.411 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T12:36:32.411 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.l 2024-09-12T12:36:32.729 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:32 smithi189 systemd[1]: Stopping Ceph mds.l for dd66268a-70fe-11ef-bcea-c7b262605968... 2024-09-12T12:36:33.091 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l[65234]: 2024-09-12T12:36:32.729+0000 7f3b3a006640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T12:36:33.091 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:32 smithi189 ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l[65234]: 2024-09-12T12:36:32.729+0000 7f3b3a006640 -1 mds.l *** got signal Terminated *** 2024-09-12T12:36:35.822 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:35 smithi189 podman[76443]: 2024-09-12 12:36:35.565250325 +0000 UTC m=+2.934307367 container died f5d5c33814cce466709e5d6312e76167eb86b59da803a3838713b44781a70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T12:36:36.080 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:35 smithi189 podman[76443]: 2024-09-12 12:36:35.896332307 +0000 UTC m=+3.265389351 container cleanup f5d5c33814cce466709e5d6312e76167eb86b59da803a3838713b44781a70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T12:36:36.080 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:35 smithi189 bash[76443]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l 2024-09-12T12:36:36.324 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T12:36:36.340 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:36 smithi189 podman[76455]: 2024-09-12 12:36:36.080669854 +0000 UTC m=+0.511157290 container remove f5d5c33814cce466709e5d6312e76167eb86b59da803a3838713b44781a70588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-dd66268a-70fe-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T12:36:36.341 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:36 smithi189 systemd[1]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-12T12:36:36.341 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:36 smithi189 systemd[1]: Stopped Ceph mds.l for dd66268a-70fe-11ef-bcea-c7b262605968. 2024-09-12T12:36:36.341 INFO:journalctl@ceph.mds.l.smithi189.stdout:Sep 12 12:36:36 smithi189 systemd[1]: ceph-dd66268a-70fe-11ef-bcea-c7b262605968@mds.l.service: Consumed 29.235s CPU time. 2024-09-12T12:36:36.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T12:36:36.357 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T12:36:36.358 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dd66268a-70fe-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T12:36:36.559 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:36:39.038 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dd66268a-70fe-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T12:36:39.247 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:36:41.828 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dd66268a-70fe-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T12:36:42.030 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:36:44.489 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:36:44.523 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:36:44.557 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T12:36:44.584 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T12:36:44.586 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi137/crash 2024-09-12T12:36:44.587 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash -- . 2024-09-12T12:36:44.632 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T12:36:44.632 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-09-12T12:36:44.634 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi171/crash 2024-09-12T12:36:44.635 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash -- . 2024-09-12T12:36:44.678 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T12:36:44.678 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-09-12T12:36:44.680 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi189/crash 2024-09-12T12:36:44.681 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash -- . 2024-09-12T12:36:44.723 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T12:36:44.723 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-09-12T12:36:44.723 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T12:36:44.724 DEBUG:teuthology.orchestra.run.smithi137:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-12T12:36:44.762 INFO:tasks.cephadm:Compressing logs... 2024-09-12T12:36:44.762 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T12:36:44.805 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T12:36:44.808 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T12:36:44.831 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T12:36:44.831 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T12:36:44.832 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T12:36:44.832 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T12:36:44.832 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T12:36:44.834 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T12:36:44.835 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T12:36:44.835 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T12:36:44.835 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T12:36:44.835 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.a.log: 85.0% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log.gzgzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T12:36:44.835 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-09-12T12:36:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.g.log: 90.4% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T12:36:44.838 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log 2024-09-12T12:36:44.839 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T12:36:44.842 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T12:36:44.842 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-09-12T12:36:44.842 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T12:36:44.842 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T12:36:44.842 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T12:36:44.842 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.b.log: /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T12:36:44.843 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T12:36:44.844 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T12:36:44.844 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T12:36:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log: 94.7% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T12:36:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T12:36:44.848 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T12:36:44.849 INFO:teuthology.orchestra.run.smithi137.stderr: 87.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T12:36:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T12:36:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T12:36:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.j.log: 91.0% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T12:36:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T12:36:44.849 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.7% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T12:36:44.850 INFO:teuthology.orchestra.run.smithi137.stderr: 94.7% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T12:36:44.852 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T12:36:44.852 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T12:36:44.853 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T12:36:44.853 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T12:36:44.853 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T12:36:44.853 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T12:36:44.853 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T12:36:44.853 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T12:36:44.853 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T12:36:44.854 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log 2024-09-12T12:36:44.854 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T12:36:44.854 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T12:36:44.854 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T12:36:44.856 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T12:36:44.857 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T12:36:44.857 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T12:36:44.860 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/cephadm.log: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T12:36:44.895 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.9% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T12:36:44.905 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 82.9% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T12:36:44.988 INFO:teuthology.orchestra.run.smithi171.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T12:36:44.992 INFO:teuthology.orchestra.run.smithi171.stderr: 94.6% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T12:36:45.028 INFO:teuthology.orchestra.run.smithi171.stderr: 90.5% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T12:36:45.034 INFO:teuthology.orchestra.run.smithi189.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T12:36:45.085 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T12:36:45.087 INFO:teuthology.orchestra.run.smithi189.stderr: 91.0% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T12:36:45.125 INFO:teuthology.orchestra.run.smithi171.stderr: 87.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T12:36:45.294 INFO:teuthology.orchestra.run.smithi171.stderr: 92.4% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T12:36:45.342 INFO:teuthology.orchestra.run.smithi137.stderr: 90.2% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T12:36:45.363 INFO:teuthology.orchestra.run.smithi171.stderr: 94.5% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T12:36:45.414 INFO:teuthology.orchestra.run.smithi189.stderr: 87.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T12:36:45.420 INFO:teuthology.orchestra.run.smithi189.stderr: 90.6% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T12:36:45.524 INFO:teuthology.orchestra.run.smithi189.stderr: 92.5% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T12:36:47.864 INFO:teuthology.orchestra.run.smithi137.stderr: 92.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T12:36:48.448 INFO:teuthology.orchestra.run.smithi189.stderr: 92.0% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T12:36:48.831 INFO:teuthology.orchestra.run.smithi171.stderr: 91.9% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T12:37:10.467 INFO:teuthology.orchestra.run.smithi137.stderr: 91.2% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T12:37:13.976 INFO:teuthology.orchestra.run.smithi137.stderr: 91.6% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T12:37:15.754 INFO:teuthology.orchestra.run.smithi137.stderr: 91.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T12:37:19.934 INFO:teuthology.orchestra.run.smithi137.stderr: 91.2% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T12:37:19.936 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-09-12T12:37:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m35.116s 2024-09-12T12:37:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:user 2m0.247s 2024-09-12T12:37:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m3.916s 2024-09-12T12:38:06.879 INFO:teuthology.orchestra.run.smithi189.stderr: 95.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T12:38:22.382 INFO:teuthology.orchestra.run.smithi189.stderr: 95.3% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T12:38:33.485 INFO:teuthology.orchestra.run.smithi189.stderr: 91.4% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T12:38:40.454 INFO:teuthology.orchestra.run.smithi189.stderr: 90.9% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T12:39:13.162 INFO:teuthology.orchestra.run.smithi189.stderr: 91.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T12:39:47.239 INFO:teuthology.orchestra.run.smithi171.stderr: 94.9% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T12:40:00.474 INFO:teuthology.orchestra.run.smithi171.stderr: 91.2% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T12:40:28.017 INFO:teuthology.orchestra.run.smithi171.stderr: 91.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T12:40:36.288 INFO:teuthology.orchestra.run.smithi171.stderr: 91.5% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T12:40:44.872 INFO:teuthology.orchestra.run.smithi189.stderr: 91.1% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T12:41:32.865 INFO:teuthology.orchestra.run.smithi171.stderr: 91.2% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T12:44:40.478 INFO:teuthology.orchestra.run.smithi171.stderr: 93.4% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T12:44:40.480 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-12T12:44:40.480 INFO:teuthology.orchestra.run.smithi171.stderr:real 7m55.657s 2024-09-12T12:44:40.481 INFO:teuthology.orchestra.run.smithi171.stderr:user 5m31.663s 2024-09-12T12:44:40.481 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m13.157s 2024-09-12T12:47:59.149 INFO:teuthology.orchestra.run.smithi189.stderr: 93.2% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T12:48:21.576 INFO:teuthology.orchestra.run.smithi189.stderr: 93.4% -- replaced with /var/log/ceph/dd66268a-70fe-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T12:48:21.578 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-12T12:48:21.579 INFO:teuthology.orchestra.run.smithi189.stderr:real 11m36.755s 2024-09-12T12:48:21.579 INFO:teuthology.orchestra.run.smithi189.stderr:user 7m12.774s 2024-09-12T12:48:21.579 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m18.658s 2024-09-12T12:48:21.580 INFO:tasks.cephadm:Archiving logs... 2024-09-12T12:48:21.580 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi137/log 2024-09-12T12:48:21.582 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T12:48:41.968 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi171/log 2024-09-12T12:48:41.969 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T12:49:37.875 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi189/log 2024-09-12T12:49:37.879 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T12:50:49.716 INFO:tasks.cephadm:Removing cluster... 2024-09-12T12:50:49.737 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dd66268a-70fe-11ef-bcea-c7b262605968 --force 2024-09-12T12:50:49.939 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:50:50.387 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dd66268a-70fe-11ef-bcea-c7b262605968 --force 2024-09-12T12:50:50.602 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:50:51.221 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dd66268a-70fe-11ef-bcea-c7b262605968 --force 2024-09-12T12:50:51.551 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: dd66268a-70fe-11ef-bcea-c7b262605968 2024-09-12T12:50:52.243 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T12:50:52.244 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T12:50:52.263 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T12:50:52.283 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T12:50:52.299 INFO:tasks.cephadm:Teardown complete 2024-09-12T12:50:52.300 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T12:50:52.312 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T12:50:52.312 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T12:50:52.315 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T12:50:52.325 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T12:50:52.454 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T12:50:52.466 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T12:50:52.466 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:50:52.469 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:50:52.472 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T12:50:52.484 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:50:52.486 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:50:52.487 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-12T12:50:52.490 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:50:52.490 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-09-12T12:50:52.490 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 -1417us[-1411us] +/- 64ms 2024-09-12T12:50:52.490 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 +951us[ +957us] +/- 75ms 2024-09-12T12:50:52.490 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +752us[ +758us] +/- 76ms 2024-09-12T12:50:52.490 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:50:52.492 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:50:52.492 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-12T12:50:52.492 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 52 -1389us[-1369us] +/- 64ms 2024-09-12T12:50:52.492 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 +984us[ +984us] +/- 75ms 2024-09-12T12:50:52.493 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +773us[ +773us] +/- 76ms 2024-09-12T12:50:52.493 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:50:52.511 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T12:50:52.512 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-12T12:50:52.512 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 -1430us[-1437us] +/- 64ms 2024-09-12T12:50:52.512 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 +931us[ +924us] +/- 74ms 2024-09-12T12:50:52.512 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 +712us[ +705us] +/- 76ms 2024-09-12T12:50:52.512 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T12:50:52.513 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T12:50:52.524 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T12:50:52.524 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T12:50:52.538 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T12:50:52.624 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T12:50:52.727 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T12:50:52.846 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T12:50:52.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:50:52.879 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T12:50:52.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:50:52.912 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T12:50:52.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:50:52.940 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T12:50:52.951 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T12:50:52.963 INFO:teuthology.task.internal:Duration was 3280.715266 seconds 2024-09-12T12:50:52.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T12:50:52.974 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T12:50:52.975 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T12:50:52.977 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T12:50:52.980 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T12:50:53.017 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T12:50:53.046 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T12:50:53.047 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T12:50:53.393 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T12:50:53.394 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-09-12T12:50:53.394 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T12:50:53.416 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-09-12T12:50:53.416 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T12:50:53.440 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-12T12:50:53.440 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T12:50:53.472 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T12:50:53.473 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T12:50:53.475 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T12:50:53.483 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T12:50:53.541 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T12:50:53.542 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T12:50:53.544 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T12:50:53.551 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T12:50:54.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T12:50:54.381 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T12:50:54.381 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T12:50:54.408 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T12:50:54.437 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T12:50:54.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T12:50:54.481 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:50:54.484 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:50:54.486 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:50:54.505 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-09-12T12:50:54.510 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-09-12T12:50:54.542 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-12T12:50:54.566 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:50:54.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:50:54.591 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:50:54.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:50:54.607 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T12:50:54.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T12:50:54.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T12:50:54.643 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T12:50:54.643 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi137 2024-09-12T12:50:54.644 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T12:50:54.702 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi171 2024-09-12T12:50:54.702 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T12:50:54.740 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901698/remote/smithi189 2024-09-12T12:50:54.740 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T12:50:54.779 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T12:50:54.780 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T12:50:54.782 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T12:50:54.785 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T12:50:54.830 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T12:50:54.843 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T12:50:54.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T12:50:54.862 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T12:50:54.862 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T12:50:54.865 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T12:50:54.867 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T12:50:54.877 INFO:teuthology.orchestra.run.smithi137.stdout: 265933 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 12:50 /home/ubuntu/cephtest 2024-09-12T12:50:54.880 INFO:teuthology.orchestra.run.smithi171.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 12:50 /home/ubuntu/cephtest 2024-09-12T12:50:54.886 INFO:teuthology.orchestra.run.smithi189.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 12:50 /home/ubuntu/cephtest 2024-09-12T12:50:54.888 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T12:50:54.902 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T12:50:54.932 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 3280.7152659893036 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi137 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=05658077be8f42f6a4aafa817f1afdc8 status: fail success: false 2024-09-12T12:50:54.932 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T12:50:55.023 INFO:teuthology.run:FAIL